2024-03-07T07:18:35.837 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-07T07:18:35.837 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/leonidus-2024-03-07_05:46:12-fs-wip-lusov-qdb-distro-default-smithi/7583972 --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/suites/fsync-tester}} --name leonidus-2024-03-07_05:46:12-fs-wip-lusov-qdb-distro-default-smithi --owner scheduled_leonidus@teuthology --verbose -- /home/teuthworker/archive/leonidus-2024-03-07_05:46:12-fs-wip-lusov-qdb-distro-default-smithi/7583972/orig.config.yaml 2024-03-07T07:18:35.890 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-07T07:18:35.975 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-03-07_05:46:12-fs-wip-lusov-qdb-distro-default-smithi/7583972 branch: wip-lusov-qdb description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/suites/fsync-tester}} email: null first_in_suite: false job_id: '7583972' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: leonidus-2024-03-07_05:46:12-fs-wip-lusov-qdb-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-qdb ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 8b896a9e145796119f0451201dc4d53ddac97db0 subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt dry_run: true install: ceph: flavor: default sha1: 8b896a9e145796119f0451201dc4d53ddac97db0 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-qdb sha1: 8b896a9e145796119f0451201dc4d53ddac97db0 owner: scheduled_leonidus@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 9221 sha1: 8b896a9e145796119f0451201dc4d53ddac97db0 sleep_before_teardown: 0 subset: 111/10000 suite: fs suite_branch: wip-lusov-qdb suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_8b896a9e145796119f0451201dc4d53ddac97db0/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 8b896a9e145796119f0451201dc4d53ddac97db0 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi160.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEnyQs9uQLXfZEoZ2Tm1Acn3ntgsNJxul0I9jisv5u1TsPGeb7Dy8C3PsCPRw/v5VgTOSA98m6Qu0PVkzhvgwL8= smithi177.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNSrXChjdXzc1UOt8ymKReExe1A2gYLoXHB15wWccOLA0f3kgA5e58U3hDgLQZWVEtC+oR9mEcL2IKWTwfj4L5Q= smithi178.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKaf4hAhG8x96cvNOfFgEuyoN0ZyKUf9N/UJUeUXGqudGR6+w3nylXQxPI8ee2sw2bVpW90zIrChT50Rm5PGoPk= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/fsync-tester.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-07_05:46:12 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-07T07:18:35.975 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_8b896a9e145796119f0451201dc4d53ddac97db0/qa; will attempt to use it 2024-03-07T07:18:35.977 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_8b896a9e145796119f0451201dc4d53ddac97db0/qa/tasks 2024-03-07T07:18:35.977 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-07T07:18:35.978 INFO:teuthology.task.internal:Checking packages... 2024-03-07T07:18:36.004 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '8b896a9e145796119f0451201dc4d53ddac97db0' 2024-03-07T07:18:36.004 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-07T07:18:36.004 INFO:teuthology.packaging:ref: None 2024-03-07T07:18:36.004 INFO:teuthology.packaging:tag: None 2024-03-07T07:18:36.004 INFO:teuthology.packaging:branch: wip-lusov-qdb 2024-03-07T07:18:36.004 INFO:teuthology.packaging:sha1: 8b896a9e145796119f0451201dc4d53ddac97db0 2024-03-07T07:18:36.005 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-qdb 2024-03-07T07:18:36.154 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1848.g8b896a9e 2024-03-07T07:18:36.156 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-07T07:18:36.161 INFO:teuthology.task.internal:no buildpackages task found 2024-03-07T07:18:36.161 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-07T07:18:36.174 INFO:teuthology.task.internal:Saving configuration 2024-03-07T07:18:36.194 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-07T07:18:36.200 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-07T07:18:36.224 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi160.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-07_05:46:12-fs-wip-lusov-qdb-distro-default-smithi/7583972', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-07 07:13:03.390007', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEnyQs9uQLXfZEoZ2Tm1Acn3ntgsNJxul0I9jisv5u1TsPGeb7Dy8C3PsCPRw/v5VgTOSA98m6Qu0PVkzhvgwL8='} 2024-03-07T07:18:36.249 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi177.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-07_05:46:12-fs-wip-lusov-qdb-distro-default-smithi/7583972', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-07 07:13:03.392931', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNSrXChjdXzc1UOt8ymKReExe1A2gYLoXHB15wWccOLA0f3kgA5e58U3hDgLQZWVEtC+oR9mEcL2IKWTwfj4L5Q='} 2024-03-07T07:18:36.271 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi178.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-07_05:46:12-fs-wip-lusov-qdb-distro-default-smithi/7583972', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-07 07:13:03.391561', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKaf4hAhG8x96cvNOfFgEuyoN0ZyKUf9N/UJUeUXGqudGR6+w3nylXQxPI8ee2sw2bVpW90zIrChT50Rm5PGoPk='} 2024-03-07T07:18:36.271 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-07T07:18:36.277 INFO:teuthology.task.internal:roles: ubuntu@smithi160.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-03-07T07:18:36.277 INFO:teuthology.task.internal:roles: ubuntu@smithi177.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-03-07T07:18:36.277 INFO:teuthology.task.internal:roles: ubuntu@smithi178.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-03-07T07:18:36.278 INFO:teuthology.run_tasks:Running task console_log... 2024-03-07T07:18:36.375 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ff08ca51f70>, signals=[15]) 2024-03-07T07:18:36.376 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-07T07:18:36.382 INFO:teuthology.task.internal:Opening connections... 2024-03-07T07:18:36.382 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi160.front.sepia.ceph.com 2024-03-07T07:18:36.384 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T07:18:36.479 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi177.front.sepia.ceph.com 2024-03-07T07:18:36.480 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T07:18:36.567 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi178.front.sepia.ceph.com 2024-03-07T07:18:36.568 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T07:18:36.655 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-07T07:18:36.661 DEBUG:teuthology.orchestra.run.smithi160:> uname -m 2024-03-07T07:18:36.679 INFO:teuthology.orchestra.run.smithi160.stdout:x86_64 2024-03-07T07:18:36.680 DEBUG:teuthology.orchestra.run.smithi160:> cat /etc/os-release 2024-03-07T07:18:36.737 INFO:teuthology.orchestra.run.smithi160.stdout:NAME="CentOS Stream" 2024-03-07T07:18:36.737 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION="9" 2024-03-07T07:18:36.738 INFO:teuthology.orchestra.run.smithi160.stdout:ID="centos" 2024-03-07T07:18:36.738 INFO:teuthology.orchestra.run.smithi160.stdout:ID_LIKE="rhel fedora" 2024-03-07T07:18:36.738 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION_ID="9" 2024-03-07T07:18:36.738 INFO:teuthology.orchestra.run.smithi160.stdout:PLATFORM_ID="platform:el9" 2024-03-07T07:18:36.738 INFO:teuthology.orchestra.run.smithi160.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-07T07:18:36.738 INFO:teuthology.orchestra.run.smithi160.stdout:ANSI_COLOR="0;31" 2024-03-07T07:18:36.738 INFO:teuthology.orchestra.run.smithi160.stdout:LOGO="fedora-logo-icon" 2024-03-07T07:18:36.738 INFO:teuthology.orchestra.run.smithi160.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-07T07:18:36.738 INFO:teuthology.orchestra.run.smithi160.stdout:HOME_URL="https://centos.org/" 2024-03-07T07:18:36.739 INFO:teuthology.orchestra.run.smithi160.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-07T07:18:36.739 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-07T07:18:36.739 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-07T07:18:36.739 INFO:teuthology.lock.ops:Updating smithi160.front.sepia.ceph.com on lock server 2024-03-07T07:18:36.761 DEBUG:teuthology.orchestra.run.smithi177:> uname -m 2024-03-07T07:18:36.779 INFO:teuthology.orchestra.run.smithi177.stdout:x86_64 2024-03-07T07:18:36.780 DEBUG:teuthology.orchestra.run.smithi177:> cat /etc/os-release 2024-03-07T07:18:36.837 INFO:teuthology.orchestra.run.smithi177.stdout:NAME="CentOS Stream" 2024-03-07T07:18:36.837 INFO:teuthology.orchestra.run.smithi177.stdout:VERSION="9" 2024-03-07T07:18:36.837 INFO:teuthology.orchestra.run.smithi177.stdout:ID="centos" 2024-03-07T07:18:36.837 INFO:teuthology.orchestra.run.smithi177.stdout:ID_LIKE="rhel fedora" 2024-03-07T07:18:36.837 INFO:teuthology.orchestra.run.smithi177.stdout:VERSION_ID="9" 2024-03-07T07:18:36.837 INFO:teuthology.orchestra.run.smithi177.stdout:PLATFORM_ID="platform:el9" 2024-03-07T07:18:36.837 INFO:teuthology.orchestra.run.smithi177.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-07T07:18:36.838 INFO:teuthology.orchestra.run.smithi177.stdout:ANSI_COLOR="0;31" 2024-03-07T07:18:36.838 INFO:teuthology.orchestra.run.smithi177.stdout:LOGO="fedora-logo-icon" 2024-03-07T07:18:36.838 INFO:teuthology.orchestra.run.smithi177.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-07T07:18:36.838 INFO:teuthology.orchestra.run.smithi177.stdout:HOME_URL="https://centos.org/" 2024-03-07T07:18:36.838 INFO:teuthology.orchestra.run.smithi177.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-07T07:18:36.838 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-07T07:18:36.838 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-07T07:18:36.838 INFO:teuthology.lock.ops:Updating smithi177.front.sepia.ceph.com on lock server 2024-03-07T07:18:36.861 DEBUG:teuthology.orchestra.run.smithi178:> uname -m 2024-03-07T07:18:36.878 INFO:teuthology.orchestra.run.smithi178.stdout:x86_64 2024-03-07T07:18:36.878 DEBUG:teuthology.orchestra.run.smithi178:> cat /etc/os-release 2024-03-07T07:18:36.936 INFO:teuthology.orchestra.run.smithi178.stdout:NAME="CentOS Stream" 2024-03-07T07:18:36.936 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION="9" 2024-03-07T07:18:36.936 INFO:teuthology.orchestra.run.smithi178.stdout:ID="centos" 2024-03-07T07:18:36.936 INFO:teuthology.orchestra.run.smithi178.stdout:ID_LIKE="rhel fedora" 2024-03-07T07:18:36.936 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION_ID="9" 2024-03-07T07:18:36.936 INFO:teuthology.orchestra.run.smithi178.stdout:PLATFORM_ID="platform:el9" 2024-03-07T07:18:36.936 INFO:teuthology.orchestra.run.smithi178.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-07T07:18:36.936 INFO:teuthology.orchestra.run.smithi178.stdout:ANSI_COLOR="0;31" 2024-03-07T07:18:36.936 INFO:teuthology.orchestra.run.smithi178.stdout:LOGO="fedora-logo-icon" 2024-03-07T07:18:36.936 INFO:teuthology.orchestra.run.smithi178.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-07T07:18:36.937 INFO:teuthology.orchestra.run.smithi178.stdout:HOME_URL="https://centos.org/" 2024-03-07T07:18:36.937 INFO:teuthology.orchestra.run.smithi178.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-07T07:18:36.937 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-07T07:18:36.937 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-07T07:18:36.937 INFO:teuthology.lock.ops:Updating smithi178.front.sepia.ceph.com on lock server 2024-03-07T07:18:36.961 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-07T07:18:36.971 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-07T07:18:36.977 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-07T07:18:36.977 DEBUG:teuthology.orchestra.run.smithi160:> test '!' -e /home/ubuntu/cephtest 2024-03-07T07:18:36.980 DEBUG:teuthology.orchestra.run.smithi177:> test '!' -e /home/ubuntu/cephtest 2024-03-07T07:18:36.983 DEBUG:teuthology.orchestra.run.smithi178:> test '!' -e /home/ubuntu/cephtest 2024-03-07T07:18:36.999 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-07T07:18:37.005 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-07T07:18:37.005 DEBUG:teuthology.orchestra.run.smithi160:> test -z $(ls -A /var/lib/ceph) 2024-03-07T07:18:37.037 DEBUG:teuthology.orchestra.run.smithi177:> test -z $(ls -A /var/lib/ceph) 2024-03-07T07:18:37.039 DEBUG:teuthology.orchestra.run.smithi178:> test -z $(ls -A /var/lib/ceph) 2024-03-07T07:18:37.128 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-07T07:18:37.251 INFO:teuthology.run_tasks:Running task kernel... 2024-03-07T07:18:37.265 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-03-07T07:18:37.266 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-03-07T07:18:37.266 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-03-07T07:18:37.266 DEBUG:teuthology.orchestra.run.smithi160:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-07T07:18:37.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-07T07:18:37.283 DEBUG:teuthology.orchestra.run.smithi160:> uname -r 2024-03-07T07:18:37.341 INFO:teuthology.orchestra.run.smithi160.stdout:5.14.0-361.el9.x86_64 2024-03-07T07:18:37.341 INFO:teuthology.task.kernel:Running kernel on smithi160: 5.14.0-361.el9.x86_64 2024-03-07T07:18:37.342 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum install -y kernel 2024-03-07T07:18:39.453 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS-9 - AppStream 17 MB/s | 8.1 MB 00:00 2024-03-07T07:18:41.162 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS-9 - BaseOS 6.8 MB/s | 2.5 MB 00:00 2024-03-07T07:18:42.186 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-03-07T07:18:44.928 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS Stream 9 - AppStream 12 MB/s | 19 MB 00:01 2024-03-07T07:18:49.987 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS Stream 9 - CRB 4.1 MB/s | 6.1 MB 00:01 2024-03-07T07:18:51.454 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS Stream 9 - Extras packages 51 kB/s | 15 kB 00:00 2024-03-07T07:18:53.072 INFO:teuthology.orchestra.run.smithi160.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-03-07T07:18:58.122 INFO:teuthology.orchestra.run.smithi160.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-03-07T07:19:00.330 INFO:teuthology.orchestra.run.smithi160.stderr:Modular dependency problems: 2024-03-07T07:19:00.331 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-03-07T07:19:00.331 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 1: conflicting requests 2024-03-07T07:19:00.331 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240227110532:20125149.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.331 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 2: conflicting requests 2024-03-07T07:19:00.331 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.331 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 3: conflicting requests 2024-03-07T07:19:00.331 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.331 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 4: conflicting requests 2024-03-07T07:19:00.331 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.331 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 5: conflicting requests 2024-03-07T07:19:00.331 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.332 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 6: conflicting requests 2024-03-07T07:19:00.332 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.332 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 7: conflicting requests 2024-03-07T07:19:00.332 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240222150954:49cc9d1b.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.332 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 8: conflicting requests 2024-03-07T07:19:00.332 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.332 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 9: conflicting requests 2024-03-07T07:19:00.332 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.332 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 10: conflicting requests 2024-03-07T07:19:00.332 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.332 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 11: conflicting requests 2024-03-07T07:19:00.332 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.332 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 12: conflicting requests 2024-03-07T07:19:00.332 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.333 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 13: conflicting requests 2024-03-07T07:19:00.333 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.333 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 14: conflicting requests 2024-03-07T07:19:00.333 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.333 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 15: conflicting requests 2024-03-07T07:19:00.333 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.333 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 16: conflicting requests 2024-03-07T07:19:00.333 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.333 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 17: conflicting requests 2024-03-07T07:19:00.333 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.333 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 18: conflicting requests 2024-03-07T07:19:00.333 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.333 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 19: conflicting requests 2024-03-07T07:19:00.333 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.334 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 20: conflicting requests 2024-03-07T07:19:00.334 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.334 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 21: conflicting requests 2024-03-07T07:19:00.334 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.334 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 22: conflicting requests 2024-03-07T07:19:00.334 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.334 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 23: conflicting requests 2024-03-07T07:19:00.334 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.334 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 24: conflicting requests 2024-03-07T07:19:00.334 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.334 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 25: conflicting requests 2024-03-07T07:19:00.334 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.334 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 26: conflicting requests 2024-03-07T07:19:00.334 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.335 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 27: conflicting requests 2024-03-07T07:19:00.335 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.335 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 28: conflicting requests 2024-03-07T07:19:00.335 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.335 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 29: conflicting requests 2024-03-07T07:19:00.335 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.335 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 30: conflicting requests 2024-03-07T07:19:00.335 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.335 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 31: conflicting requests 2024-03-07T07:19:00.335 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.335 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 32: conflicting requests 2024-03-07T07:19:00.335 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.335 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 33: conflicting requests 2024-03-07T07:19:00.335 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.336 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 34: conflicting requests 2024-03-07T07:19:00.336 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.336 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 35: conflicting requests 2024-03-07T07:19:00.336 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.336 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 36: conflicting requests 2024-03-07T07:19:00.336 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.336 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 37: conflicting requests 2024-03-07T07:19:00.336 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.336 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.336 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.336 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 38: cannot install the best candidate for the job 2024-03-07T07:19:00.336 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.336 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.336 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.337 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.337 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 39: conflicting requests 2024-03-07T07:19:00.337 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.337 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.337 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.337 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.337 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.337 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.337 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 40: cannot install the best candidate for the job 2024-03-07T07:19:00.337 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.337 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.337 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.337 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.337 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 41: cannot install the best candidate for the job 2024-03-07T07:19:00.337 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.338 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.338 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.338 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.338 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 42: cannot install the best candidate for the job 2024-03-07T07:19:00.338 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.338 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.338 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.338 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.338 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 43: cannot install the best candidate for the job 2024-03-07T07:19:00.338 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.338 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.338 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.338 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.338 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 44: cannot install the best candidate for the job 2024-03-07T07:19:00.338 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.339 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.339 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.339 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.339 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 45: cannot install the best candidate for the job 2024-03-07T07:19:00.339 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.339 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.339 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.339 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-03-07T07:19:00.738 INFO:teuthology.orchestra.run.smithi160.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-07T07:19:00.738 INFO:teuthology.orchestra.run.smithi160.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-07T07:19:00.843 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-03-07T07:19:00.851 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-03-07T07:19:00.851 INFO:teuthology.orchestra.run.smithi160.stdout: Package Architecture Version Repository Size 2024-03-07T07:19:00.852 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-03-07T07:19:00.852 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2024-03-07T07:19:00.852 INFO:teuthology.orchestra.run.smithi160.stdout: kernel x86_64 5.14.0-427.el9 baseos 6.0 M 2024-03-07T07:19:00.852 INFO:teuthology.orchestra.run.smithi160.stdout:Installing dependencies: 2024-03-07T07:19:00.852 INFO:teuthology.orchestra.run.smithi160.stdout: kernel-core x86_64 5.14.0-427.el9 baseos 21 M 2024-03-07T07:19:00.852 INFO:teuthology.orchestra.run.smithi160.stdout: kernel-modules x86_64 5.14.0-427.el9 baseos 39 M 2024-03-07T07:19:00.852 INFO:teuthology.orchestra.run.smithi160.stdout: kernel-modules-core x86_64 5.14.0-427.el9 baseos 33 M 2024-03-07T07:19:00.852 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:19:00.852 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2024-03-07T07:19:00.852 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-03-07T07:19:00.852 INFO:teuthology.orchestra.run.smithi160.stdout:Install 4 Packages 2024-03-07T07:19:00.852 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:19:00.853 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 100 M 2024-03-07T07:19:00.853 INFO:teuthology.orchestra.run.smithi160.stdout:Installed size: 123 M 2024-03-07T07:19:00.853 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading Packages: 2024-03-07T07:19:02.707 INFO:teuthology.orchestra.run.smithi160.stdout:(1/4): kernel-5.14.0-427.el9.x86_64.rpm 3.3 MB/s | 6.0 MB 00:01 2024-03-07T07:19:22.996 INFO:teuthology.orchestra.run.smithi160.stdout:(2/4): kernel-core-5.14.0-427.el9.x86_64.rpm 966 kB/s | 21 MB 00:22 2024-03-07T07:19:39.503 INFO:teuthology.orchestra.run.smithi160.stdout:(3/4): kernel-modules-core-5.14.0-427.el9.x86_6 930 kB/s | 33 MB 00:36 2024-03-07T07:19:43.436 INFO:teuthology.orchestra.run.smithi160.stdout:(4/4): kernel-modules-5.14.0-427.el9.x86_64.rpm 946 kB/s | 39 MB 00:42 2024-03-07T07:19:43.436 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2024-03-07T07:19:43.436 INFO:teuthology.orchestra.run.smithi160.stdout:Total 2.3 MB/s | 100 MB 00:42 2024-03-07T07:19:44.320 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2024-03-07T07:19:44.697 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction check succeeded. 2024-03-07T07:19:44.697 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2024-03-07T07:19:46.333 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded. 2024-03-07T07:19:46.334 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2024-03-07T07:19:49.295 INFO:teuthology.orchestra.run.smithi160.stdout: Preparing : 1/1 2024-03-07T07:19:49.949 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : kernel-modules-core-5.14.0-427.el9.x86_64 1/4 2024-03-07T07:19:50.100 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-07T07:19:52.921 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-07T07:19:53.097 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-07T07:19:57.827 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-07T07:19:57.949 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : kernel-5.14.0-427.el9.x86_64 4/4 2024-03-07T07:20:02.735 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-07T07:20:31.832 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 4/4 2024-03-07T07:20:31.832 INFO:teuthology.orchestra.run.smithi160.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-07T07:20:31.832 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:20:31.918 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 4/4 2024-03-07T07:20:32.379 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: kernel-5.14.0-427.el9.x86_64 4/4 2024-03-07T07:20:32.379 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : kernel-5.14.0-427.el9.x86_64 1/4 2024-03-07T07:20:32.379 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-07T07:20:32.379 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-07T07:20:32.818 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-07T07:20:32.818 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:20:32.818 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2024-03-07T07:20:32.818 INFO:teuthology.orchestra.run.smithi160.stdout: kernel-5.14.0-427.el9.x86_64 2024-03-07T07:20:32.818 INFO:teuthology.orchestra.run.smithi160.stdout: kernel-core-5.14.0-427.el9.x86_64 2024-03-07T07:20:32.818 INFO:teuthology.orchestra.run.smithi160.stdout: kernel-modules-5.14.0-427.el9.x86_64 2024-03-07T07:20:32.818 INFO:teuthology.orchestra.run.smithi160.stdout: kernel-modules-core-5.14.0-427.el9.x86_64 2024-03-07T07:20:32.818 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:20:32.818 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-03-07T07:20:33.335 DEBUG:teuthology.orchestra.run.smithi160:> rpm -q kernel | sort -rV | head -n 1 2024-03-07T07:20:33.524 INFO:teuthology.orchestra.run.smithi160.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-07T07:20:33.524 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-07T07:20:33.524 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-427.el9.x86_64 2024-03-07T07:20:33.524 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-07T07:20:33.524 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-07T07:20:33.524 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-03-07T07:20:33.525 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi160.front.sepia.ceph.com, path=None, version=distro) 2024-03-07T07:20:33.525 DEBUG:teuthology.orchestra.run.smithi160:> rpm -q kernel | sort -rV | head -n 1 2024-03-07T07:20:33.645 INFO:teuthology.orchestra.run.smithi160.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-07T07:20:33.646 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-07T07:20:33.646 DEBUG:teuthology.orchestra.run.smithi160:> sudo rpm -qi grub2-tools 2024-03-07T07:20:33.679 INFO:teuthology.orchestra.run.smithi160.stdout:Name : grub2-tools 2024-03-07T07:20:33.680 INFO:teuthology.orchestra.run.smithi160.stdout:Epoch : 1 2024-03-07T07:20:33.680 INFO:teuthology.orchestra.run.smithi160.stdout:Version : 2.06 2024-03-07T07:20:33.680 INFO:teuthology.orchestra.run.smithi160.stdout:Release : 68.el9 2024-03-07T07:20:33.680 INFO:teuthology.orchestra.run.smithi160.stdout:Architecture: x86_64 2024-03-07T07:20:33.680 INFO:teuthology.orchestra.run.smithi160.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-07T07:20:33.680 INFO:teuthology.orchestra.run.smithi160.stdout:Group : Unspecified 2024-03-07T07:20:33.680 INFO:teuthology.orchestra.run.smithi160.stdout:Size : 8264254 2024-03-07T07:20:33.680 INFO:teuthology.orchestra.run.smithi160.stdout:License : GPLv3+ 2024-03-07T07:20:33.680 INFO:teuthology.orchestra.run.smithi160.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-07T07:20:33.680 INFO:teuthology.orchestra.run.smithi160.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-07T07:20:33.680 INFO:teuthology.orchestra.run.smithi160.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-07T07:20:33.680 INFO:teuthology.orchestra.run.smithi160.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-07T07:20:33.680 INFO:teuthology.orchestra.run.smithi160.stdout:Packager : builder@centos.org 2024-03-07T07:20:33.681 INFO:teuthology.orchestra.run.smithi160.stdout:Vendor : CentOS 2024-03-07T07:20:33.681 INFO:teuthology.orchestra.run.smithi160.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-07T07:20:33.681 INFO:teuthology.orchestra.run.smithi160.stdout:Summary : Support tools for GRUB. 2024-03-07T07:20:33.681 INFO:teuthology.orchestra.run.smithi160.stdout:Description : 2024-03-07T07:20:33.681 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:20:33.681 INFO:teuthology.orchestra.run.smithi160.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-07T07:20:33.681 INFO:teuthology.orchestra.run.smithi160.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-07T07:20:33.681 INFO:teuthology.orchestra.run.smithi160.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-07T07:20:33.681 INFO:teuthology.orchestra.run.smithi160.stdout:hardware devices. 2024-03-07T07:20:33.681 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:20:33.681 INFO:teuthology.orchestra.run.smithi160.stdout:This subpackage provides tools for support of all platforms. 2024-03-07T07:20:33.682 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-07T07:20:33.682 INFO:teuthology.task.kernel:Updating grub on smithi160 to boot 5.14.0-427.el9.x86_64 2024-03-07T07:20:33.682 DEBUG:teuthology.orchestra.run.smithi160:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-07T07:20:34.331 INFO:teuthology.orchestra.run.smithi160.stderr:Generating grub configuration file ... 2024-03-07T07:20:35.660 INFO:teuthology.orchestra.run.smithi160.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-07T07:20:35.680 INFO:teuthology.orchestra.run.smithi160.stderr:done 2024-03-07T07:20:35.683 DEBUG:teuthology.orchestra.run.smithi160:> mktemp 2024-03-07T07:20:35.698 INFO:teuthology.orchestra.run.smithi160.stdout:/tmp/tmp.QjuVMqLHvA 2024-03-07T07:20:35.699 DEBUG:teuthology.orchestra.run.smithi160:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.QjuVMqLHvA 2024-03-07T07:20:35.772 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod 0666 /tmp/tmp.QjuVMqLHvA 2024-03-07T07:20:35.921 DEBUG:teuthology.orchestra.remote:smithi160:/tmp/tmp.QjuVMqLHvA is 6KB 2024-03-07T07:20:35.970 DEBUG:teuthology.orchestra.run.smithi160:> rm -fr /tmp/tmp.QjuVMqLHvA 2024-03-07T07:20:35.983 DEBUG:teuthology.orchestra.run.smithi160:> sudo /bin/ls /boot/loader/entries || true 2024-03-07T07:20:36.048 INFO:teuthology.orchestra.run.smithi160.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-07T07:20:36.049 INFO:teuthology.orchestra.run.smithi160.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-07T07:20:36.049 INFO:teuthology.orchestra.run.smithi160.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-07T07:20:36.049 INFO:teuthology.orchestra.run.smithi160.stdout:15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64.conf 2024-03-07T07:20:36.050 DEBUG:teuthology.orchestra.run.smithi160:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64 2024-03-07T07:20:36.202 DEBUG:teuthology.orchestra.run.smithi160:> sudo shutdown -r now 2024-03-07T07:21:06.205 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-07T07:21:06.206 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-03-07T07:21:06.207 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T07:21:24.540 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.160 2024-03-07T07:21:33.541 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-03-07T07:21:33.542 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T07:21:36.604 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.160 2024-03-07T07:21:48.604 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-03-07T07:21:48.605 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T07:22:48.608 DEBUG:teuthology.orchestra.remote:timed out 2024-03-07T07:23:03.610 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-03-07T07:23:03.611 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T07:23:04.155 DEBUG:teuthology.orchestra.run.smithi160:> true 2024-03-07T07:23:04.535 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-03-07T07:23:04.535 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-427.el9.x86_64"... 2024-03-07T07:23:04.535 DEBUG:teuthology.orchestra.run.smithi160:> uname -r 2024-03-07T07:23:04.551 INFO:teuthology.orchestra.run.smithi160.stdout:5.14.0-427.el9.x86_64 2024-03-07T07:23:04.551 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-427.el9.x86_64 vs 5.14.0-427.el9.x86_64 2024-03-07T07:23:04.551 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-07T07:23:04.551 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-07T07:23:05.552 DEBUG:teuthology.parallel:result is None 2024-03-07T07:23:05.552 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-07T07:23:05.560 INFO:teuthology.task.internal:Creating test directory... 2024-03-07T07:23:05.561 DEBUG:teuthology.orchestra.run.smithi160:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-07T07:23:05.563 DEBUG:teuthology.orchestra.run.smithi177:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-07T07:23:05.566 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-07T07:23:05.584 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-07T07:23:05.591 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-07T07:23:05.597 INFO:teuthology.task.internal:Creating archive directory... 2024-03-07T07:23:05.598 DEBUG:teuthology.orchestra.run.smithi160:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-07T07:23:05.621 DEBUG:teuthology.orchestra.run.smithi177:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-07T07:23:05.624 DEBUG:teuthology.orchestra.run.smithi178:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-07T07:23:05.672 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-07T07:23:05.679 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-07T07:23:05.679 DEBUG:teuthology.orchestra.run.smithi177:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-07T07:23:05.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-07T07:23:05.721 DEBUG:teuthology.orchestra.run.smithi178:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-07T07:23:05.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-07T07:23:05.738 DEBUG:teuthology.orchestra.run.smithi160:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-07T07:23:05.741 DEBUG:teuthology.orchestra.run.smithi177:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-07T07:23:05.764 DEBUG:teuthology.orchestra.run.smithi178:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-07T07:23:05.799 INFO:teuthology.orchestra.run.smithi177.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-07T07:23:05.816 INFO:teuthology.orchestra.run.smithi177.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-07T07:23:05.828 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-07T07:23:05.851 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-07T07:23:05.852 INFO:teuthology.orchestra.run.smithi160.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-07T07:23:05.884 INFO:teuthology.orchestra.run.smithi160.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-07T07:23:05.886 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-07T07:23:05.893 INFO:teuthology.task.internal:Configuring sudo... 2024-03-07T07:23:05.893 DEBUG:teuthology.orchestra.run.smithi160:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-07T07:23:05.896 DEBUG:teuthology.orchestra.run.smithi177:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-07T07:23:05.899 DEBUG:teuthology.orchestra.run.smithi178:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-07T07:23:05.929 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-07T07:23:05.939 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-07T07:23:05.939 DEBUG:teuthology.orchestra.run.smithi160:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-07T07:23:05.967 DEBUG:teuthology.orchestra.run.smithi177:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-07T07:23:05.969 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-07T07:23:05.986 DEBUG:teuthology.orchestra.run.smithi160:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-07T07:23:06.069 DEBUG:teuthology.orchestra.run.smithi160:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-07T07:23:06.112 DEBUG:teuthology.orchestra.run.smithi160:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-07T07:23:06.199 DEBUG:teuthology.orchestra.run.smithi160:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-07T07:23:06.241 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-03-07T07:23:06.241 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-07T07:23:06.309 DEBUG:teuthology.orchestra.run.smithi177:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-07T07:23:06.352 DEBUG:teuthology.orchestra.run.smithi177:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-07T07:23:06.412 DEBUG:teuthology.orchestra.run.smithi177:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-07T07:23:06.496 DEBUG:teuthology.orchestra.run.smithi177:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-07T07:23:06.538 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-03-07T07:23:06.539 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-07T07:23:06.606 DEBUG:teuthology.orchestra.run.smithi178:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-07T07:23:06.651 DEBUG:teuthology.orchestra.run.smithi178:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-07T07:23:06.713 DEBUG:teuthology.orchestra.run.smithi178:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-07T07:23:06.755 DEBUG:teuthology.orchestra.run.smithi178:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-07T07:23:06.801 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-03-07T07:23:06.801 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-07T07:23:06.870 DEBUG:teuthology.orchestra.run.smithi160:> sudo service rsyslog restart 2024-03-07T07:23:06.872 DEBUG:teuthology.orchestra.run.smithi177:> sudo service rsyslog restart 2024-03-07T07:23:06.874 DEBUG:teuthology.orchestra.run.smithi178:> sudo service rsyslog restart 2024-03-07T07:23:06.913 INFO:teuthology.orchestra.run.smithi177.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-07T07:23:06.916 INFO:teuthology.orchestra.run.smithi160.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-07T07:23:06.949 INFO:teuthology.orchestra.run.smithi178.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-07T07:23:07.385 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-07T07:23:07.394 INFO:teuthology.task.internal:Starting timer... 2024-03-07T07:23:07.394 INFO:teuthology.run_tasks:Running task pcp... 2024-03-07T07:23:07.404 INFO:teuthology.run_tasks:Running task selinux... 2024-03-07T07:23:07.412 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-03-07T07:23:07.413 DEBUG:teuthology.orchestra.run.smithi160:> sudo service auditd rotate 2024-03-07T07:23:07.488 INFO:teuthology.orchestra.run.smithi160.stdout:Rotating logs: 2024-03-07T07:23:07.490 DEBUG:teuthology.orchestra.run.smithi177:> sudo service auditd rotate 2024-03-07T07:23:07.544 INFO:teuthology.orchestra.run.smithi177.stdout:Rotating logs: 2024-03-07T07:23:07.546 DEBUG:teuthology.orchestra.run.smithi178:> sudo service auditd rotate 2024-03-07T07:23:07.601 INFO:teuthology.orchestra.run.smithi178.stdout:Rotating logs: 2024-03-07T07:23:07.603 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-07T07:23:07.603 DEBUG:teuthology.orchestra.run.smithi160:> /usr/sbin/getenforce 2024-03-07T07:23:07.628 INFO:teuthology.orchestra.run.smithi160.stdout:Permissive 2024-03-07T07:23:07.628 DEBUG:teuthology.orchestra.run.smithi177:> /usr/sbin/getenforce 2024-03-07T07:23:07.648 INFO:teuthology.orchestra.run.smithi177.stdout:Permissive 2024-03-07T07:23:07.648 DEBUG:teuthology.orchestra.run.smithi178:> /usr/sbin/getenforce 2024-03-07T07:23:07.673 INFO:teuthology.orchestra.run.smithi178.stdout:Permissive 2024-03-07T07:23:07.674 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi160.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi177.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi178.front.sepia.ceph.com': 'permissive'} 2024-03-07T07:23:07.674 DEBUG:teuthology.orchestra.run.smithi160:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-07T07:23:07.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-07T07:23:07.702 DEBUG:teuthology.orchestra.run.smithi177:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-07T07:23:07.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-07T07:23:07.731 DEBUG:teuthology.orchestra.run.smithi178:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-07T07:23:07.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-07T07:23:07.758 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-07T07:23:07.758 DEBUG:teuthology.orchestra.run.smithi160:> sudo /usr/sbin/setenforce permissive 2024-03-07T07:23:07.788 DEBUG:teuthology.orchestra.run.smithi177:> sudo /usr/sbin/setenforce permissive 2024-03-07T07:23:07.825 DEBUG:teuthology.orchestra.run.smithi178:> sudo /usr/sbin/setenforce permissive 2024-03-07T07:23:07.859 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-07T07:23:07.868 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-07T07:23:07.874 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-07T07:23:08.026 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-07T07:23:08.042 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-07T07:23:08.043 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi160.front.sepia.ceph.com,smithi177.front.sepia.ceph.com,smithi178.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-07T07:27:50.716 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi160.front.sepia.ceph.com'), Remote(name='ubuntu@smithi177.front.sepia.ceph.com'), Remote(name='ubuntu@smithi178.front.sepia.ceph.com')] 2024-03-07T07:27:50.717 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-03-07T07:27:50.718 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T07:27:50.811 DEBUG:teuthology.orchestra.run.smithi160:> true 2024-03-07T07:27:50.892 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-03-07T07:27:50.894 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-03-07T07:27:50.895 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T07:27:50.984 DEBUG:teuthology.orchestra.run.smithi177:> true 2024-03-07T07:27:51.051 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-03-07T07:27:51.051 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-03-07T07:27:51.052 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T07:27:51.140 DEBUG:teuthology.orchestra.run.smithi178:> true 2024-03-07T07:27:51.211 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-03-07T07:27:51.211 INFO:teuthology.run_tasks:Running task clock... 2024-03-07T07:27:51.222 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-07T07:27:51.222 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-07T07:27:51.223 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-07T07:27:51.226 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-07T07:27:51.226 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-07T07:27:51.228 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-07T07:27:51.228 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-07T07:27:51.254 INFO:teuthology.orchestra.run.smithi160.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-07T07:27:51.258 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-07T07:27:51.271 INFO:teuthology.orchestra.run.smithi160.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-07T07:27:51.275 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-07T07:27:51.283 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-07T07:27:51.299 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-07T07:27:51.300 INFO:teuthology.orchestra.run.smithi160.stderr:sudo: ntpd: command not found 2024-03-07T07:27:51.305 INFO:teuthology.orchestra.run.smithi177.stderr:sudo: ntpd: command not found 2024-03-07T07:27:51.314 INFO:teuthology.orchestra.run.smithi160.stdout:506 Cannot talk to daemon 2024-03-07T07:27:51.320 INFO:teuthology.orchestra.run.smithi177.stdout:506 Cannot talk to daemon 2024-03-07T07:27:51.329 INFO:teuthology.orchestra.run.smithi178.stderr:sudo: ntpd: command not found 2024-03-07T07:27:51.330 INFO:teuthology.orchestra.run.smithi160.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-07T07:27:51.336 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-07T07:27:51.342 INFO:teuthology.orchestra.run.smithi178.stdout:506 Cannot talk to daemon 2024-03-07T07:27:51.347 INFO:teuthology.orchestra.run.smithi160.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-07T07:27:51.353 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-07T07:27:51.358 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-07T07:27:51.375 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-07T07:27:51.397 INFO:teuthology.orchestra.run.smithi177.stderr:bash: line 1: ntpq: command not found 2024-03-07T07:27:51.398 INFO:teuthology.orchestra.run.smithi160.stderr:bash: line 1: ntpq: command not found 2024-03-07T07:27:51.401 INFO:teuthology.orchestra.run.smithi177.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-07T07:27:51.401 INFO:teuthology.orchestra.run.smithi177.stdout:=============================================================================== 2024-03-07T07:27:51.401 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-07T07:27:51.401 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-07T07:27:51.401 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-07T07:27:51.401 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-07T07:27:51.402 INFO:teuthology.orchestra.run.smithi160.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-07T07:27:51.402 INFO:teuthology.orchestra.run.smithi160.stdout:=============================================================================== 2024-03-07T07:27:51.402 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-07T07:27:51.402 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-07T07:27:51.402 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-07T07:27:51.403 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-07T07:27:51.418 INFO:teuthology.orchestra.run.smithi178.stderr:bash: line 1: ntpq: command not found 2024-03-07T07:27:51.422 INFO:teuthology.orchestra.run.smithi178.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-07T07:27:51.422 INFO:teuthology.orchestra.run.smithi178.stdout:=============================================================================== 2024-03-07T07:27:51.422 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-07T07:27:51.422 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-07T07:27:51.422 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-07T07:27:51.422 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-07T07:27:51.422 INFO:teuthology.run_tasks:Running task install... 2024-03-07T07:27:51.431 DEBUG:teuthology.task.install:project ceph 2024-03-07T07:27:51.431 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8b896a9e145796119f0451201dc4d53ddac97db0'}} 2024-03-07T07:27:51.431 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8b896a9e145796119f0451201dc4d53ddac97db0'} 2024-03-07T07:27:51.431 INFO:teuthology.task.install:Using flavor: default 2024-03-07T07:27:51.439 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-07T07:27:51.439 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-07T07:27:51.440 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8b896a9e145796119f0451201dc4d53ddac97db0', 'tag': None, 'wait_for_package': False} 2024-03-07T07:27:51.440 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8b896a9e145796119f0451201dc4d53ddac97db0 2024-03-07T07:27:51.442 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8b896a9e145796119f0451201dc4d53ddac97db0', 'tag': None, 'wait_for_package': False} 2024-03-07T07:27:51.442 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8b896a9e145796119f0451201dc4d53ddac97db0 2024-03-07T07:27:51.443 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8b896a9e145796119f0451201dc4d53ddac97db0', 'tag': None, 'wait_for_package': False} 2024-03-07T07:27:51.444 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8b896a9e145796119f0451201dc4d53ddac97db0 2024-03-07T07:27:51.589 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-lusov-qdb/8b896a9e145796119f0451201dc4d53ddac97db0/centos/9/flavors/default/ 2024-03-07T07:27:51.589 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1848.g8b896a9e 2024-03-07T07:27:51.597 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-lusov-qdb/8b896a9e145796119f0451201dc4d53ddac97db0/centos/9/flavors/default/ 2024-03-07T07:27:51.597 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1848.g8b896a9e 2024-03-07T07:27:51.661 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-lusov-qdb/8b896a9e145796119f0451201dc4d53ddac97db0/centos/9/flavors/default/ 2024-03-07T07:27:51.661 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1848.g8b896a9e 2024-03-07T07:27:51.707 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-lusov-qdb/8b896a9e145796119f0451201dc4d53ddac97db0/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-lusov-qdb/8b896a9e145796119f0451201dc4d53ddac97db0/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-lusov-qdb/8b896a9e145796119f0451201dc4d53ddac97db0/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-07T07:27:51.708 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-03-07T07:27:51.708 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-07T07:27:51.736 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-07T07:27:51.736 DEBUG:teuthology.orchestra.run.smithi178:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8b896a9e145796119f0451201dc4d53ddac97db0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-07T07:27:51.757 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-lusov-qdb/8b896a9e145796119f0451201dc4d53ddac97db0/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-lusov-qdb/8b896a9e145796119f0451201dc4d53ddac97db0/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-lusov-qdb/8b896a9e145796119f0451201dc4d53ddac97db0/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-07T07:27:51.757 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-03-07T07:27:51.757 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-07T07:27:51.772 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-lusov-qdb/8b896a9e145796119f0451201dc4d53ddac97db0/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-lusov-qdb/8b896a9e145796119f0451201dc4d53ddac97db0/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-lusov-qdb/8b896a9e145796119f0451201dc4d53ddac97db0/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-07T07:27:51.772 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-03-07T07:27:51.772 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-07T07:27:51.787 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-07T07:27:51.787 DEBUG:teuthology.orchestra.run.smithi177:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8b896a9e145796119f0451201dc4d53ddac97db0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-07T07:27:51.800 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-07T07:27:51.801 DEBUG:teuthology.orchestra.run.smithi160:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8b896a9e145796119f0451201dc4d53ddac97db0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-07T07:27:51.804 DEBUG:teuthology.orchestra.run.smithi178:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-07T07:27:51.856 DEBUG:teuthology.orchestra.run.smithi177:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-07T07:27:51.870 DEBUG:teuthology.orchestra.run.smithi160:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-07T07:27:51.885 DEBUG:teuthology.orchestra.run.smithi178:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-07T07:27:51.937 DEBUG:teuthology.orchestra.run.smithi177:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-07T07:27:51.950 DEBUG:teuthology.orchestra.run.smithi160:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-07T07:27:51.951 INFO:teuthology.orchestra.run.smithi178.stdout:check_obsoletes = 1 2024-03-07T07:27:51.952 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum clean all 2024-03-07T07:27:52.003 INFO:teuthology.orchestra.run.smithi177.stdout:check_obsoletes = 1 2024-03-07T07:27:52.005 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum clean all 2024-03-07T07:27:52.017 INFO:teuthology.orchestra.run.smithi160.stdout:check_obsoletes = 1 2024-03-07T07:27:52.019 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum clean all 2024-03-07T07:27:52.188 INFO:teuthology.orchestra.run.smithi178.stdout:74 files removed 2024-03-07T07:27:52.215 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-07T07:27:52.240 INFO:teuthology.orchestra.run.smithi177.stdout:74 files removed 2024-03-07T07:27:52.262 INFO:teuthology.orchestra.run.smithi160.stdout:74 files removed 2024-03-07T07:27:52.267 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-07T07:27:52.290 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-07T07:27:52.937 INFO:teuthology.orchestra.run.smithi160.stdout:ceph packages for x86_64 254 kB/s | 83 kB 00:00 2024-03-07T07:27:52.978 INFO:teuthology.orchestra.run.smithi178.stdout:ceph packages for x86_64 176 kB/s | 83 kB 00:00 2024-03-07T07:27:53.016 INFO:teuthology.orchestra.run.smithi177.stdout:ceph packages for x86_64 182 kB/s | 83 kB 00:00 2024-03-07T07:27:53.354 INFO:teuthology.orchestra.run.smithi160.stdout:ceph noarch packages 27 kB/s | 10 kB 00:00 2024-03-07T07:27:53.395 INFO:teuthology.orchestra.run.smithi178.stdout:ceph noarch packages 27 kB/s | 10 kB 00:00 2024-03-07T07:27:53.430 INFO:teuthology.orchestra.run.smithi177.stdout:ceph noarch packages 27 kB/s | 10 kB 00:00 2024-03-07T07:27:53.796 INFO:teuthology.orchestra.run.smithi160.stdout:ceph source packages 4.7 kB/s | 1.9 kB 00:00 2024-03-07T07:27:53.797 INFO:teuthology.orchestra.run.smithi177.stdout:ceph source packages 5.6 kB/s | 1.9 kB 00:00 2024-03-07T07:27:53.812 INFO:teuthology.orchestra.run.smithi178.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2024-03-07T07:27:54.466 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS-9 - AppStream 13 MB/s | 8.1 MB 00:00 2024-03-07T07:27:54.568 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS-9 - AppStream 11 MB/s | 8.1 MB 00:00 2024-03-07T07:27:54.657 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS-9 - AppStream 9.9 MB/s | 8.1 MB 00:00 2024-03-07T07:27:56.207 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS-9 - BaseOS 7.9 MB/s | 2.5 MB 00:00 2024-03-07T07:27:56.241 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS-9 - BaseOS 6.9 MB/s | 2.5 MB 00:00 2024-03-07T07:27:56.257 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS-9 - BaseOS 8.6 MB/s | 2.5 MB 00:00 2024-03-07T07:27:59.631 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS Stream 9 - BaseOS 2.8 MB/s | 8.0 MB 00:02 2024-03-07T07:28:01.364 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - BaseOS 1.7 MB/s | 8.0 MB 00:04 2024-03-07T07:28:04.231 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS Stream 9 - AppStream 5.4 MB/s | 19 MB 00:03 2024-03-07T07:28:05.714 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - BaseOS 910 kB/s | 8.0 MB 00:09 2024-03-07T07:28:06.697 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - AppStream 4.4 MB/s | 19 MB 00:04 2024-03-07T07:28:08.174 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.1 MB 00:00 2024-03-07T07:28:08.698 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - AppStream 10 MB/s | 19 MB 00:01 2024-03-07T07:28:09.558 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS Stream 9 - Extras packages 71 kB/s | 15 kB 00:00 2024-03-07T07:28:11.409 INFO:teuthology.orchestra.run.smithi160.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-03-07T07:28:11.656 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 6.1 MB 00:01 2024-03-07T07:28:12.673 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - CRB 17 MB/s | 6.1 MB 00:00 2024-03-07T07:28:13.082 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - Extras packages 59 kB/s | 15 kB 00:00 2024-03-07T07:28:14.133 INFO:teuthology.orchestra.run.smithi178.stdout:Extra Packages for Enterprise Linux 23 MB/s | 21 MB 00:00 2024-03-07T07:28:14.608 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - Extras packages 20 kB/s | 15 kB 00:00 2024-03-07T07:28:16.384 INFO:teuthology.orchestra.run.smithi160.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-03-07T07:28:18.500 INFO:teuthology.orchestra.run.smithi177.stdout:Extra Packages for Enterprise Linux 5.5 MB/s | 21 MB 00:03 2024-03-07T07:28:18.908 INFO:teuthology.orchestra.run.smithi160.stderr:Modular dependency problems: 2024-03-07T07:28:18.908 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-03-07T07:28:18.908 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 1: conflicting requests 2024-03-07T07:28:18.909 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240227110532:20125149.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.909 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 2: conflicting requests 2024-03-07T07:28:18.909 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.909 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 3: conflicting requests 2024-03-07T07:28:18.909 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.909 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 4: conflicting requests 2024-03-07T07:28:18.910 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.910 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 5: conflicting requests 2024-03-07T07:28:18.910 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.910 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 6: conflicting requests 2024-03-07T07:28:18.910 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.911 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 7: conflicting requests 2024-03-07T07:28:18.911 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240222150954:49cc9d1b.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.911 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 8: conflicting requests 2024-03-07T07:28:18.911 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.911 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 9: conflicting requests 2024-03-07T07:28:18.912 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.912 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 10: conflicting requests 2024-03-07T07:28:18.912 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.915 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 11: conflicting requests 2024-03-07T07:28:18.915 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.916 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 12: conflicting requests 2024-03-07T07:28:18.916 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.916 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 13: conflicting requests 2024-03-07T07:28:18.916 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.916 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 14: conflicting requests 2024-03-07T07:28:18.917 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.917 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 15: conflicting requests 2024-03-07T07:28:18.917 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.917 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 16: conflicting requests 2024-03-07T07:28:18.917 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.918 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 17: conflicting requests 2024-03-07T07:28:18.918 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.918 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 18: conflicting requests 2024-03-07T07:28:18.918 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.918 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 19: conflicting requests 2024-03-07T07:28:18.918 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.919 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 20: conflicting requests 2024-03-07T07:28:18.919 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.919 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 21: conflicting requests 2024-03-07T07:28:18.919 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.919 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 22: conflicting requests 2024-03-07T07:28:18.920 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.920 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 23: conflicting requests 2024-03-07T07:28:18.920 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.920 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 24: conflicting requests 2024-03-07T07:28:18.920 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.920 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 25: conflicting requests 2024-03-07T07:28:18.921 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.921 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 26: conflicting requests 2024-03-07T07:28:18.921 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.921 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 27: conflicting requests 2024-03-07T07:28:18.921 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.922 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 28: conflicting requests 2024-03-07T07:28:18.922 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.922 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 29: conflicting requests 2024-03-07T07:28:18.922 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.922 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 30: conflicting requests 2024-03-07T07:28:18.922 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.922 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 31: conflicting requests 2024-03-07T07:28:18.922 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.923 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 32: conflicting requests 2024-03-07T07:28:18.923 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.923 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 33: conflicting requests 2024-03-07T07:28:18.923 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.923 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 34: conflicting requests 2024-03-07T07:28:18.923 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.923 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 35: conflicting requests 2024-03-07T07:28:18.924 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.924 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 36: conflicting requests 2024-03-07T07:28:18.924 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.924 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 37: conflicting requests 2024-03-07T07:28:18.924 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.924 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.924 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.925 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 38: cannot install the best candidate for the job 2024-03-07T07:28:18.925 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.925 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.925 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.925 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.925 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 39: conflicting requests 2024-03-07T07:28:18.925 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.925 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.925 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.925 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.925 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.925 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.925 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 40: cannot install the best candidate for the job 2024-03-07T07:28:18.926 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.926 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.926 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.926 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.926 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 41: cannot install the best candidate for the job 2024-03-07T07:28:18.926 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.926 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.926 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.926 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.926 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 42: cannot install the best candidate for the job 2024-03-07T07:28:18.926 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.926 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.927 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.927 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.927 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 43: cannot install the best candidate for the job 2024-03-07T07:28:18.927 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.927 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.927 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.927 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.927 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 44: cannot install the best candidate for the job 2024-03-07T07:28:18.927 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.927 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.927 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.928 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.928 INFO:teuthology.orchestra.run.smithi160.stderr: Problem 45: cannot install the best candidate for the job 2024-03-07T07:28:18.928 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.928 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.928 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-03-07T07:28:18.928 INFO:teuthology.orchestra.run.smithi160.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-03-07T07:28:19.083 INFO:teuthology.orchestra.run.smithi178.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-03-07T07:28:19.421 INFO:teuthology.orchestra.run.smithi160.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-07T07:28:19.422 INFO:teuthology.orchestra.run.smithi160.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-07T07:28:19.435 INFO:teuthology.orchestra.run.smithi160.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-07T07:28:19.436 INFO:teuthology.orchestra.run.smithi160.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-07T07:28:19.437 INFO:teuthology.orchestra.run.smithi160.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-07T07:28:19.438 INFO:teuthology.orchestra.run.smithi160.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-07T07:28:19.439 INFO:teuthology.orchestra.run.smithi160.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-07T07:28:19.442 INFO:teuthology.orchestra.run.smithi160.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-07T07:28:19.443 INFO:teuthology.orchestra.run.smithi160.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-07T07:28:19.534 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-03-07T07:28:19.546 INFO:teuthology.orchestra.run.smithi160.stdout:========================================================================================= 2024-03-07T07:28:19.546 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repository Size 2024-03-07T07:28:19.546 INFO:teuthology.orchestra.run.smithi160.stdout:========================================================================================= 2024-03-07T07:28:19.546 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2024-03-07T07:28:19.546 INFO:teuthology.orchestra.run.smithi160.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-03-07T07:28:19.546 INFO:teuthology.orchestra.run.smithi160.stdout: ceph x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 6.6 k 2024-03-07T07:28:19.547 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-base x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 5.3 M 2024-03-07T07:28:19.547 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-fuse x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 889 k 2024-03-07T07:28:19.547 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 145 k 2024-03-07T07:28:19.547 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 1.5 M 2024-03-07T07:28:19.547 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 138 k 2024-03-07T07:28:19.547 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 1.8 M 2024-03-07T07:28:19.547 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 7.4 M 2024-03-07T07:28:19.547 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-rook noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 49 k 2024-03-07T07:28:19.547 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-radosgw x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 9.8 M 2024-03-07T07:28:19.547 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-test x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 47 M 2024-03-07T07:28:19.547 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-volume noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 264 k 2024-03-07T07:28:19.547 INFO:teuthology.orchestra.run.smithi160.stdout: cephadm noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 748 k 2024-03-07T07:28:19.547 INFO:teuthology.orchestra.run.smithi160.stdout: cephfs-mirror x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 220 k 2024-03-07T07:28:19.548 INFO:teuthology.orchestra.run.smithi160.stdout: cephfs-shell x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 27 k 2024-03-07T07:28:19.548 INFO:teuthology.orchestra.run.smithi160.stdout: cephfs-top noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 18 k 2024-03-07T07:28:19.548 INFO:teuthology.orchestra.run.smithi160.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-07T07:28:19.548 INFO:teuthology.orchestra.run.smithi160.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-03-07T07:28:19.548 INFO:teuthology.orchestra.run.smithi160.stdout: libaio-devel x86_64 0.3.112-1.el8 CentOS-BaseOS 18 k 2024-03-07T07:28:19.548 INFO:teuthology.orchestra.run.smithi160.stdout: libcephfs-devel x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 31 k 2024-03-07T07:28:19.548 INFO:teuthology.orchestra.run.smithi160.stdout: libcephfs2 x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 746 k 2024-03-07T07:28:19.548 INFO:teuthology.orchestra.run.smithi160.stdout: librados-devel x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 127 k 2024-03-07T07:28:19.548 INFO:teuthology.orchestra.run.smithi160.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-03-07T07:28:19.548 INFO:teuthology.orchestra.run.smithi160.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-07T07:28:19.548 INFO:teuthology.orchestra.run.smithi160.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-07T07:28:19.548 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql x86_64 13.11-1.el9 appstream 1.6 M 2024-03-07T07:28:19.548 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-contrib x86_64 13.11-1.el9 appstream 905 k 2024-03-07T07:28:19.549 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-server x86_64 13.11-1.el9 appstream 5.7 M 2024-03-07T07:28:19.549 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cephfs x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 161 k 2024-03-07T07:28:19.549 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rados x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 320 k 2024-03-07T07:28:19.549 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rbd x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 298 k 2024-03-07T07:28:19.549 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rgw x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 99 k 2024-03-07T07:28:19.549 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-fuse x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 85 k 2024-03-07T07:28:19.549 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-mirror x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 3.1 M 2024-03-07T07:28:19.549 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-nbd x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 170 k 2024-03-07T07:28:19.549 INFO:teuthology.orchestra.run.smithi160.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-07T07:28:19.549 INFO:teuthology.orchestra.run.smithi160.stdout:Upgrading: 2024-03-07T07:28:19.549 INFO:teuthology.orchestra.run.smithi160.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-07T07:28:19.549 INFO:teuthology.orchestra.run.smithi160.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-07T07:28:19.549 INFO:teuthology.orchestra.run.smithi160.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-07T07:28:19.549 INFO:teuthology.orchestra.run.smithi160.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-07T07:28:19.550 INFO:teuthology.orchestra.run.smithi160.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-07T07:28:19.550 INFO:teuthology.orchestra.run.smithi160.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-07T07:28:19.550 INFO:teuthology.orchestra.run.smithi160.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-07T07:28:19.550 INFO:teuthology.orchestra.run.smithi160.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-07T07:28:19.550 INFO:teuthology.orchestra.run.smithi160.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-07T07:28:19.550 INFO:teuthology.orchestra.run.smithi160.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-07T07:28:19.550 INFO:teuthology.orchestra.run.smithi160.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-07T07:28:19.550 INFO:teuthology.orchestra.run.smithi160.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-07T07:28:19.550 INFO:teuthology.orchestra.run.smithi160.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-07T07:28:19.550 INFO:teuthology.orchestra.run.smithi160.stdout: librados2 x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 3.4 M 2024-03-07T07:28:19.550 INFO:teuthology.orchestra.run.smithi160.stdout: librbd1 x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 3.1 M 2024-03-07T07:28:19.550 INFO:teuthology.orchestra.run.smithi160.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-07T07:28:19.550 INFO:teuthology.orchestra.run.smithi160.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-07T07:28:19.551 INFO:teuthology.orchestra.run.smithi160.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-07T07:28:19.551 INFO:teuthology.orchestra.run.smithi160.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-07T07:28:19.551 INFO:teuthology.orchestra.run.smithi160.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-07T07:28:19.551 INFO:teuthology.orchestra.run.smithi160.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-07T07:28:19.551 INFO:teuthology.orchestra.run.smithi160.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-07T07:28:19.551 INFO:teuthology.orchestra.run.smithi160.stdout: xfsdump x86_64 3.1.12-4.el9 baseos 323 k 2024-03-07T07:28:19.551 INFO:teuthology.orchestra.run.smithi160.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-07T07:28:19.551 INFO:teuthology.orchestra.run.smithi160.stdout:Installing dependencies: 2024-03-07T07:28:19.551 INFO:teuthology.orchestra.run.smithi160.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-07T07:28:19.551 INFO:teuthology.orchestra.run.smithi160.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-03-07T07:28:19.551 INFO:teuthology.orchestra.run.smithi160.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-03-07T07:28:19.551 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-common x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 20 M 2024-03-07T07:28:19.551 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 31 k 2024-03-07T07:28:19.551 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mds x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 2.3 M 2024-03-07T07:28:19.552 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 248 k 2024-03-07T07:28:19.552 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mon x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 4.7 M 2024-03-07T07:28:19.552 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-osd x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 17 M 2024-03-07T07:28:19.552 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 16 k 2024-03-07T07:28:19.552 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-selinux x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 25 k 2024-03-07T07:28:19.552 INFO:teuthology.orchestra.run.smithi160.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-03-07T07:28:19.552 INFO:teuthology.orchestra.run.smithi160.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-03-07T07:28:19.552 INFO:teuthology.orchestra.run.smithi160.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-03-07T07:28:19.552 INFO:teuthology.orchestra.run.smithi160.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-03-07T07:28:19.552 INFO:teuthology.orchestra.run.smithi160.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-07T07:28:19.552 INFO:teuthology.orchestra.run.smithi160.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-07T07:28:19.553 INFO:teuthology.orchestra.run.smithi160.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-07T07:28:19.553 INFO:teuthology.orchestra.run.smithi160.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-07T07:28:19.553 INFO:teuthology.orchestra.run.smithi160.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-07T07:28:19.553 INFO:teuthology.orchestra.run.smithi160.stdout: libcephsqlite x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 163 k 2024-03-07T07:28:19.553 INFO:teuthology.orchestra.run.smithi160.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-07T07:28:19.553 INFO:teuthology.orchestra.run.smithi160.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-07T07:28:19.553 INFO:teuthology.orchestra.run.smithi160.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-03-07T07:28:19.553 INFO:teuthology.orchestra.run.smithi160.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-07T07:28:19.553 INFO:teuthology.orchestra.run.smithi160.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-03-07T07:28:19.553 INFO:teuthology.orchestra.run.smithi160.stdout: libradosstriper1 x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 501 k 2024-03-07T07:28:19.553 INFO:teuthology.orchestra.run.smithi160.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-03-07T07:28:19.553 INFO:teuthology.orchestra.run.smithi160.stdout: librgw2 x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 5.1 M 2024-03-07T07:28:19.553 INFO:teuthology.orchestra.run.smithi160.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-07T07:28:19.553 INFO:teuthology.orchestra.run.smithi160.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-03-07T07:28:19.554 INFO:teuthology.orchestra.run.smithi160.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-03-07T07:28:19.554 INFO:teuthology.orchestra.run.smithi160.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-03-07T07:28:19.554 INFO:teuthology.orchestra.run.smithi160.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-07T07:28:19.554 INFO:teuthology.orchestra.run.smithi160.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-07T07:28:19.554 INFO:teuthology.orchestra.run.smithi160.stdout: oniguruma x86_64 6.9.6-1.el9.5 baseos 218 k 2024-03-07T07:28:19.554 INFO:teuthology.orchestra.run.smithi160.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-03-07T07:28:19.554 INFO:teuthology.orchestra.run.smithi160.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-03-07T07:28:19.554 INFO:teuthology.orchestra.run.smithi160.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-07T07:28:19.554 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-private-libs x86_64 13.11-1.el9 appstream 135 k 2024-03-07T07:28:19.554 INFO:teuthology.orchestra.run.smithi160.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-03-07T07:28:19.554 INFO:teuthology.orchestra.run.smithi160.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-07T07:28:19.554 INFO:teuthology.orchestra.run.smithi160.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-07T07:28:19.554 INFO:teuthology.orchestra.run.smithi160.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-03-07T07:28:19.554 INFO:teuthology.orchestra.run.smithi160.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-03-07T07:28:19.555 INFO:teuthology.orchestra.run.smithi160.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-07T07:28:19.555 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-07T07:28:19.555 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ceph-argparse x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 45 k 2024-03-07T07:28:19.555 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ceph-common x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 131 k 2024-03-07T07:28:19.555 INFO:teuthology.orchestra.run.smithi160.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-07T07:28:19.555 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-03-07T07:28:19.555 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-07T07:28:19.555 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-07T07:28:19.555 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-07T07:28:19.555 INFO:teuthology.orchestra.run.smithi160.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-07T07:28:19.555 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-03-07T07:28:19.555 INFO:teuthology.orchestra.run.smithi160.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-07T07:28:19.555 INFO:teuthology.orchestra.run.smithi160.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-07T07:28:19.556 INFO:teuthology.orchestra.run.smithi160.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-07T07:28:19.556 INFO:teuthology.orchestra.run.smithi160.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-07T07:28:19.556 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-07T07:28:19.556 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-07T07:28:19.556 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-07T07:28:19.556 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-07T07:28:19.556 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-07T07:28:19.556 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-07T07:28:19.556 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-03-07T07:28:19.556 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-03-07T07:28:19.556 INFO:teuthology.orchestra.run.smithi160.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-07T07:28:19.557 INFO:teuthology.orchestra.run.smithi160.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-07T07:28:19.557 INFO:teuthology.orchestra.run.smithi160.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-03-07T07:28:19.557 INFO:teuthology.orchestra.run.smithi160.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-03-07T07:28:19.557 INFO:teuthology.orchestra.run.smithi160.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-07T07:28:19.557 INFO:teuthology.orchestra.run.smithi160.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-07T07:28:19.557 INFO:teuthology.orchestra.run.smithi160.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-03-07T07:28:19.557 INFO:teuthology.orchestra.run.smithi160.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-03-07T07:28:19.557 INFO:teuthology.orchestra.run.smithi160.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-03-07T07:28:19.557 INFO:teuthology.orchestra.run.smithi160.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-03-07T07:28:19.557 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-07T07:28:19.557 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-03-07T07:28:19.557 INFO:teuthology.orchestra.run.smithi160.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-07T07:28:19.557 INFO:teuthology.orchestra.run.smithi160.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-03-07T07:28:19.557 INFO:teuthology.orchestra.run.smithi160.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-03-07T07:28:19.558 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-07T07:28:19.558 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-03-07T07:28:19.558 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-03-07T07:28:19.558 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-03-07T07:28:19.558 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-07T07:28:19.558 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-03-07T07:28:19.558 INFO:teuthology.orchestra.run.smithi160.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-07T07:28:19.558 INFO:teuthology.orchestra.run.smithi160.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-07T07:28:19.558 INFO:teuthology.orchestra.run.smithi160.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-03-07T07:28:19.558 INFO:teuthology.orchestra.run.smithi160.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-07T07:28:19.558 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-07T07:28:19.558 INFO:teuthology.orchestra.run.smithi160.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-03-07T07:28:19.558 INFO:teuthology.orchestra.run.smithi160.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-07T07:28:19.559 INFO:teuthology.orchestra.run.smithi160.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-03-07T07:28:19.559 INFO:teuthology.orchestra.run.smithi160.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-07T07:28:19.559 INFO:teuthology.orchestra.run.smithi160.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-03-07T07:28:19.559 INFO:teuthology.orchestra.run.smithi160.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-03-07T07:28:19.559 INFO:teuthology.orchestra.run.smithi160.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-07T07:28:19.559 INFO:teuthology.orchestra.run.smithi160.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-07T07:28:19.559 INFO:teuthology.orchestra.run.smithi160.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-07T07:28:19.559 INFO:teuthology.orchestra.run.smithi160.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-07T07:28:19.559 INFO:teuthology.orchestra.run.smithi160.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-07T07:28:19.559 INFO:teuthology.orchestra.run.smithi160.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-07T07:28:19.559 INFO:teuthology.orchestra.run.smithi160.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-07T07:28:19.559 INFO:teuthology.orchestra.run.smithi160.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-03-07T07:28:19.559 INFO:teuthology.orchestra.run.smithi160.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-07T07:28:19.559 INFO:teuthology.orchestra.run.smithi160.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-03-07T07:28:19.560 INFO:teuthology.orchestra.run.smithi160.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-03-07T07:28:19.560 INFO:teuthology.orchestra.run.smithi160.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2024-03-07T07:28:19.560 INFO:teuthology.orchestra.run.smithi160.stdout:Installing weak dependencies: 2024-03-07T07:28:19.560 INFO:teuthology.orchestra.run.smithi160.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-07T07:28:19.560 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:28:19.560 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2024-03-07T07:28:19.560 INFO:teuthology.orchestra.run.smithi160.stdout:========================================================================================= 2024-03-07T07:28:19.560 INFO:teuthology.orchestra.run.smithi160.stdout:Install 150 Packages 2024-03-07T07:28:19.560 INFO:teuthology.orchestra.run.smithi160.stdout:Upgrade 24 Packages 2024-03-07T07:28:19.560 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:28:19.561 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 226 M 2024-03-07T07:28:19.561 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading Packages: 2024-03-07T07:28:20.410 INFO:teuthology.orchestra.run.smithi160.stdout:(1/174): ceph-19.0.0-1848.g8b896a9e.el9.x86_64. 26 kB/s | 6.6 kB 00:00 2024-03-07T07:28:21.360 INFO:teuthology.orchestra.run.smithi160.stdout:(2/174): ceph-common-19.0.0-1848.g8b896a9e.el9. 17 MB/s | 20 MB 00:01 2024-03-07T07:28:21.402 INFO:teuthology.orchestra.run.smithi160.stdout:(3/174): ceph-fuse-19.0.0-1848.g8b896a9e.el9.x8 896 kB/s | 889 kB 00:00 2024-03-07T07:28:21.427 INFO:teuthology.orchestra.run.smithi160.stdout:(4/174): ceph-immutable-object-cache-19.0.0-184 2.1 MB/s | 145 kB 00:00 2024-03-07T07:28:21.528 INFO:teuthology.orchestra.run.smithi160.stdout:(5/174): ceph-mgr-19.0.0-1848.g8b896a9e.el9.x86 15 MB/s | 1.5 MB 00:00 2024-03-07T07:28:21.609 INFO:teuthology.orchestra.run.smithi178.stderr:Modular dependency problems: 2024-03-07T07:28:21.610 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-03-07T07:28:21.610 INFO:teuthology.orchestra.run.smithi178.stderr: Problem 1: conflicting requests 2024-03-07T07:28:21.610 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240227110532:20125149.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.610 INFO:teuthology.orchestra.run.smithi178.stderr: Problem 2: conflicting requests 2024-03-07T07:28:21.610 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.610 INFO:teuthology.orchestra.run.smithi178.stderr: Problem 3: conflicting requests 2024-03-07T07:28:21.610 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.610 INFO:teuthology.orchestra.run.smithi178.stderr: Problem 4: conflicting requests 2024-03-07T07:28:21.610 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.610 INFO:teuthology.orchestra.run.smithi178.stderr: Problem 5: conflicting requests 2024-03-07T07:28:21.611 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.611 INFO:teuthology.orchestra.run.smithi178.stderr: Problem 6: conflicting requests 2024-03-07T07:28:21.611 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.611 INFO:teuthology.orchestra.run.smithi178.stderr: Problem 7: conflicting requests 2024-03-07T07:28:21.611 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240222150954:49cc9d1b.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.611 INFO:teuthology.orchestra.run.smithi178.stderr: Problem 8: conflicting requests 2024-03-07T07:28:21.611 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.611 INFO:teuthology.orchestra.run.smithi178.stderr: Problem 9: conflicting requests 2024-03-07T07:28:21.611 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.611 INFO:teuthology.orchestra.run.smithi178.stderr: Problem 10: conflicting requests 2024-03-07T07:28:21.611 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.611 INFO:teuthology.orchestra.run.smithi178.stderr: Problem 11: conflicting requests 2024-03-07T07:28:21.611 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.612 INFO:teuthology.orchestra.run.smithi178.stderr: Problem 12: conflicting requests 2024-03-07T07:28:21.612 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.612 INFO:teuthology.orchestra.run.smithi178.stderr: Problem 13: conflicting requests 2024-03-07T07:28:21.612 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.612 INFO:teuthology.orchestra.run.smithi178.stderr: Problem 14: conflicting requests 2024-03-07T07:28:21.612 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.612 INFO:teuthology.orchestra.run.smithi178.stderr: Problem 15: conflicting requests 2024-03-07T07:28:21.612 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.612 INFO:teuthology.orchestra.run.smithi178.stderr: Problem 16: conflicting requests 2024-03-07T07:28:21.612 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.612 INFO:teuthology.orchestra.run.smithi178.stderr: Problem 17: conflicting requests 2024-03-07T07:28:21.613 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.613 INFO:teuthology.orchestra.run.smithi178.stderr: Problem 18: conflicting requests 2024-03-07T07:28:21.613 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.613 INFO:teuthology.orchestra.run.smithi178.stderr: Problem 19: conflicting requests 2024-03-07T07:28:21.613 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.613 INFO:teuthology.orchestra.run.smithi178.stderr: Problem 20: conflicting requests 2024-03-07T07:28:21.613 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.613 INFO:teuthology.orchestra.run.smithi178.stderr: Problem 21: conflicting requests 2024-03-07T07:28:21.613 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.613 INFO:teuthology.orchestra.run.smithi178.stderr: Problem 22: conflicting requests 2024-03-07T07:28:21.613 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.613 INFO:teuthology.orchestra.run.smithi178.stderr: Problem 23: conflicting requests 2024-03-07T07:28:21.613 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.614 INFO:teuthology.orchestra.run.smithi178.stderr: Problem 24: conflicting requests 2024-03-07T07:28:21.614 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.614 INFO:teuthology.orchestra.run.smithi178.stderr: Problem 25: conflicting requests 2024-03-07T07:28:21.614 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.614 INFO:teuthology.orchestra.run.smithi178.stderr: Problem 26: conflicting requests 2024-03-07T07:28:21.614 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.614 INFO:teuthology.orchestra.run.smithi178.stderr: Problem 27: conflicting requests 2024-03-07T07:28:21.614 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.615 INFO:teuthology.orchestra.run.smithi178.stderr: Problem 28: conflicting requests 2024-03-07T07:28:21.615 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.615 INFO:teuthology.orchestra.run.smithi178.stderr: Problem 29: conflicting requests 2024-03-07T07:28:21.615 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.615 INFO:teuthology.orchestra.run.smithi178.stderr: Problem 30: conflicting requests 2024-03-07T07:28:21.615 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.615 INFO:teuthology.orchestra.run.smithi178.stderr: Problem 31: conflicting requests 2024-03-07T07:28:21.615 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.615 INFO:teuthology.orchestra.run.smithi178.stderr: Problem 32: conflicting requests 2024-03-07T07:28:21.615 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.615 INFO:teuthology.orchestra.run.smithi178.stderr: Problem 33: conflicting requests 2024-03-07T07:28:21.616 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.616 INFO:teuthology.orchestra.run.smithi178.stderr: Problem 34: conflicting requests 2024-03-07T07:28:21.616 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.616 INFO:teuthology.orchestra.run.smithi178.stderr: Problem 35: conflicting requests 2024-03-07T07:28:21.616 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.616 INFO:teuthology.orchestra.run.smithi178.stderr: Problem 36: conflicting requests 2024-03-07T07:28:21.616 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.616 INFO:teuthology.orchestra.run.smithi178.stderr: Problem 37: conflicting requests 2024-03-07T07:28:21.616 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.616 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.616 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.616 INFO:teuthology.orchestra.run.smithi178.stderr: Problem 38: cannot install the best candidate for the job 2024-03-07T07:28:21.617 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.617 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.617 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.617 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.617 INFO:teuthology.orchestra.run.smithi178.stderr: Problem 39: conflicting requests 2024-03-07T07:28:21.617 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.617 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.617 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.617 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.618 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.618 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.618 INFO:teuthology.orchestra.run.smithi178.stderr: Problem 40: cannot install the best candidate for the job 2024-03-07T07:28:21.618 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.618 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.618 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.618 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.618 INFO:teuthology.orchestra.run.smithi178.stderr: Problem 41: cannot install the best candidate for the job 2024-03-07T07:28:21.618 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.618 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.618 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.618 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.618 INFO:teuthology.orchestra.run.smithi178.stderr: Problem 42: cannot install the best candidate for the job 2024-03-07T07:28:21.619 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.619 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.619 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.619 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.620 INFO:teuthology.orchestra.run.smithi178.stderr: Problem 43: cannot install the best candidate for the job 2024-03-07T07:28:21.620 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.620 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.620 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.620 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.620 INFO:teuthology.orchestra.run.smithi178.stderr: Problem 44: cannot install the best candidate for the job 2024-03-07T07:28:21.620 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.620 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.620 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.620 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.620 INFO:teuthology.orchestra.run.smithi178.stderr: Problem 45: cannot install the best candidate for the job 2024-03-07T07:28:21.621 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.621 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.621 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.621 INFO:teuthology.orchestra.run.smithi178.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-03-07T07:28:21.803 INFO:teuthology.orchestra.run.smithi160.stdout:(6/174): ceph-mon-19.0.0-1848.g8b896a9e.el9.x86 17 MB/s | 4.7 MB 00:00 2024-03-07T07:28:22.004 INFO:teuthology.orchestra.run.smithi178.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-07T07:28:22.005 INFO:teuthology.orchestra.run.smithi178.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-07T07:28:22.018 INFO:teuthology.orchestra.run.smithi178.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-07T07:28:22.019 INFO:teuthology.orchestra.run.smithi178.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-07T07:28:22.020 INFO:teuthology.orchestra.run.smithi178.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-07T07:28:22.021 INFO:teuthology.orchestra.run.smithi178.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-07T07:28:22.023 INFO:teuthology.orchestra.run.smithi178.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-07T07:28:22.026 INFO:teuthology.orchestra.run.smithi178.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-07T07:28:22.027 INFO:teuthology.orchestra.run.smithi178.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-07T07:28:22.119 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-03-07T07:28:22.139 INFO:teuthology.orchestra.run.smithi178.stdout:========================================================================================= 2024-03-07T07:28:22.139 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2024-03-07T07:28:22.139 INFO:teuthology.orchestra.run.smithi178.stdout:========================================================================================= 2024-03-07T07:28:22.139 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-03-07T07:28:22.139 INFO:teuthology.orchestra.run.smithi178.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-03-07T07:28:22.139 INFO:teuthology.orchestra.run.smithi178.stdout: ceph x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 6.6 k 2024-03-07T07:28:22.139 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-base x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 5.3 M 2024-03-07T07:28:22.140 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-fuse x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 889 k 2024-03-07T07:28:22.140 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 145 k 2024-03-07T07:28:22.140 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 1.5 M 2024-03-07T07:28:22.140 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 138 k 2024-03-07T07:28:22.140 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 1.8 M 2024-03-07T07:28:22.140 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 7.4 M 2024-03-07T07:28:22.140 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-rook noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 49 k 2024-03-07T07:28:22.140 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-radosgw x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 9.8 M 2024-03-07T07:28:22.140 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-test x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 47 M 2024-03-07T07:28:22.140 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-volume noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 264 k 2024-03-07T07:28:22.140 INFO:teuthology.orchestra.run.smithi178.stdout: cephadm noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 748 k 2024-03-07T07:28:22.140 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-mirror x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 220 k 2024-03-07T07:28:22.140 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-shell x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 27 k 2024-03-07T07:28:22.140 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-top noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 18 k 2024-03-07T07:28:22.141 INFO:teuthology.orchestra.run.smithi178.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-07T07:28:22.141 INFO:teuthology.orchestra.run.smithi178.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-03-07T07:28:22.141 INFO:teuthology.orchestra.run.smithi178.stdout: libaio-devel x86_64 0.3.112-1.el8 CentOS-BaseOS 18 k 2024-03-07T07:28:22.141 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs-devel x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 31 k 2024-03-07T07:28:22.141 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs2 x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 746 k 2024-03-07T07:28:22.141 INFO:teuthology.orchestra.run.smithi178.stdout: librados-devel x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 127 k 2024-03-07T07:28:22.141 INFO:teuthology.orchestra.run.smithi178.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-03-07T07:28:22.141 INFO:teuthology.orchestra.run.smithi178.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-07T07:28:22.141 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-07T07:28:22.141 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql x86_64 13.11-1.el9 appstream 1.6 M 2024-03-07T07:28:22.141 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-contrib x86_64 13.11-1.el9 appstream 905 k 2024-03-07T07:28:22.141 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-server x86_64 13.11-1.el9 appstream 5.7 M 2024-03-07T07:28:22.142 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cephfs x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 161 k 2024-03-07T07:28:22.142 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rados x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 320 k 2024-03-07T07:28:22.142 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rbd x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 298 k 2024-03-07T07:28:22.142 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rgw x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 99 k 2024-03-07T07:28:22.142 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-fuse x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 85 k 2024-03-07T07:28:22.142 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-mirror x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 3.1 M 2024-03-07T07:28:22.142 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-nbd x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 170 k 2024-03-07T07:28:22.142 INFO:teuthology.orchestra.run.smithi178.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-07T07:28:22.142 INFO:teuthology.orchestra.run.smithi178.stdout:Upgrading: 2024-03-07T07:28:22.142 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-07T07:28:22.142 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-07T07:28:22.142 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-07T07:28:22.142 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-07T07:28:22.142 INFO:teuthology.orchestra.run.smithi178.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-07T07:28:22.143 INFO:teuthology.orchestra.run.smithi178.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-07T07:28:22.143 INFO:teuthology.orchestra.run.smithi178.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-07T07:28:22.143 INFO:teuthology.orchestra.run.smithi178.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-07T07:28:22.143 INFO:teuthology.orchestra.run.smithi178.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-07T07:28:22.143 INFO:teuthology.orchestra.run.smithi178.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-07T07:28:22.143 INFO:teuthology.orchestra.run.smithi178.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-07T07:28:22.143 INFO:teuthology.orchestra.run.smithi178.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-07T07:28:22.143 INFO:teuthology.orchestra.run.smithi178.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-07T07:28:22.143 INFO:teuthology.orchestra.run.smithi178.stdout: librados2 x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 3.4 M 2024-03-07T07:28:22.143 INFO:teuthology.orchestra.run.smithi178.stdout: librbd1 x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 3.1 M 2024-03-07T07:28:22.143 INFO:teuthology.orchestra.run.smithi178.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-07T07:28:22.143 INFO:teuthology.orchestra.run.smithi178.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-07T07:28:22.143 INFO:teuthology.orchestra.run.smithi178.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-07T07:28:22.143 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-07T07:28:22.144 INFO:teuthology.orchestra.run.smithi178.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-07T07:28:22.144 INFO:teuthology.orchestra.run.smithi178.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-07T07:28:22.144 INFO:teuthology.orchestra.run.smithi178.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-07T07:28:22.144 INFO:teuthology.orchestra.run.smithi178.stdout: xfsdump x86_64 3.1.12-4.el9 baseos 323 k 2024-03-07T07:28:22.144 INFO:teuthology.orchestra.run.smithi178.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-07T07:28:22.144 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2024-03-07T07:28:22.144 INFO:teuthology.orchestra.run.smithi178.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-07T07:28:22.144 INFO:teuthology.orchestra.run.smithi178.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-03-07T07:28:22.144 INFO:teuthology.orchestra.run.smithi178.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-03-07T07:28:22.144 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-common x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 20 M 2024-03-07T07:28:22.144 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 31 k 2024-03-07T07:28:22.144 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mds x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 2.3 M 2024-03-07T07:28:22.144 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 248 k 2024-03-07T07:28:22.145 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mon x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 4.7 M 2024-03-07T07:28:22.145 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-osd x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 17 M 2024-03-07T07:28:22.145 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 16 k 2024-03-07T07:28:22.145 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-selinux x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 25 k 2024-03-07T07:28:22.145 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-03-07T07:28:22.145 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-03-07T07:28:22.145 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-03-07T07:28:22.145 INFO:teuthology.orchestra.run.smithi178.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-03-07T07:28:22.145 INFO:teuthology.orchestra.run.smithi178.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-07T07:28:22.145 INFO:teuthology.orchestra.run.smithi178.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-07T07:28:22.145 INFO:teuthology.orchestra.run.smithi178.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-07T07:28:22.145 INFO:teuthology.orchestra.run.smithi178.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-07T07:28:22.145 INFO:teuthology.orchestra.run.smithi178.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-07T07:28:22.145 INFO:teuthology.orchestra.run.smithi178.stdout: libcephsqlite x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 163 k 2024-03-07T07:28:22.146 INFO:teuthology.orchestra.run.smithi178.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-07T07:28:22.146 INFO:teuthology.orchestra.run.smithi178.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-07T07:28:22.146 INFO:teuthology.orchestra.run.smithi178.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-03-07T07:28:22.146 INFO:teuthology.orchestra.run.smithi178.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-07T07:28:22.146 INFO:teuthology.orchestra.run.smithi178.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-03-07T07:28:22.146 INFO:teuthology.orchestra.run.smithi178.stdout: libradosstriper1 x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 501 k 2024-03-07T07:28:22.146 INFO:teuthology.orchestra.run.smithi178.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-03-07T07:28:22.146 INFO:teuthology.orchestra.run.smithi178.stdout: librgw2 x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 5.1 M 2024-03-07T07:28:22.146 INFO:teuthology.orchestra.run.smithi178.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-07T07:28:22.146 INFO:teuthology.orchestra.run.smithi178.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-03-07T07:28:22.146 INFO:teuthology.orchestra.run.smithi178.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-03-07T07:28:22.146 INFO:teuthology.orchestra.run.smithi178.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-03-07T07:28:22.146 INFO:teuthology.orchestra.run.smithi178.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-07T07:28:22.147 INFO:teuthology.orchestra.run.smithi178.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-07T07:28:22.147 INFO:teuthology.orchestra.run.smithi178.stdout: oniguruma x86_64 6.9.6-1.el9.5 baseos 218 k 2024-03-07T07:28:22.147 INFO:teuthology.orchestra.run.smithi178.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-03-07T07:28:22.147 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-03-07T07:28:22.147 INFO:teuthology.orchestra.run.smithi178.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-07T07:28:22.147 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-private-libs x86_64 13.11-1.el9 appstream 135 k 2024-03-07T07:28:22.147 INFO:teuthology.orchestra.run.smithi178.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-03-07T07:28:22.147 INFO:teuthology.orchestra.run.smithi178.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-07T07:28:22.147 INFO:teuthology.orchestra.run.smithi178.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-07T07:28:22.147 INFO:teuthology.orchestra.run.smithi178.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-03-07T07:28:22.147 INFO:teuthology.orchestra.run.smithi178.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-03-07T07:28:22.147 INFO:teuthology.orchestra.run.smithi178.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-07T07:28:22.147 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-07T07:28:22.147 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-argparse x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 45 k 2024-03-07T07:28:22.148 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-common x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 131 k 2024-03-07T07:28:22.148 INFO:teuthology.orchestra.run.smithi178.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-07T07:28:22.148 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-03-07T07:28:22.148 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-07T07:28:22.148 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-07T07:28:22.148 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-07T07:28:22.148 INFO:teuthology.orchestra.run.smithi178.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-07T07:28:22.148 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-03-07T07:28:22.148 INFO:teuthology.orchestra.run.smithi178.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-07T07:28:22.148 INFO:teuthology.orchestra.run.smithi178.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-07T07:28:22.148 INFO:teuthology.orchestra.run.smithi178.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-07T07:28:22.148 INFO:teuthology.orchestra.run.smithi178.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-07T07:28:22.148 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-07T07:28:22.148 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-07T07:28:22.149 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-07T07:28:22.149 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-07T07:28:22.149 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-07T07:28:22.149 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-07T07:28:22.149 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-03-07T07:28:22.149 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-03-07T07:28:22.149 INFO:teuthology.orchestra.run.smithi178.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-07T07:28:22.149 INFO:teuthology.orchestra.run.smithi178.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-07T07:28:22.149 INFO:teuthology.orchestra.run.smithi178.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-03-07T07:28:22.149 INFO:teuthology.orchestra.run.smithi178.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-03-07T07:28:22.149 INFO:teuthology.orchestra.run.smithi178.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-07T07:28:22.149 INFO:teuthology.orchestra.run.smithi178.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-07T07:28:22.149 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-03-07T07:28:22.149 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-03-07T07:28:22.150 INFO:teuthology.orchestra.run.smithi178.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-03-07T07:28:22.150 INFO:teuthology.orchestra.run.smithi178.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-03-07T07:28:22.150 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-07T07:28:22.150 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-03-07T07:28:22.150 INFO:teuthology.orchestra.run.smithi178.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-07T07:28:22.150 INFO:teuthology.orchestra.run.smithi178.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-03-07T07:28:22.150 INFO:teuthology.orchestra.run.smithi178.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-03-07T07:28:22.150 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-07T07:28:22.150 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-03-07T07:28:22.150 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-03-07T07:28:22.150 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-03-07T07:28:22.150 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-07T07:28:22.151 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-03-07T07:28:22.151 INFO:teuthology.orchestra.run.smithi178.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-07T07:28:22.151 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-07T07:28:22.151 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-03-07T07:28:22.151 INFO:teuthology.orchestra.run.smithi178.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-07T07:28:22.151 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-07T07:28:22.151 INFO:teuthology.orchestra.run.smithi178.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-03-07T07:28:22.151 INFO:teuthology.orchestra.run.smithi178.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-07T07:28:22.151 INFO:teuthology.orchestra.run.smithi178.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-03-07T07:28:22.151 INFO:teuthology.orchestra.run.smithi178.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-07T07:28:22.151 INFO:teuthology.orchestra.run.smithi178.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-03-07T07:28:22.151 INFO:teuthology.orchestra.run.smithi178.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-03-07T07:28:22.151 INFO:teuthology.orchestra.run.smithi178.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-07T07:28:22.151 INFO:teuthology.orchestra.run.smithi178.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-07T07:28:22.152 INFO:teuthology.orchestra.run.smithi178.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-07T07:28:22.152 INFO:teuthology.orchestra.run.smithi178.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-07T07:28:22.152 INFO:teuthology.orchestra.run.smithi178.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-07T07:28:22.152 INFO:teuthology.orchestra.run.smithi178.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-07T07:28:22.152 INFO:teuthology.orchestra.run.smithi178.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-07T07:28:22.152 INFO:teuthology.orchestra.run.smithi178.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-03-07T07:28:22.152 INFO:teuthology.orchestra.run.smithi178.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-07T07:28:22.152 INFO:teuthology.orchestra.run.smithi178.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-03-07T07:28:22.152 INFO:teuthology.orchestra.run.smithi178.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-03-07T07:28:22.152 INFO:teuthology.orchestra.run.smithi178.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2024-03-07T07:28:22.152 INFO:teuthology.orchestra.run.smithi178.stdout:Installing weak dependencies: 2024-03-07T07:28:22.152 INFO:teuthology.orchestra.run.smithi178.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-07T07:28:22.152 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-03-07T07:28:22.152 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-03-07T07:28:22.153 INFO:teuthology.orchestra.run.smithi178.stdout:========================================================================================= 2024-03-07T07:28:22.153 INFO:teuthology.orchestra.run.smithi178.stdout:Install 150 Packages 2024-03-07T07:28:22.153 INFO:teuthology.orchestra.run.smithi178.stdout:Upgrade 24 Packages 2024-03-07T07:28:22.153 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-03-07T07:28:22.153 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 226 M 2024-03-07T07:28:22.153 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-03-07T07:28:22.520 INFO:teuthology.orchestra.run.smithi160.stdout:(7/174): ceph-mds-19.0.0-1848.g8b896a9e.el9.x86 2.1 MB/s | 2.3 MB 00:01 2024-03-07T07:28:22.659 INFO:teuthology.orchestra.run.smithi178.stdout:(1/174): ceph-19.0.0-1848.g8b896a9e.el9.x86_64. 32 kB/s | 6.6 kB 00:00 2024-03-07T07:28:22.687 INFO:teuthology.orchestra.run.smithi160.stdout:(8/174): ceph-osd-19.0.0-1848.g8b896a9e.el9.x86 19 MB/s | 17 MB 00:00 2024-03-07T07:28:22.762 INFO:teuthology.orchestra.run.smithi160.stdout:(9/174): ceph-base-19.0.0-1848.g8b896a9e.el9.x8 2.1 MB/s | 5.3 MB 00:02 2024-03-07T07:28:22.787 INFO:teuthology.orchestra.run.smithi160.stdout:(10/174): ceph-selinux-19.0.0-1848.g8b896a9e.el 251 kB/s | 25 kB 00:00 2024-03-07T07:28:22.860 INFO:teuthology.orchestra.run.smithi178.stdout:(2/174): ceph-fuse-19.0.0-1848.g8b896a9e.el9.x8 4.3 MB/s | 889 kB 00:00 2024-03-07T07:28:22.937 INFO:teuthology.orchestra.run.smithi160.stdout:(11/174): cephfs-mirror-19.0.0-1848.g8b896a9e.e 2.2 MB/s | 220 kB 00:00 2024-03-07T07:28:22.961 INFO:teuthology.orchestra.run.smithi178.stdout:(3/174): ceph-immutable-object-cache-19.0.0-184 1.4 MB/s | 145 kB 00:00 2024-03-07T07:28:22.990 INFO:teuthology.orchestra.run.smithi160.stdout:(12/174): cephfs-shell-19.0.0-1848.g8b896a9e.el 271 kB/s | 27 kB 00:00 2024-03-07T07:28:23.105 INFO:teuthology.orchestra.run.smithi160.stdout:(13/174): libcephfs-devel-19.0.0-1848.g8b896a9e 269 kB/s | 31 kB 00:00 2024-03-07T07:28:23.214 INFO:teuthology.orchestra.run.smithi160.stdout:(14/174): libcephfs2-19.0.0-1848.g8b896a9e.el9. 6.7 MB/s | 746 kB 00:00 2024-03-07T07:28:23.264 INFO:teuthology.orchestra.run.smithi160.stdout:(15/174): libcephsqlite-19.0.0-1848.g8b896a9e.e 3.2 MB/s | 163 kB 00:00 2024-03-07T07:28:23.269 INFO:teuthology.orchestra.run.smithi178.stdout:(4/174): ceph-base-19.0.0-1848.g8b896a9e.el9.x8 6.6 MB/s | 5.3 MB 00:00 2024-03-07T07:28:23.311 INFO:teuthology.orchestra.run.smithi178.stdout:(5/174): ceph-mds-19.0.0-1848.g8b896a9e.el9.x86 6.6 MB/s | 2.3 MB 00:00 2024-03-07T07:28:23.331 INFO:teuthology.orchestra.run.smithi160.stdout:(16/174): librados-devel-19.0.0-1848.g8b896a9e. 1.8 MB/s | 127 kB 00:00 2024-03-07T07:28:23.431 INFO:teuthology.orchestra.run.smithi160.stdout:(17/174): libradosstriper1-19.0.0-1848.g8b896a9 4.9 MB/s | 501 kB 00:00 2024-03-07T07:28:23.475 INFO:teuthology.orchestra.run.smithi177.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-03-07T07:28:23.711 INFO:teuthology.orchestra.run.smithi178.stdout:(6/174): ceph-mgr-19.0.0-1848.g8b896a9e.el9.x86 3.4 MB/s | 1.5 MB 00:00 2024-03-07T07:28:23.998 INFO:teuthology.orchestra.run.smithi160.stdout:(18/174): librgw2-19.0.0-1848.g8b896a9e.el9.x86 9.0 MB/s | 5.1 MB 00:00 2024-03-07T07:28:24.011 INFO:teuthology.orchestra.run.smithi178.stdout:(7/174): ceph-mon-19.0.0-1848.g8b896a9e.el9.x86 6.7 MB/s | 4.7 MB 00:00 2024-03-07T07:28:24.048 INFO:teuthology.orchestra.run.smithi160.stdout:(19/174): python3-ceph-argparse-19.0.0-1848.g8b 908 kB/s | 45 kB 00:00 2024-03-07T07:28:24.123 INFO:teuthology.orchestra.run.smithi160.stdout:(20/174): python3-ceph-common-19.0.0-1848.g8b89 1.7 MB/s | 131 kB 00:00 2024-03-07T07:28:24.190 INFO:teuthology.orchestra.run.smithi160.stdout:(21/174): python3-cephfs-19.0.0-1848.g8b896a9e. 2.4 MB/s | 161 kB 00:00 2024-03-07T07:28:24.257 INFO:teuthology.orchestra.run.smithi160.stdout:(22/174): python3-rados-19.0.0-1848.g8b896a9e.e 4.7 MB/s | 320 kB 00:00 2024-03-07T07:28:24.324 INFO:teuthology.orchestra.run.smithi160.stdout:(23/174): python3-rbd-19.0.0-1848.g8b896a9e.el9 4.4 MB/s | 298 kB 00:00 2024-03-07T07:28:24.383 INFO:teuthology.orchestra.run.smithi160.stdout:(24/174): python3-rgw-19.0.0-1848.g8b896a9e.el9 1.7 MB/s | 99 kB 00:00 2024-03-07T07:28:24.449 INFO:teuthology.orchestra.run.smithi160.stdout:(25/174): rbd-fuse-19.0.0-1848.g8b896a9e.el9.x8 1.3 MB/s | 85 kB 00:00 2024-03-07T07:28:24.758 INFO:teuthology.orchestra.run.smithi160.stdout:(26/174): rbd-mirror-19.0.0-1848.g8b896a9e.el9. 10 MB/s | 3.1 MB 00:00 2024-03-07T07:28:24.833 INFO:teuthology.orchestra.run.smithi160.stdout:(27/174): rbd-nbd-19.0.0-1848.g8b896a9e.el9.x86 2.2 MB/s | 170 kB 00:00 2024-03-07T07:28:24.884 INFO:teuthology.orchestra.run.smithi160.stdout:(28/174): ceph-grafana-dashboards-19.0.0-1848.g 632 kB/s | 31 kB 00:00 2024-03-07T07:28:24.950 INFO:teuthology.orchestra.run.smithi160.stdout:(29/174): ceph-mgr-cephadm-19.0.0-1848.g8b896a9 2.0 MB/s | 138 kB 00:00 2024-03-07T07:28:25.092 INFO:teuthology.orchestra.run.smithi160.stdout:(30/174): ceph-radosgw-19.0.0-1848.g8b896a9e.el 3.8 MB/s | 9.8 MB 00:02 2024-03-07T07:28:25.159 INFO:teuthology.orchestra.run.smithi160.stdout:(31/174): ceph-mgr-dashboard-19.0.0-1848.g8b896 8.5 MB/s | 1.8 MB 00:00 2024-03-07T07:28:25.218 INFO:teuthology.orchestra.run.smithi160.stdout:(32/174): ceph-mgr-modules-core-19.0.0-1848.g8b 4.2 MB/s | 248 kB 00:00 2024-03-07T07:28:25.268 INFO:teuthology.orchestra.run.smithi160.stdout:(33/174): ceph-mgr-rook-19.0.0-1848.g8b896a9e.e 990 kB/s | 49 kB 00:00 2024-03-07T07:28:25.269 INFO:teuthology.orchestra.run.smithi178.stdout:(8/174): ceph-radosgw-19.0.0-1848.g8b896a9e.el9 7.8 MB/s | 9.8 MB 00:01 2024-03-07T07:28:25.327 INFO:teuthology.orchestra.run.smithi160.stdout:(34/174): ceph-prometheus-alerts-19.0.0-1848.g8 282 kB/s | 16 kB 00:00 2024-03-07T07:28:25.394 INFO:teuthology.orchestra.run.smithi160.stdout:(35/174): ceph-volume-19.0.0-1848.g8b896a9e.el9 3.9 MB/s | 264 kB 00:00 2024-03-07T07:28:25.469 INFO:teuthology.orchestra.run.smithi160.stdout:(36/174): cephadm-19.0.0-1848.g8b896a9e.el9.noa 9.8 MB/s | 748 kB 00:00 2024-03-07T07:28:25.511 INFO:teuthology.orchestra.run.smithi160.stdout:(37/174): cephfs-top-19.0.0-1848.g8b896a9e.el9. 438 kB/s | 18 kB 00:00 2024-03-07T07:28:25.561 INFO:teuthology.orchestra.run.smithi178.stdout:(9/174): ceph-selinux-19.0.0-1848.g8b896a9e.el9 86 kB/s | 25 kB 00:00 2024-03-07T07:28:25.594 INFO:teuthology.orchestra.run.smithi160.stdout:(38/174): libaio-devel-0.3.112-1.el8.x86_64.rpm 212 kB/s | 18 kB 00:00 2024-03-07T07:28:25.728 INFO:teuthology.orchestra.run.smithi160.stdout:(39/174): fuse-2.9.9-15.el9.x86_64.rpm 603 kB/s | 80 kB 00:00 2024-03-07T07:28:25.811 INFO:teuthology.orchestra.run.smithi160.stdout:(40/174): jq-1.6-16.el9.x86_64.rpm 2.2 MB/s | 187 kB 00:00 2024-03-07T07:28:25.979 INFO:teuthology.orchestra.run.smithi177.stderr:Modular dependency problems: 2024-03-07T07:28:25.979 INFO:teuthology.orchestra.run.smithi177.stderr: 2024-03-07T07:28:25.979 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 1: conflicting requests 2024-03-07T07:28:25.980 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240227110532:20125149.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.980 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 2: conflicting requests 2024-03-07T07:28:25.980 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.980 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 3: conflicting requests 2024-03-07T07:28:25.980 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.980 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 4: conflicting requests 2024-03-07T07:28:25.980 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.980 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 5: conflicting requests 2024-03-07T07:28:25.980 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.980 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 6: conflicting requests 2024-03-07T07:28:25.980 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.980 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 7: conflicting requests 2024-03-07T07:28:25.981 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240222150954:49cc9d1b.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.981 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 8: conflicting requests 2024-03-07T07:28:25.981 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.981 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 9: conflicting requests 2024-03-07T07:28:25.981 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.981 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 10: conflicting requests 2024-03-07T07:28:25.981 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.981 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 11: conflicting requests 2024-03-07T07:28:25.981 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.981 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 12: conflicting requests 2024-03-07T07:28:25.981 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.981 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 13: conflicting requests 2024-03-07T07:28:25.981 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.981 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 14: conflicting requests 2024-03-07T07:28:25.982 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.982 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 15: conflicting requests 2024-03-07T07:28:25.982 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.982 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 16: conflicting requests 2024-03-07T07:28:25.982 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.982 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 17: conflicting requests 2024-03-07T07:28:25.982 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.982 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 18: conflicting requests 2024-03-07T07:28:25.982 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.982 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 19: conflicting requests 2024-03-07T07:28:25.982 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.982 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 20: conflicting requests 2024-03-07T07:28:25.982 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.983 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 21: conflicting requests 2024-03-07T07:28:25.983 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.983 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 22: conflicting requests 2024-03-07T07:28:25.983 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.983 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 23: conflicting requests 2024-03-07T07:28:25.983 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.983 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 24: conflicting requests 2024-03-07T07:28:25.983 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.983 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 25: conflicting requests 2024-03-07T07:28:25.983 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.983 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 26: conflicting requests 2024-03-07T07:28:25.983 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.983 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 27: conflicting requests 2024-03-07T07:28:25.983 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.984 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 28: conflicting requests 2024-03-07T07:28:25.984 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.984 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 29: conflicting requests 2024-03-07T07:28:25.984 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.984 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 30: conflicting requests 2024-03-07T07:28:25.984 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.984 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 31: conflicting requests 2024-03-07T07:28:25.984 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.984 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 32: conflicting requests 2024-03-07T07:28:25.984 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.984 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 33: conflicting requests 2024-03-07T07:28:25.984 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.985 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 34: conflicting requests 2024-03-07T07:28:25.985 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.985 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 35: conflicting requests 2024-03-07T07:28:25.985 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.985 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 36: conflicting requests 2024-03-07T07:28:25.985 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.985 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 37: conflicting requests 2024-03-07T07:28:25.985 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.985 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.985 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.985 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 38: cannot install the best candidate for the job 2024-03-07T07:28:25.986 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.986 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.986 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.986 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.986 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 39: conflicting requests 2024-03-07T07:28:25.986 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.986 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.986 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.986 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.986 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.986 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.986 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 40: cannot install the best candidate for the job 2024-03-07T07:28:25.986 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.987 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.987 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.987 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.987 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 41: cannot install the best candidate for the job 2024-03-07T07:28:25.987 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.987 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.987 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.987 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.987 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 42: cannot install the best candidate for the job 2024-03-07T07:28:25.987 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.987 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.987 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.987 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.987 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 43: cannot install the best candidate for the job 2024-03-07T07:28:25.988 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.988 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.988 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.988 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.988 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 44: cannot install the best candidate for the job 2024-03-07T07:28:25.988 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.988 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.988 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.988 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.988 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 45: cannot install the best candidate for the job 2024-03-07T07:28:25.988 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.988 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.988 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-03-07T07:28:25.989 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-03-07T07:28:26.020 INFO:teuthology.orchestra.run.smithi160.stdout:(41/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 803 kB 00:00 2024-03-07T07:28:26.104 INFO:teuthology.orchestra.run.smithi160.stdout:(42/174): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-03-07T07:28:26.196 INFO:teuthology.orchestra.run.smithi160.stdout:(43/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.3 MB/s | 218 kB 00:00 2024-03-07T07:28:26.279 INFO:teuthology.orchestra.run.smithi160.stdout:(44/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-03-07T07:28:26.387 INFO:teuthology.orchestra.run.smithi177.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-07T07:28:26.388 INFO:teuthology.orchestra.run.smithi177.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-07T07:28:26.401 INFO:teuthology.orchestra.run.smithi177.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-07T07:28:26.402 INFO:teuthology.orchestra.run.smithi177.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-07T07:28:26.403 INFO:teuthology.orchestra.run.smithi177.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-07T07:28:26.406 INFO:teuthology.orchestra.run.smithi177.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-07T07:28:26.406 INFO:teuthology.orchestra.run.smithi177.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-07T07:28:26.409 INFO:teuthology.orchestra.run.smithi177.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-07T07:28:26.410 INFO:teuthology.orchestra.run.smithi177.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-07T07:28:26.503 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-03-07T07:28:26.524 INFO:teuthology.orchestra.run.smithi177.stdout:========================================================================================= 2024-03-07T07:28:26.525 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repository Size 2024-03-07T07:28:26.525 INFO:teuthology.orchestra.run.smithi177.stdout:========================================================================================= 2024-03-07T07:28:26.525 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-03-07T07:28:26.525 INFO:teuthology.orchestra.run.smithi177.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-03-07T07:28:26.525 INFO:teuthology.orchestra.run.smithi177.stdout: ceph x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 6.6 k 2024-03-07T07:28:26.525 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-base x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 5.3 M 2024-03-07T07:28:26.525 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-fuse x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 889 k 2024-03-07T07:28:26.525 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 145 k 2024-03-07T07:28:26.526 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 1.5 M 2024-03-07T07:28:26.526 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 138 k 2024-03-07T07:28:26.526 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 1.8 M 2024-03-07T07:28:26.526 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 7.4 M 2024-03-07T07:28:26.526 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-rook noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 49 k 2024-03-07T07:28:26.526 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-radosgw x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 9.8 M 2024-03-07T07:28:26.526 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-test x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 47 M 2024-03-07T07:28:26.526 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-volume noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 264 k 2024-03-07T07:28:26.526 INFO:teuthology.orchestra.run.smithi177.stdout: cephadm noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 748 k 2024-03-07T07:28:26.526 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-mirror x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 220 k 2024-03-07T07:28:26.526 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-shell x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 27 k 2024-03-07T07:28:26.526 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-top noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 18 k 2024-03-07T07:28:26.526 INFO:teuthology.orchestra.run.smithi177.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-07T07:28:26.527 INFO:teuthology.orchestra.run.smithi177.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-03-07T07:28:26.527 INFO:teuthology.orchestra.run.smithi177.stdout: libaio-devel x86_64 0.3.112-1.el8 CentOS-BaseOS 18 k 2024-03-07T07:28:26.527 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs-devel x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 31 k 2024-03-07T07:28:26.527 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs2 x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 746 k 2024-03-07T07:28:26.527 INFO:teuthology.orchestra.run.smithi177.stdout: librados-devel x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 127 k 2024-03-07T07:28:26.527 INFO:teuthology.orchestra.run.smithi177.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-03-07T07:28:26.527 INFO:teuthology.orchestra.run.smithi177.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-07T07:28:26.527 INFO:teuthology.orchestra.run.smithi177.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-07T07:28:26.527 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql x86_64 13.11-1.el9 appstream 1.6 M 2024-03-07T07:28:26.527 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-contrib x86_64 13.11-1.el9 appstream 905 k 2024-03-07T07:28:26.527 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-server x86_64 13.11-1.el9 appstream 5.7 M 2024-03-07T07:28:26.527 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cephfs x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 161 k 2024-03-07T07:28:26.528 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rados x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 320 k 2024-03-07T07:28:26.528 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rbd x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 298 k 2024-03-07T07:28:26.528 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rgw x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 99 k 2024-03-07T07:28:26.528 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-fuse x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 85 k 2024-03-07T07:28:26.528 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-mirror x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 3.1 M 2024-03-07T07:28:26.528 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-nbd x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 170 k 2024-03-07T07:28:26.528 INFO:teuthology.orchestra.run.smithi177.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-07T07:28:26.528 INFO:teuthology.orchestra.run.smithi177.stdout:Upgrading: 2024-03-07T07:28:26.528 INFO:teuthology.orchestra.run.smithi177.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-07T07:28:26.528 INFO:teuthology.orchestra.run.smithi177.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-07T07:28:26.528 INFO:teuthology.orchestra.run.smithi177.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-07T07:28:26.528 INFO:teuthology.orchestra.run.smithi177.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-07T07:28:26.528 INFO:teuthology.orchestra.run.smithi177.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-07T07:28:26.529 INFO:teuthology.orchestra.run.smithi177.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-07T07:28:26.529 INFO:teuthology.orchestra.run.smithi177.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-07T07:28:26.529 INFO:teuthology.orchestra.run.smithi177.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-07T07:28:26.529 INFO:teuthology.orchestra.run.smithi177.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-07T07:28:26.529 INFO:teuthology.orchestra.run.smithi177.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-07T07:28:26.529 INFO:teuthology.orchestra.run.smithi177.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-07T07:28:26.529 INFO:teuthology.orchestra.run.smithi177.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-07T07:28:26.529 INFO:teuthology.orchestra.run.smithi177.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-07T07:28:26.529 INFO:teuthology.orchestra.run.smithi177.stdout: librados2 x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 3.4 M 2024-03-07T07:28:26.529 INFO:teuthology.orchestra.run.smithi177.stdout: librbd1 x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 3.1 M 2024-03-07T07:28:26.529 INFO:teuthology.orchestra.run.smithi177.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-07T07:28:26.529 INFO:teuthology.orchestra.run.smithi177.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-07T07:28:26.529 INFO:teuthology.orchestra.run.smithi177.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-07T07:28:26.529 INFO:teuthology.orchestra.run.smithi177.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-07T07:28:26.530 INFO:teuthology.orchestra.run.smithi177.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-07T07:28:26.530 INFO:teuthology.orchestra.run.smithi177.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-07T07:28:26.530 INFO:teuthology.orchestra.run.smithi177.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-07T07:28:26.530 INFO:teuthology.orchestra.run.smithi177.stdout: xfsdump x86_64 3.1.12-4.el9 baseos 323 k 2024-03-07T07:28:26.530 INFO:teuthology.orchestra.run.smithi177.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-07T07:28:26.530 INFO:teuthology.orchestra.run.smithi177.stdout:Installing dependencies: 2024-03-07T07:28:26.530 INFO:teuthology.orchestra.run.smithi177.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-07T07:28:26.530 INFO:teuthology.orchestra.run.smithi177.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-03-07T07:28:26.530 INFO:teuthology.orchestra.run.smithi177.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-03-07T07:28:26.530 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-common x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 20 M 2024-03-07T07:28:26.530 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 31 k 2024-03-07T07:28:26.530 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mds x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 2.3 M 2024-03-07T07:28:26.530 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 248 k 2024-03-07T07:28:26.531 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mon x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 4.7 M 2024-03-07T07:28:26.531 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-osd x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 17 M 2024-03-07T07:28:26.531 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 16 k 2024-03-07T07:28:26.531 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-selinux x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 25 k 2024-03-07T07:28:26.531 INFO:teuthology.orchestra.run.smithi177.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-03-07T07:28:26.531 INFO:teuthology.orchestra.run.smithi177.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-03-07T07:28:26.531 INFO:teuthology.orchestra.run.smithi177.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-03-07T07:28:26.531 INFO:teuthology.orchestra.run.smithi177.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-03-07T07:28:26.531 INFO:teuthology.orchestra.run.smithi177.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-07T07:28:26.531 INFO:teuthology.orchestra.run.smithi177.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-07T07:28:26.531 INFO:teuthology.orchestra.run.smithi177.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-07T07:28:26.531 INFO:teuthology.orchestra.run.smithi177.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-07T07:28:26.531 INFO:teuthology.orchestra.run.smithi177.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-07T07:28:26.532 INFO:teuthology.orchestra.run.smithi177.stdout: libcephsqlite x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 163 k 2024-03-07T07:28:26.532 INFO:teuthology.orchestra.run.smithi177.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-07T07:28:26.532 INFO:teuthology.orchestra.run.smithi177.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-07T07:28:26.532 INFO:teuthology.orchestra.run.smithi177.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-03-07T07:28:26.532 INFO:teuthology.orchestra.run.smithi177.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-07T07:28:26.532 INFO:teuthology.orchestra.run.smithi177.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-03-07T07:28:26.532 INFO:teuthology.orchestra.run.smithi177.stdout: libradosstriper1 x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 501 k 2024-03-07T07:28:26.532 INFO:teuthology.orchestra.run.smithi177.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-03-07T07:28:26.532 INFO:teuthology.orchestra.run.smithi177.stdout: librgw2 x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 5.1 M 2024-03-07T07:28:26.532 INFO:teuthology.orchestra.run.smithi177.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-07T07:28:26.533 INFO:teuthology.orchestra.run.smithi177.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-03-07T07:28:26.533 INFO:teuthology.orchestra.run.smithi177.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-03-07T07:28:26.533 INFO:teuthology.orchestra.run.smithi177.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-03-07T07:28:26.533 INFO:teuthology.orchestra.run.smithi177.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-07T07:28:26.533 INFO:teuthology.orchestra.run.smithi177.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-07T07:28:26.533 INFO:teuthology.orchestra.run.smithi177.stdout: oniguruma x86_64 6.9.6-1.el9.5 baseos 218 k 2024-03-07T07:28:26.533 INFO:teuthology.orchestra.run.smithi177.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-03-07T07:28:26.533 INFO:teuthology.orchestra.run.smithi177.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-03-07T07:28:26.533 INFO:teuthology.orchestra.run.smithi177.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-07T07:28:26.533 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-private-libs x86_64 13.11-1.el9 appstream 135 k 2024-03-07T07:28:26.533 INFO:teuthology.orchestra.run.smithi177.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-03-07T07:28:26.533 INFO:teuthology.orchestra.run.smithi177.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-07T07:28:26.534 INFO:teuthology.orchestra.run.smithi177.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-07T07:28:26.534 INFO:teuthology.orchestra.run.smithi177.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-03-07T07:28:26.534 INFO:teuthology.orchestra.run.smithi177.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-03-07T07:28:26.534 INFO:teuthology.orchestra.run.smithi177.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-07T07:28:26.534 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-07T07:28:26.534 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-argparse x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 45 k 2024-03-07T07:28:26.534 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-common x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 131 k 2024-03-07T07:28:26.534 INFO:teuthology.orchestra.run.smithi177.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-07T07:28:26.534 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-03-07T07:28:26.534 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-07T07:28:26.534 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-07T07:28:26.534 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-07T07:28:26.534 INFO:teuthology.orchestra.run.smithi177.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-07T07:28:26.535 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-03-07T07:28:26.535 INFO:teuthology.orchestra.run.smithi177.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-07T07:28:26.535 INFO:teuthology.orchestra.run.smithi177.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-07T07:28:26.535 INFO:teuthology.orchestra.run.smithi177.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-07T07:28:26.535 INFO:teuthology.orchestra.run.smithi177.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-07T07:28:26.535 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-07T07:28:26.535 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-07T07:28:26.535 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-07T07:28:26.535 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-07T07:28:26.535 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-07T07:28:26.535 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-07T07:28:26.535 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-03-07T07:28:26.535 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-03-07T07:28:26.535 INFO:teuthology.orchestra.run.smithi177.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-07T07:28:26.536 INFO:teuthology.orchestra.run.smithi177.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-07T07:28:26.536 INFO:teuthology.orchestra.run.smithi177.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-03-07T07:28:26.536 INFO:teuthology.orchestra.run.smithi177.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-03-07T07:28:26.536 INFO:teuthology.orchestra.run.smithi177.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-07T07:28:26.536 INFO:teuthology.orchestra.run.smithi177.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-07T07:28:26.536 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-03-07T07:28:26.536 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-03-07T07:28:26.536 INFO:teuthology.orchestra.run.smithi177.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-03-07T07:28:26.536 INFO:teuthology.orchestra.run.smithi177.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-03-07T07:28:26.536 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-07T07:28:26.536 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-03-07T07:28:26.536 INFO:teuthology.orchestra.run.smithi177.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-07T07:28:26.536 INFO:teuthology.orchestra.run.smithi177.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-03-07T07:28:26.537 INFO:teuthology.orchestra.run.smithi177.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-03-07T07:28:26.537 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-07T07:28:26.537 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-03-07T07:28:26.537 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-03-07T07:28:26.537 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-03-07T07:28:26.537 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-07T07:28:26.537 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-03-07T07:28:26.537 INFO:teuthology.orchestra.run.smithi177.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-07T07:28:26.537 INFO:teuthology.orchestra.run.smithi177.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-07T07:28:26.537 INFO:teuthology.orchestra.run.smithi177.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-03-07T07:28:26.537 INFO:teuthology.orchestra.run.smithi177.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-07T07:28:26.537 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-07T07:28:26.537 INFO:teuthology.orchestra.run.smithi177.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-03-07T07:28:26.537 INFO:teuthology.orchestra.run.smithi177.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-07T07:28:26.538 INFO:teuthology.orchestra.run.smithi177.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-03-07T07:28:26.538 INFO:teuthology.orchestra.run.smithi177.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-07T07:28:26.538 INFO:teuthology.orchestra.run.smithi177.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-03-07T07:28:26.538 INFO:teuthology.orchestra.run.smithi177.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-03-07T07:28:26.538 INFO:teuthology.orchestra.run.smithi177.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-07T07:28:26.538 INFO:teuthology.orchestra.run.smithi177.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-07T07:28:26.538 INFO:teuthology.orchestra.run.smithi177.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-07T07:28:26.538 INFO:teuthology.orchestra.run.smithi177.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-07T07:28:26.538 INFO:teuthology.orchestra.run.smithi177.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-07T07:28:26.538 INFO:teuthology.orchestra.run.smithi177.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-07T07:28:26.538 INFO:teuthology.orchestra.run.smithi177.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-07T07:28:26.538 INFO:teuthology.orchestra.run.smithi177.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-03-07T07:28:26.538 INFO:teuthology.orchestra.run.smithi177.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-07T07:28:26.538 INFO:teuthology.orchestra.run.smithi177.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-03-07T07:28:26.539 INFO:teuthology.orchestra.run.smithi177.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-03-07T07:28:26.539 INFO:teuthology.orchestra.run.smithi177.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2024-03-07T07:28:26.539 INFO:teuthology.orchestra.run.smithi177.stdout:Installing weak dependencies: 2024-03-07T07:28:26.539 INFO:teuthology.orchestra.run.smithi177.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-07T07:28:26.539 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-07T07:28:26.539 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-03-07T07:28:26.539 INFO:teuthology.orchestra.run.smithi177.stdout:========================================================================================= 2024-03-07T07:28:26.539 INFO:teuthology.orchestra.run.smithi177.stdout:Install 150 Packages 2024-03-07T07:28:26.539 INFO:teuthology.orchestra.run.smithi177.stdout:Upgrade 24 Packages 2024-03-07T07:28:26.539 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-07T07:28:26.540 INFO:teuthology.orchestra.run.smithi160.stdout:(45/174): python3-cryptography-36.0.1-4.el9.x86 4.7 MB/s | 1.2 MB 00:00 2024-03-07T07:28:26.540 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 226 M 2024-03-07T07:28:26.540 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-03-07T07:28:26.588 INFO:teuthology.orchestra.run.smithi160.stdout:(46/174): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-03-07T07:28:26.655 INFO:teuthology.orchestra.run.smithi160.stdout:(47/174): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-03-07T07:28:26.714 INFO:teuthology.orchestra.run.smithi160.stdout:(48/174): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-03-07T07:28:26.798 INFO:teuthology.orchestra.run.smithi160.stdout:(49/174): ceph-mgr-diskprediction-local-19.0.0- 4.4 MB/s | 7.4 MB 00:01 2024-03-07T07:28:26.840 INFO:teuthology.orchestra.run.smithi160.stdout:(50/174): python3-urllib3-1.26.5-5.el9.noarch.r 1.7 MB/s | 215 kB 00:00 2024-03-07T07:28:26.995 INFO:teuthology.orchestra.run.smithi178.stdout:(10/174): ceph-osd-19.0.0-1848.g8b896a9e.el9.x8 5.1 MB/s | 17 MB 00:03 2024-03-07T07:28:27.053 INFO:teuthology.orchestra.run.smithi178.stdout:(11/174): cephfs-mirror-19.0.0-1848.g8b896a9e.e 3.7 MB/s | 220 kB 00:00 2024-03-07T07:28:27.104 INFO:teuthology.orchestra.run.smithi178.stdout:(12/174): cephfs-shell-19.0.0-1848.g8b896a9e.el 542 kB/s | 27 kB 00:00 2024-03-07T07:28:27.146 INFO:teuthology.orchestra.run.smithi178.stdout:(13/174): libcephfs-devel-19.0.0-1848.g8b896a9e 750 kB/s | 31 kB 00:00 2024-03-07T07:28:27.193 INFO:teuthology.orchestra.run.smithi177.stdout:(1/174): ceph-19.0.0-1848.g8b896a9e.el9.x86_64. 39 kB/s | 6.6 kB 00:00 2024-03-07T07:28:27.305 INFO:teuthology.orchestra.run.smithi178.stdout:(14/174): libcephfs2-19.0.0-1848.g8b896a9e.el9. 4.6 MB/s | 746 kB 00:00 2024-03-07T07:28:27.363 INFO:teuthology.orchestra.run.smithi178.stdout:(15/174): libcephsqlite-19.0.0-1848.g8b896a9e.e 2.7 MB/s | 163 kB 00:00 2024-03-07T07:28:27.422 INFO:teuthology.orchestra.run.smithi178.stdout:(16/174): librados-devel-19.0.0-1848.g8b896a9e. 2.1 MB/s | 127 kB 00:00 2024-03-07T07:28:27.482 INFO:teuthology.orchestra.run.smithi160.stdout:(51/174): automake-1.16.2-8.el9.noarch.rpm 1.0 MB/s | 689 kB 00:00 2024-03-07T07:28:27.572 INFO:teuthology.orchestra.run.smithi178.stdout:(17/174): libradosstriper1-19.0.0-1848.g8b896a9 3.3 MB/s | 501 kB 00:00 2024-03-07T07:28:27.626 INFO:teuthology.orchestra.run.smithi177.stdout:(2/174): ceph-fuse-19.0.0-1848.g8b896a9e.el9.x8 2.0 MB/s | 889 kB 00:00 2024-03-07T07:28:27.684 INFO:teuthology.orchestra.run.smithi160.stdout:(52/174): boost-program-options-1.75.0-8.el9.x8 527 kB/s | 107 kB 00:00 2024-03-07T07:28:27.757 INFO:teuthology.orchestra.run.smithi160.stdout:(53/174): bison-3.7.4-5.el9.x86_64.rpm 1.0 MB/s | 943 kB 00:00 2024-03-07T07:28:27.790 INFO:teuthology.orchestra.run.smithi160.stdout:(54/174): elfutils-libelf-devel-0.190-2.el9.x86 203 kB/s | 21 kB 00:00 2024-03-07T07:28:27.818 INFO:teuthology.orchestra.run.smithi177.stdout:(3/174): ceph-immutable-object-cache-19.0.0-184 758 kB/s | 145 kB 00:00 2024-03-07T07:28:27.866 INFO:teuthology.orchestra.run.smithi160.stdout:(55/174): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-03-07T07:28:27.891 INFO:teuthology.orchestra.run.smithi160.stdout:(56/174): flexiblas-3.0.4-8.el9.x86_64.rpm 326 kB/s | 33 kB 00:00 2024-03-07T07:28:28.000 INFO:teuthology.orchestra.run.smithi160.stdout:(57/174): flexiblas-openblas-openmp-3.0.4-8.el9 168 kB/s | 18 kB 00:00 2024-03-07T07:28:28.117 INFO:teuthology.orchestra.run.smithi160.stdout:(58/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-03-07T07:28:28.233 INFO:teuthology.orchestra.run.smithi160.stdout:(59/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 388 kB/s | 45 kB 00:00 2024-03-07T07:28:28.284 INFO:teuthology.orchestra.run.smithi160.stdout:(60/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.2 MB/s | 3.0 MB 00:00 2024-03-07T07:28:28.418 INFO:teuthology.orchestra.run.smithi160.stdout:(61/174): libtool-2.4.6-45.el9.x86_64.rpm 4.3 MB/s | 582 kB 00:00 2024-03-07T07:28:28.459 INFO:teuthology.orchestra.run.smithi160.stdout:(62/174): librdkafka-1.6.1-102.el9.x86_64.rpm 2.9 MB/s | 662 kB 00:00 2024-03-07T07:28:28.535 INFO:teuthology.orchestra.run.smithi160.stdout:(63/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 218 kB/s | 25 kB 00:00 2024-03-07T07:28:28.585 INFO:teuthology.orchestra.run.smithi160.stdout:(64/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 379 kB/s | 47 kB 00:00 2024-03-07T07:28:28.652 INFO:teuthology.orchestra.run.smithi177.stdout:(4/174): ceph-mds-19.0.0-1848.g8b896a9e.el9.x86 2.8 MB/s | 2.3 MB 00:00 2024-03-07T07:28:28.660 INFO:teuthology.orchestra.run.smithi160.stdout:(65/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-03-07T07:28:28.710 INFO:teuthology.orchestra.run.smithi160.stdout:(66/174): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-03-07T07:28:28.722 INFO:teuthology.orchestra.run.smithi178.stdout:(18/174): librgw2-19.0.0-1848.g8b896a9e.el9.x86 4.5 MB/s | 5.1 MB 00:01 2024-03-07T07:28:28.777 INFO:teuthology.orchestra.run.smithi160.stdout:(67/174): openblas-0.3.21-2.el9.x86_64.rpm 301 kB/s | 35 kB 00:00 2024-03-07T07:28:28.781 INFO:teuthology.orchestra.run.smithi178.stdout:(19/174): python3-ceph-argparse-19.0.0-1848.g8b 777 kB/s | 45 kB 00:00 2024-03-07T07:28:28.873 INFO:teuthology.orchestra.run.smithi178.stdout:(20/174): python3-ceph-common-19.0.0-1848.g8b89 1.4 MB/s | 131 kB 00:00 2024-03-07T07:28:28.957 INFO:teuthology.orchestra.run.smithi178.stdout:(21/174): python3-cephfs-19.0.0-1848.g8b896a9e. 1.9 MB/s | 161 kB 00:00 2024-03-07T07:28:28.977 INFO:teuthology.orchestra.run.smithi177.stdout:(5/174): ceph-base-19.0.0-1848.g8b896a9e.el9.x8 2.7 MB/s | 5.3 MB 00:01 2024-03-07T07:28:29.019 INFO:teuthology.orchestra.run.smithi177.stdout:(6/174): ceph-mgr-19.0.0-1848.g8b896a9e.el9.x86 4.1 MB/s | 1.5 MB 00:00 2024-03-07T07:28:29.082 INFO:teuthology.orchestra.run.smithi178.stdout:(22/174): python3-rados-19.0.0-1848.g8b896a9e.e 2.5 MB/s | 320 kB 00:00 2024-03-07T07:28:29.282 INFO:teuthology.orchestra.run.smithi178.stdout:(23/174): ceph-common-19.0.0-1848.g8b896a9e.el9 3.0 MB/s | 20 MB 00:06 2024-03-07T07:28:29.307 INFO:teuthology.orchestra.run.smithi178.stdout:(24/174): python3-rbd-19.0.0-1848.g8b896a9e.el9 1.3 MB/s | 298 kB 00:00 2024-03-07T07:28:29.391 INFO:teuthology.orchestra.run.smithi178.stdout:(25/174): python3-rgw-19.0.0-1848.g8b896a9e.el9 915 kB/s | 99 kB 00:00 2024-03-07T07:28:29.416 INFO:teuthology.orchestra.run.smithi178.stdout:(26/174): rbd-fuse-19.0.0-1848.g8b896a9e.el9.x8 784 kB/s | 85 kB 00:00 2024-03-07T07:28:29.911 INFO:teuthology.orchestra.run.smithi177.stdout:(7/174): ceph-mon-19.0.0-1848.g8b896a9e.el9.x86 5.0 MB/s | 4.7 MB 00:00 2024-03-07T07:28:29.950 INFO:teuthology.orchestra.run.smithi178.stdout:(27/174): rbd-nbd-19.0.0-1848.g8b896a9e.el9.x86 320 kB/s | 170 kB 00:00 2024-03-07T07:28:30.042 INFO:teuthology.orchestra.run.smithi178.stdout:(28/174): ceph-grafana-dashboards-19.0.0-1848.g 344 kB/s | 31 kB 00:00 2024-03-07T07:28:30.134 INFO:teuthology.orchestra.run.smithi178.stdout:(29/174): ceph-mgr-cephadm-19.0.0-1848.g8b896a9 1.5 MB/s | 138 kB 00:00 2024-03-07T07:28:30.609 INFO:teuthology.orchestra.run.smithi178.stdout:(30/174): rbd-mirror-19.0.0-1848.g8b896a9e.el9. 2.5 MB/s | 3.1 MB 00:01 2024-03-07T07:28:30.876 INFO:teuthology.orchestra.run.smithi178.stdout:(31/174): ceph-mgr-dashboard-19.0.0-1848.g8b896 2.4 MB/s | 1.8 MB 00:00 2024-03-07T07:28:31.001 INFO:teuthology.orchestra.run.smithi178.stdout:(32/174): ceph-mgr-modules-core-19.0.0-1848.g8b 1.9 MB/s | 248 kB 00:00 2024-03-07T07:28:31.060 INFO:teuthology.orchestra.run.smithi178.stdout:(33/174): ceph-mgr-rook-19.0.0-1848.g8b896a9e.e 846 kB/s | 49 kB 00:00 2024-03-07T07:28:31.119 INFO:teuthology.orchestra.run.smithi178.stdout:(34/174): ceph-prometheus-alerts-19.0.0-1848.g8 282 kB/s | 16 kB 00:00 2024-03-07T07:28:31.235 INFO:teuthology.orchestra.run.smithi178.stdout:(35/174): ceph-volume-19.0.0-1848.g8b896a9e.el9 2.2 MB/s | 264 kB 00:00 2024-03-07T07:28:31.361 INFO:teuthology.orchestra.run.smithi177.stdout:(8/174): ceph-common-19.0.0-1848.g8b896a9e.el9. 4.7 MB/s | 20 MB 00:04 2024-03-07T07:28:31.461 INFO:teuthology.orchestra.run.smithi178.stdout:(36/174): cephadm-19.0.0-1848.g8b896a9e.el9.noa 3.2 MB/s | 748 kB 00:00 2024-03-07T07:28:31.469 INFO:teuthology.orchestra.run.smithi177.stdout:(9/174): ceph-selinux-19.0.0-1848.g8b896a9e.el9 232 kB/s | 25 kB 00:00 2024-03-07T07:28:31.536 INFO:teuthology.orchestra.run.smithi178.stdout:(37/174): cephfs-top-19.0.0-1848.g8b896a9e.el9. 243 kB/s | 18 kB 00:00 2024-03-07T07:28:31.620 INFO:teuthology.orchestra.run.smithi178.stdout:(38/174): libaio-devel-0.3.112-1.el8.x86_64.rpm 211 kB/s | 18 kB 00:00 2024-03-07T07:28:31.835 INFO:teuthology.orchestra.run.smithi160.stdout:(68/174): openblas-openmp-0.3.21-2.el9.x86_64.r 1.5 MB/s | 4.7 MB 00:03 2024-03-07T07:28:31.853 INFO:teuthology.orchestra.run.smithi177.stdout:(10/174): ceph-radosgw-19.0.0-1848.g8b896a9e.el 5.0 MB/s | 9.8 MB 00:01 2024-03-07T07:28:31.936 INFO:teuthology.orchestra.run.smithi160.stdout:(69/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 1.3 MB/s | 4.1 MB 00:03 2024-03-07T07:28:31.936 INFO:teuthology.orchestra.run.smithi177.stdout:(11/174): cephfs-mirror-19.0.0-1848.g8b896a9e.e 2.6 MB/s | 220 kB 00:00 2024-03-07T07:28:32.112 INFO:teuthology.orchestra.run.smithi178.stdout:(39/174): ceph-test-19.0.0-1848.g8b896a9e.el9.x 7.2 MB/s | 47 MB 00:06 2024-03-07T07:28:32.128 INFO:teuthology.orchestra.run.smithi177.stdout:(12/174): ceph-osd-19.0.0-1848.g8b896a9e.el9.x8 5.4 MB/s | 17 MB 00:03 2024-03-07T07:28:32.153 INFO:teuthology.orchestra.run.smithi177.stdout:(13/174): cephfs-shell-19.0.0-1848.g8b896a9e.el 125 kB/s | 27 kB 00:00 2024-03-07T07:28:32.179 INFO:teuthology.orchestra.run.smithi177.stdout:(14/174): libcephfs-devel-19.0.0-1848.g8b896a9e 627 kB/s | 31 kB 00:00 2024-03-07T07:28:32.246 INFO:teuthology.orchestra.run.smithi177.stdout:(15/174): libcephsqlite-19.0.0-1848.g8b896a9e.e 2.4 MB/s | 163 kB 00:00 2024-03-07T07:28:32.304 INFO:teuthology.orchestra.run.smithi177.stdout:(16/174): libcephfs2-19.0.0-1848.g8b896a9e.el9. 4.8 MB/s | 746 kB 00:00 2024-03-07T07:28:32.329 INFO:teuthology.orchestra.run.smithi177.stdout:(17/174): librados-devel-19.0.0-1848.g8b896a9e. 1.5 MB/s | 127 kB 00:00 2024-03-07T07:28:32.370 INFO:teuthology.orchestra.run.smithi178.stdout:(40/174): fuse-2.9.9-15.el9.x86_64.rpm 107 kB/s | 80 kB 00:00 2024-03-07T07:28:32.388 INFO:teuthology.orchestra.run.smithi177.stdout:(18/174): libradosstriper1-19.0.0-1848.g8b896a9 5.9 MB/s | 501 kB 00:00 2024-03-07T07:28:32.480 INFO:teuthology.orchestra.run.smithi177.stdout:(19/174): python3-ceph-argparse-19.0.0-1848.g8b 493 kB/s | 45 kB 00:00 2024-03-07T07:28:32.572 INFO:teuthology.orchestra.run.smithi177.stdout:(20/174): python3-ceph-common-19.0.0-1848.g8b89 1.4 MB/s | 131 kB 00:00 2024-03-07T07:28:32.621 INFO:teuthology.orchestra.run.smithi178.stdout:(41/174): jq-1.6-16.el9.x86_64.rpm 368 kB/s | 187 kB 00:00 2024-03-07T07:28:32.656 INFO:teuthology.orchestra.run.smithi177.stdout:(21/174): python3-cephfs-19.0.0-1848.g8b896a9e. 1.9 MB/s | 161 kB 00:00 2024-03-07T07:28:32.748 INFO:teuthology.orchestra.run.smithi177.stdout:(22/174): python3-rados-19.0.0-1848.g8b896a9e.e 3.4 MB/s | 320 kB 00:00 2024-03-07T07:28:32.771 INFO:teuthology.orchestra.run.smithi178.stdout:(42/174): ceph-mgr-diskprediction-local-19.0.0- 3.4 MB/s | 7.4 MB 00:02 2024-03-07T07:28:32.805 INFO:teuthology.orchestra.run.smithi178.stdout:(43/174): libgfortran-11.4.1-3.el9.x86_64.rpm 1.8 MB/s | 803 kB 00:00 2024-03-07T07:28:32.830 INFO:teuthology.orchestra.run.smithi178.stdout:(44/174): libquadmath-11.4.1-3.el9.x86_64.rpm 935 kB/s | 195 kB 00:00 2024-03-07T07:28:32.839 INFO:teuthology.orchestra.run.smithi177.stdout:(23/174): python3-rbd-19.0.0-1848.g8b896a9e.el9 3.2 MB/s | 298 kB 00:00 2024-03-07T07:28:32.922 INFO:teuthology.orchestra.run.smithi178.stdout:(45/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.1 MB/s | 253 kB 00:00 2024-03-07T07:28:32.923 INFO:teuthology.orchestra.run.smithi177.stdout:(24/174): python3-rgw-19.0.0-1848.g8b896a9e.el9 1.2 MB/s | 99 kB 00:00 2024-03-07T07:28:33.022 INFO:teuthology.orchestra.run.smithi178.stdout:(46/174): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 106 kB 00:00 2024-03-07T07:28:33.023 INFO:teuthology.orchestra.run.smithi177.stdout:(25/174): librgw2-19.0.0-1848.g8b896a9e.el9.x86 7.4 MB/s | 5.1 MB 00:00 2024-03-07T07:28:33.057 INFO:teuthology.orchestra.run.smithi177.stdout:(26/174): rbd-fuse-19.0.0-1848.g8b896a9e.el9.x8 638 kB/s | 85 kB 00:00 2024-03-07T07:28:33.078 INFO:teuthology.orchestra.run.smithi160.stdout:(70/174): postgresql-contrib-13.11-1.el9.x86_64 793 kB/s | 905 kB 00:01 2024-03-07T07:28:33.122 INFO:teuthology.orchestra.run.smithi178.stdout:(47/174): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 135 kB 00:00 2024-03-07T07:28:33.149 INFO:teuthology.orchestra.run.smithi177.stdout:(27/174): rbd-nbd-19.0.0-1848.g8b896a9e.el9.x86 1.8 MB/s | 170 kB 00:00 2024-03-07T07:28:33.231 INFO:teuthology.orchestra.run.smithi178.stdout:(48/174): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-03-07T07:28:33.249 INFO:teuthology.orchestra.run.smithi177.stdout:(28/174): ceph-grafana-dashboards-19.0.0-1848.g 315 kB/s | 31 kB 00:00 2024-03-07T07:28:33.261 INFO:teuthology.orchestra.run.smithi160.stdout:(71/174): postgresql-private-libs-13.11-1.el9.x 738 kB/s | 135 kB 00:00 2024-03-07T07:28:33.273 INFO:teuthology.orchestra.run.smithi178.stdout:(49/174): python3-cryptography-36.0.1-4.el9.x86 2.8 MB/s | 1.2 MB 00:00 2024-03-07T07:28:33.298 INFO:teuthology.orchestra.run.smithi178.stdout:(50/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 413 kB/s | 218 kB 00:00 2024-03-07T07:28:33.340 INFO:teuthology.orchestra.run.smithi178.stdout:(51/174): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-03-07T07:28:33.366 INFO:teuthology.orchestra.run.smithi177.stdout:(29/174): ceph-mgr-cephadm-19.0.0-1848.g8b896a9 1.2 MB/s | 138 kB 00:00 2024-03-07T07:28:33.474 INFO:teuthology.orchestra.run.smithi178.stdout:(52/174): boost-program-options-1.75.0-8.el9.x8 803 kB/s | 107 kB 00:00 2024-03-07T07:28:33.499 INFO:teuthology.orchestra.run.smithi178.stdout:(53/174): automake-1.16.2-8.el9.noarch.rpm 3.0 MB/s | 689 kB 00:00 2024-03-07T07:28:33.516 INFO:teuthology.orchestra.run.smithi178.stdout:(54/174): elfutils-libelf-devel-0.190-2.el9.x86 515 kB/s | 21 kB 00:00 2024-03-07T07:28:33.550 INFO:teuthology.orchestra.run.smithi178.stdout:(55/174): bison-3.7.4-5.el9.x86_64.rpm 3.7 MB/s | 943 kB 00:00 2024-03-07T07:28:33.567 INFO:teuthology.orchestra.run.smithi178.stdout:(56/174): flexiblas-3.0.4-8.el9.x86_64.rpm 649 kB/s | 33 kB 00:00 2024-03-07T07:28:33.592 INFO:teuthology.orchestra.run.smithi178.stdout:(57/174): flex-2.6.4-9.el9.x86_64.rpm 3.3 MB/s | 313 kB 00:00 2024-03-07T07:28:33.617 INFO:teuthology.orchestra.run.smithi178.stdout:(58/174): flexiblas-openblas-openmp-3.0.4-8.el9 362 kB/s | 18 kB 00:00 2024-03-07T07:28:33.642 INFO:teuthology.orchestra.run.smithi178.stdout:(59/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-03-07T07:28:33.658 INFO:teuthology.orchestra.run.smithi177.stdout:(30/174): rbd-mirror-19.0.0-1848.g8b896a9e.el9. 4.9 MB/s | 3.1 MB 00:00 2024-03-07T07:28:33.668 INFO:teuthology.orchestra.run.smithi178.stdout:(60/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 901 kB/s | 45 kB 00:00 2024-03-07T07:28:33.702 INFO:teuthology.orchestra.run.smithi178.stdout:(61/174): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-03-07T07:28:33.760 INFO:teuthology.orchestra.run.smithi178.stdout:(62/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 14 MB/s | 3.0 MB 00:00 2024-03-07T07:28:33.785 INFO:teuthology.orchestra.run.smithi178.stdout:(63/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 305 kB/s | 25 kB 00:00 2024-03-07T07:28:33.794 INFO:teuthology.orchestra.run.smithi160.stdout:(72/174): postgresql-13.11-1.el9.x86_64.rpm 827 kB/s | 1.6 MB 00:01 2024-03-07T07:28:33.800 INFO:teuthology.orchestra.run.smithi177.stdout:(31/174): ceph-mgr-dashboard-19.0.0-1848.g8b896 4.1 MB/s | 1.8 MB 00:00 2024-03-07T07:28:33.819 INFO:teuthology.orchestra.run.smithi178.stdout:(64/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 806 kB/s | 47 kB 00:00 2024-03-07T07:28:33.845 INFO:teuthology.orchestra.run.smithi178.stdout:(65/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-03-07T07:28:33.870 INFO:teuthology.orchestra.run.smithi178.stdout:(66/174): libtool-2.4.6-45.el9.x86_64.rpm 2.8 MB/s | 582 kB 00:00 2024-03-07T07:28:33.895 INFO:teuthology.orchestra.run.smithi178.stdout:(67/174): openblas-0.3.21-2.el9.x86_64.rpm 702 kB/s | 35 kB 00:00 2024-03-07T07:28:33.900 INFO:teuthology.orchestra.run.smithi177.stdout:(32/174): ceph-mgr-modules-core-19.0.0-1848.g8b 2.4 MB/s | 248 kB 00:00 2024-03-07T07:28:33.920 INFO:teuthology.orchestra.run.smithi178.stdout:(68/174): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-03-07T07:28:34.004 INFO:teuthology.orchestra.run.smithi178.stdout:(69/174): postgresql-13.11-1.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-03-07T07:28:34.034 INFO:teuthology.orchestra.run.smithi177.stdout:(33/174): ceph-mgr-rook-19.0.0-1848.g8b896a9e.e 370 kB/s | 49 kB 00:00 2024-03-07T07:28:34.071 INFO:teuthology.orchestra.run.smithi178.stdout:(70/174): openblas-openmp-0.3.21-2.el9.x86_64.r 24 MB/s | 4.7 MB 00:00 2024-03-07T07:28:34.126 INFO:teuthology.orchestra.run.smithi177.stdout:(34/174): ceph-prometheus-alerts-19.0.0-1848.g8 179 kB/s | 16 kB 00:00 2024-03-07T07:28:34.136 INFO:teuthology.orchestra.run.smithi178.stdout:(71/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 17 MB/s | 4.1 MB 00:00 2024-03-07T07:28:34.162 INFO:teuthology.orchestra.run.smithi178.stdout:(72/174): postgresql-private-libs-13.11-1.el9.x 1.5 MB/s | 135 kB 00:00 2024-03-07T07:28:34.195 INFO:teuthology.orchestra.run.smithi178.stdout:(73/174): postgresql-contrib-13.11-1.el9.x86_64 4.6 MB/s | 905 kB 00:00 2024-03-07T07:28:34.238 INFO:teuthology.orchestra.run.smithi178.stdout:(74/174): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-03-07T07:28:34.259 INFO:teuthology.orchestra.run.smithi177.stdout:(35/174): ceph-volume-19.0.0-1848.g8b896a9e.el9 1.9 MB/s | 264 kB 00:00 2024-03-07T07:28:34.263 INFO:teuthology.orchestra.run.smithi178.stdout:(75/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-03-07T07:28:34.347 INFO:teuthology.orchestra.run.smithi178.stdout:(76/174): postgresql-server-13.11-1.el9.x86_64. 27 MB/s | 5.7 MB 00:00 2024-03-07T07:28:34.372 INFO:teuthology.orchestra.run.smithi178.stdout:(77/174): python3-devel-3.9.17-2.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-03-07T07:28:34.397 INFO:teuthology.orchestra.run.smithi178.stdout:(78/174): python3-jinja2-2.11.3-5.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-03-07T07:28:34.415 INFO:teuthology.orchestra.run.smithi178.stdout:(79/174): python3-jsonpatch-1.21-16.el9.noarch. 606 kB/s | 26 kB 00:00 2024-03-07T07:28:34.431 INFO:teuthology.orchestra.run.smithi178.stdout:(80/174): python3-jsonpointer-2.0-4.el9.noarch. 565 kB/s | 19 kB 00:00 2024-03-07T07:28:34.465 INFO:teuthology.orchestra.run.smithi178.stdout:(81/174): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-03-07T07:28:34.482 INFO:teuthology.orchestra.run.smithi178.stdout:(82/174): python3-markupsafe-1.1.1-12.el9.x86_6 688 kB/s | 35 kB 00:00 2024-03-07T07:28:34.510 INFO:teuthology.orchestra.run.smithi177.stdout:(36/174): cephadm-19.0.0-1848.g8b896a9e.el9.noa 2.9 MB/s | 748 kB 00:00 2024-03-07T07:28:34.558 INFO:teuthology.orchestra.run.smithi178.stdout:(83/174): python3-babel-2.9.1-2.el9.noarch.rpm 19 MB/s | 6.0 MB 00:00 2024-03-07T07:28:34.577 INFO:teuthology.orchestra.run.smithi177.stdout:(37/174): cephfs-top-19.0.0-1848.g8b896a9e.el9. 274 kB/s | 18 kB 00:00 2024-03-07T07:28:34.583 INFO:teuthology.orchestra.run.smithi178.stdout:(84/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-03-07T07:28:34.608 INFO:teuthology.orchestra.run.smithi178.stdout:(85/174): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-03-07T07:28:34.633 INFO:teuthology.orchestra.run.smithi178.stdout:(86/174): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-03-07T07:28:34.650 INFO:teuthology.orchestra.run.smithi178.stdout:(87/174): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-03-07T07:28:34.660 INFO:teuthology.orchestra.run.smithi177.stdout:(38/174): libaio-devel-0.3.112-1.el8.x86_64.rpm 211 kB/s | 18 kB 00:00 2024-03-07T07:28:34.684 INFO:teuthology.orchestra.run.smithi178.stdout:(88/174): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-03-07T07:28:34.709 INFO:teuthology.orchestra.run.smithi178.stdout:(89/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-03-07T07:28:34.734 INFO:teuthology.orchestra.run.smithi178.stdout:(90/174): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-03-07T07:28:34.752 INFO:teuthology.orchestra.run.smithi177.stdout:(39/174): fuse-2.9.9-15.el9.x86_64.rpm 878 kB/s | 80 kB 00:00 2024-03-07T07:28:34.760 INFO:teuthology.orchestra.run.smithi178.stdout:(91/174): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-03-07T07:28:34.785 INFO:teuthology.orchestra.run.smithi178.stdout:(92/174): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-03-07T07:28:34.819 INFO:teuthology.orchestra.run.smithi177.stdout:(40/174): jq-1.6-16.el9.x86_64.rpm 2.7 MB/s | 187 kB 00:00 2024-03-07T07:28:34.860 INFO:teuthology.orchestra.run.smithi178.stdout:(93/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-03-07T07:28:34.886 INFO:teuthology.orchestra.run.smithi178.stdout:(94/174): python3-toml-0.10.2-6.el9.noarch.rpm 415 kB/s | 42 kB 00:00 2024-03-07T07:28:34.902 INFO:teuthology.orchestra.run.smithi177.stdout:(41/174): libgfortran-11.4.1-3.el9.x86_64.rpm 9.4 MB/s | 803 kB 00:00 2024-03-07T07:28:34.911 INFO:teuthology.orchestra.run.smithi178.stdout:(95/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 881 kB/s | 44 kB 00:00 2024-03-07T07:28:34.911 INFO:teuthology.orchestra.run.smithi160.stdout:(73/174): ceph-test-19.0.0-1848.g8b896a9e.el9.x 3.9 MB/s | 47 MB 00:12 2024-03-07T07:28:34.936 INFO:teuthology.orchestra.run.smithi178.stdout:(96/174): qatlib-23.11.0-1.el9.x86_64.rpm 4.2 MB/s | 216 kB 00:00 2024-03-07T07:28:34.945 INFO:teuthology.orchestra.run.smithi177.stdout:(42/174): libquadmath-11.4.1-3.el9.x86_64.rpm 4.6 MB/s | 195 kB 00:00 2024-03-07T07:28:34.962 INFO:teuthology.orchestra.run.smithi178.stdout:(97/174): qatlib-service-23.11.0-1.el9.x86_64.r 693 kB/s | 35 kB 00:00 2024-03-07T07:28:34.987 INFO:teuthology.orchestra.run.smithi177.stdout:(43/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 5.1 MB/s | 218 kB 00:00 2024-03-07T07:28:34.987 INFO:teuthology.orchestra.run.smithi178.stdout:(98/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 885 kB/s | 44 kB 00:00 2024-03-07T07:28:35.012 INFO:teuthology.orchestra.run.smithi178.stdout:(99/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-03-07T07:28:35.037 INFO:teuthology.orchestra.run.smithi177.stdout:(44/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.0 MB/s | 253 kB 00:00 2024-03-07T07:28:35.037 INFO:teuthology.orchestra.run.smithi160.stdout:(74/174): protobuf-3.14.0-13.el9.x86_64.rpm 832 kB/s | 1.0 MB 00:01 2024-03-07T07:28:35.038 INFO:teuthology.orchestra.run.smithi178.stdout:(100/174): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-03-07T07:28:35.063 INFO:teuthology.orchestra.run.smithi178.stdout:(101/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-03-07T07:28:35.088 INFO:teuthology.orchestra.run.smithi178.stdout:(102/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-03-07T07:28:35.112 INFO:teuthology.orchestra.run.smithi177.stdout:(45/174): python3-cryptography-36.0.1-4.el9.x86 16 MB/s | 1.2 MB 00:00 2024-03-07T07:28:35.115 INFO:teuthology.orchestra.run.smithi178.stdout:(103/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 895 kB/s | 45 kB 00:00 2024-03-07T07:28:35.162 INFO:teuthology.orchestra.run.smithi177.stdout:(46/174): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-03-07T07:28:35.204 INFO:teuthology.orchestra.run.smithi177.stdout:(47/174): python3-pycparser-2.20-6.el9.noarch.r 3.2 MB/s | 135 kB 00:00 2024-03-07T07:28:35.214 INFO:teuthology.orchestra.run.smithi178.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 178 kB/s | 22 kB 00:00 2024-03-07T07:28:35.288 INFO:teuthology.orchestra.run.smithi177.stdout:(48/174): ceph-mgr-diskprediction-local-19.0.0- 4.6 MB/s | 7.4 MB 00:01 2024-03-07T07:28:35.313 INFO:teuthology.orchestra.run.smithi177.stdout:(49/174): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-03-07T07:28:35.337 INFO:teuthology.orchestra.run.smithi160.stdout:(75/174): python3-attrs-20.3.0-7.el9.noarch.rpm 206 kB/s | 87 kB 00:00 2024-03-07T07:28:35.414 INFO:teuthology.orchestra.run.smithi178.stdout:(105/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-03-07T07:28:35.422 INFO:teuthology.orchestra.run.smithi177.stdout:(50/174): python3-urllib3-1.26.5-5.el9.noarch.r 1.6 MB/s | 215 kB 00:00 2024-03-07T07:28:35.489 INFO:teuthology.orchestra.run.smithi177.stdout:(51/174): automake-1.16.2-8.el9.noarch.rpm 3.8 MB/s | 689 kB 00:00 2024-03-07T07:28:35.547 INFO:teuthology.orchestra.run.smithi177.stdout:(52/174): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-03-07T07:28:35.564 INFO:teuthology.orchestra.run.smithi178.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.5 MB/s | 548 kB 00:00 2024-03-07T07:28:35.597 INFO:teuthology.orchestra.run.smithi177.stdout:(53/174): elfutils-libelf-devel-0.190-2.el9.x86 431 kB/s | 21 kB 00:00 2024-03-07T07:28:35.621 INFO:teuthology.orchestra.run.smithi160.stdout:(76/174): python3-devel-3.9.17-2.el9.x86_64.rpm 866 kB/s | 245 kB 00:00 2024-03-07T07:28:35.623 INFO:teuthology.orchestra.run.smithi178.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 333 kB/s | 19 kB 00:00 2024-03-07T07:28:35.631 INFO:teuthology.orchestra.run.smithi177.stdout:(54/174): bison-3.7.4-5.el9.x86_64.rpm 4.4 MB/s | 943 kB 00:00 2024-03-07T07:28:35.648 INFO:teuthology.orchestra.run.smithi178.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 1.3 MB/s | 309 kB 00:00 2024-03-07T07:28:35.674 INFO:teuthology.orchestra.run.smithi177.stdout:(55/174): flex-2.6.4-9.el9.x86_64.rpm 4.1 MB/s | 313 kB 00:00 2024-03-07T07:28:35.699 INFO:teuthology.orchestra.run.smithi177.stdout:(56/174): flexiblas-3.0.4-8.el9.x86_64.rpm 486 kB/s | 33 kB 00:00 2024-03-07T07:28:35.707 INFO:teuthology.orchestra.run.smithi178.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 445 kB/s | 26 kB 00:00 2024-03-07T07:28:35.749 INFO:teuthology.orchestra.run.smithi177.stdout:(57/174): flexiblas-openblas-openmp-3.0.4-8.el9 366 kB/s | 18 kB 00:00 2024-03-07T07:28:35.757 INFO:teuthology.orchestra.run.smithi178.stdout:(110/174): protobuf-compiler-3.14.0-13.el9.x86_ 1.3 MB/s | 863 kB 00:00 2024-03-07T07:28:35.783 INFO:teuthology.orchestra.run.smithi178.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 643 kB/s | 49 kB 00:00 2024-03-07T07:28:35.800 INFO:teuthology.orchestra.run.smithi177.stdout:(58/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-03-07T07:28:35.813 INFO:teuthology.orchestra.run.smithi160.stdout:(77/174): python3-jinja2-2.11.3-5.el9.noarch.rp 1.3 MB/s | 248 kB 00:00 2024-03-07T07:28:35.851 INFO:teuthology.orchestra.run.smithi177.stdout:(59/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 904 kB/s | 45 kB 00:00 2024-03-07T07:28:35.871 INFO:teuthology.orchestra.run.smithi178.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 1.7 MB/s | 151 kB 00:00 2024-03-07T07:28:35.913 INFO:teuthology.orchestra.run.smithi160.stdout:(78/174): python3-jsonpatch-1.21-16.el9.noarch. 257 kB/s | 26 kB 00:00 2024-03-07T07:28:35.925 INFO:teuthology.orchestra.run.smithi178.stdout:(113/174): libunwind-1.6.2-1.el9.x86_64.rpm 403 kB/s | 67 kB 00:00 2024-03-07T07:28:35.926 INFO:teuthology.orchestra.run.smithi177.stdout:(60/174): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 662 kB 00:00 2024-03-07T07:28:35.977 INFO:teuthology.orchestra.run.smithi177.stdout:(61/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 9.9 MB/s | 3.0 MB 00:00 2024-03-07T07:28:36.005 INFO:teuthology.orchestra.run.smithi160.stdout:(79/174): python3-jsonpointer-2.0-4.el9.noarch. 209 kB/s | 19 kB 00:00 2024-03-07T07:28:36.027 INFO:teuthology.orchestra.run.smithi177.stdout:(62/174): libtool-2.4.6-45.el9.x86_64.rpm 5.7 MB/s | 582 kB 00:00 2024-03-07T07:28:36.052 INFO:teuthology.orchestra.run.smithi177.stdout:(63/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 338 kB/s | 25 kB 00:00 2024-03-07T07:28:36.077 INFO:teuthology.orchestra.run.smithi177.stdout:(64/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-03-07T07:28:36.084 INFO:teuthology.orchestra.run.smithi178.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.9 MB/s | 838 kB 00:00 2024-03-07T07:28:36.105 INFO:teuthology.orchestra.run.smithi160.stdout:(80/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 172 kB 00:00 2024-03-07T07:28:36.111 INFO:teuthology.orchestra.run.smithi177.stdout:(65/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-03-07T07:28:36.136 INFO:teuthology.orchestra.run.smithi177.stdout:(66/174): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-03-07T07:28:36.159 INFO:teuthology.orchestra.run.smithi178.stdout:(115/174): python3-asyncssh-2.13.2-4.el9.noarch 2.3 MB/s | 548 kB 00:00 2024-03-07T07:28:36.161 INFO:teuthology.orchestra.run.smithi177.stdout:(67/174): openblas-0.3.21-2.el9.x86_64.rpm 702 kB/s | 35 kB 00:00 2024-03-07T07:28:36.184 INFO:teuthology.orchestra.run.smithi178.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 434 kB/s | 43 kB 00:00 2024-03-07T07:28:36.205 INFO:teuthology.orchestra.run.smithi160.stdout:(81/174): python3-markupsafe-1.1.1-12.el9.x86_6 348 kB/s | 35 kB 00:00 2024-03-07T07:28:36.226 INFO:teuthology.orchestra.run.smithi178.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 481 kB/s | 32 kB 00:00 2024-03-07T07:28:36.243 INFO:teuthology.orchestra.run.smithi178.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 255 kB/s | 15 kB 00:00 2024-03-07T07:28:36.293 INFO:teuthology.orchestra.run.smithi178.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-03-07T07:28:36.360 INFO:teuthology.orchestra.run.smithi178.stdout:(120/174): libarrow-9.0.0-10.el9.x86_64.rpm 6.0 MB/s | 4.4 MB 00:00 2024-03-07T07:28:36.427 INFO:teuthology.orchestra.run.smithi178.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 1.9 MB/s | 358 kB 00:00 2024-03-07T07:28:36.437 INFO:teuthology.orchestra.run.smithi177.stdout:(68/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 15 MB/s | 4.1 MB 00:00 2024-03-07T07:28:36.444 INFO:teuthology.orchestra.run.smithi178.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 423 kB/s | 35 kB 00:00 2024-03-07T07:28:36.469 INFO:teuthology.orchestra.run.smithi178.stdout:(123/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.7 MB/s | 303 kB 00:00 2024-03-07T07:28:36.512 INFO:teuthology.orchestra.run.smithi177.stdout:(69/174): openblas-openmp-0.3.21-2.el9.x86_64.r 13 MB/s | 4.7 MB 00:00 2024-03-07T07:28:36.567 INFO:teuthology.orchestra.run.smithi178.stdout:(124/174): python3-google-auth-2.28.0-1.el9.noa 1.4 MB/s | 203 kB 00:00 2024-03-07T07:28:36.592 INFO:teuthology.orchestra.run.smithi178.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.2 MB/s | 144 kB 00:00 2024-03-07T07:28:36.612 INFO:teuthology.orchestra.run.smithi177.stdout:(70/174): postgresql-contrib-13.11-1.el9.x86_64 8.9 MB/s | 905 kB 00:00 2024-03-07T07:28:36.643 INFO:teuthology.orchestra.run.smithi178.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 142 kB/s | 11 kB 00:00 2024-03-07T07:28:36.646 INFO:teuthology.orchestra.run.smithi177.stdout:(71/174): postgresql-13.11-1.el9.x86_64.rpm 7.6 MB/s | 1.6 MB 00:00 2024-03-07T07:28:36.668 INFO:teuthology.orchestra.run.smithi178.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 235 kB/s | 18 kB 00:00 2024-03-07T07:28:36.696 INFO:teuthology.orchestra.run.smithi177.stdout:(72/174): postgresql-private-libs-13.11-1.el9.x 1.6 MB/s | 135 kB 00:00 2024-03-07T07:28:36.718 INFO:teuthology.orchestra.run.smithi178.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 308 kB/s | 23 kB 00:00 2024-03-07T07:28:36.744 INFO:teuthology.orchestra.run.smithi178.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 258 kB/s | 19 kB 00:00 2024-03-07T07:28:36.789 INFO:teuthology.orchestra.run.smithi160.stdout:(82/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 8.7 MB/s | 5.1 MB 00:00 2024-03-07T07:28:36.794 INFO:teuthology.orchestra.run.smithi178.stdout:(130/174): python3-grpcio-1.46.7-10.el9.x86_64. 5.8 MB/s | 2.0 MB 00:00 2024-03-07T07:28:36.796 INFO:teuthology.orchestra.run.smithi177.stdout:(73/174): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-03-07T07:28:36.819 INFO:teuthology.orchestra.run.smithi178.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 195 kB/s | 20 kB 00:00 2024-03-07T07:28:36.847 INFO:teuthology.orchestra.run.smithi177.stdout:(74/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-03-07T07:28:36.852 INFO:teuthology.orchestra.run.smithi178.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 794 kB/s | 46 kB 00:00 2024-03-07T07:28:36.878 INFO:teuthology.orchestra.run.smithi178.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-03-07T07:28:36.903 INFO:teuthology.orchestra.run.smithi178.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-03-07T07:28:36.962 INFO:teuthology.orchestra.run.smithi178.stdout:(135/174): python3-kubernetes-26.1.0-2.el9.noar 4.7 MB/s | 1.0 MB 00:00 2024-03-07T07:28:36.979 INFO:teuthology.orchestra.run.smithi178.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 218 kB/s | 16 kB 00:00 2024-03-07T07:28:37.004 INFO:teuthology.orchestra.run.smithi178.stdout:(137/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-03-07T07:28:37.030 INFO:teuthology.orchestra.run.smithi178.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-03-07T07:28:37.055 INFO:teuthology.orchestra.run.smithi178.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 373 kB/s | 28 kB 00:00 2024-03-07T07:28:37.055 INFO:teuthology.orchestra.run.smithi160.stdout:(83/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 940 kB/s | 250 kB 00:00 2024-03-07T07:28:37.080 INFO:teuthology.orchestra.run.smithi178.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-03-07T07:28:37.105 INFO:teuthology.orchestra.run.smithi178.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-03-07T07:28:37.113 INFO:teuthology.orchestra.run.smithi177.stdout:(75/174): postgresql-server-13.11-1.el9.x86_64. 12 MB/s | 5.7 MB 00:00 2024-03-07T07:28:37.131 INFO:teuthology.orchestra.run.smithi178.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 782 kB/s | 59 kB 00:00 2024-03-07T07:28:37.156 INFO:teuthology.orchestra.run.smithi178.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 473 kB/s | 36 kB 00:00 2024-03-07T07:28:37.164 INFO:teuthology.orchestra.run.smithi177.stdout:(76/174): python3-devel-3.9.17-2.el9.x86_64.rpm 4.8 MB/s | 245 kB 00:00 2024-03-07T07:28:37.181 INFO:teuthology.orchestra.run.smithi178.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 675 kB/s | 51 kB 00:00 2024-03-07T07:28:37.214 INFO:teuthology.orchestra.run.smithi177.stdout:(77/174): python3-jinja2-2.11.3-5.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-03-07T07:28:37.215 INFO:teuthology.orchestra.run.smithi178.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-03-07T07:28:37.240 INFO:teuthology.orchestra.run.smithi178.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-03-07T07:28:37.256 INFO:teuthology.orchestra.run.smithi177.stdout:(78/174): python3-jsonpatch-1.21-16.el9.noarch. 619 kB/s | 26 kB 00:00 2024-03-07T07:28:37.290 INFO:teuthology.orchestra.run.smithi178.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 265 kB/s | 20 kB 00:00 2024-03-07T07:28:37.298 INFO:teuthology.orchestra.run.smithi177.stdout:(79/174): python3-jsonpointer-2.0-4.el9.noarch. 459 kB/s | 19 kB 00:00 2024-03-07T07:28:37.316 INFO:teuthology.orchestra.run.smithi178.stdout:(148/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-03-07T07:28:37.323 INFO:teuthology.orchestra.run.smithi160.stdout:(84/174): python3-oauthlib-3.1.1-5.el9.noarch.r 833 kB/s | 222 kB 00:00 2024-03-07T07:28:37.348 INFO:teuthology.orchestra.run.smithi177.stdout:(80/174): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-03-07T07:28:37.360 INFO:teuthology.orchestra.run.smithi178.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.6 MB/s | 192 kB 00:00 2024-03-07T07:28:37.431 INFO:teuthology.orchestra.run.smithi160.stdout:(85/174): python3-packaging-20.9-5.el9.noarch.r 712 kB/s | 77 kB 00:00 2024-03-07T07:28:37.440 INFO:teuthology.orchestra.run.smithi177.stdout:(81/174): python3-babel-2.9.1-2.el9.noarch.rpm 10 MB/s | 6.0 MB 00:00 2024-03-07T07:28:37.465 INFO:teuthology.orchestra.run.smithi177.stdout:(82/174): python3-markupsafe-1.1.1-12.el9.x86_6 297 kB/s | 35 kB 00:00 2024-03-07T07:28:37.516 INFO:teuthology.orchestra.run.smithi177.stdout:(83/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-03-07T07:28:37.540 INFO:teuthology.orchestra.run.smithi160.stdout:(86/174): python3-prettytable-0.7.2-27.el9.noar 390 kB/s | 42 kB 00:00 2024-03-07T07:28:37.566 INFO:teuthology.orchestra.run.smithi177.stdout:(84/174): python3-oauthlib-3.1.1-5.el9.noarch.r 4.4 MB/s | 222 kB 00:00 2024-03-07T07:28:37.616 INFO:teuthology.orchestra.run.smithi177.stdout:(85/174): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-03-07T07:28:37.663 INFO:teuthology.orchestra.run.smithi178.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 4.3 MB/s | 1.6 MB 00:00 2024-03-07T07:28:37.666 INFO:teuthology.orchestra.run.smithi177.stdout:(86/174): python3-prettytable-0.7.2-27.el9.noar 850 kB/s | 42 kB 00:00 2024-03-07T07:28:37.717 INFO:teuthology.orchestra.run.smithi177.stdout:(87/174): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-03-07T07:28:37.800 INFO:teuthology.orchestra.run.smithi177.stdout:(88/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 159 kB 00:00 2024-03-07T07:28:37.867 INFO:teuthology.orchestra.run.smithi177.stdout:(89/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-03-07T07:28:37.892 INFO:teuthology.orchestra.run.smithi177.stdout:(90/174): python3-pyasn1-modules-0.4.8-6.el9.no 3.0 MB/s | 279 kB 00:00 2024-03-07T07:28:37.918 INFO:teuthology.orchestra.run.smithi177.stdout:(91/174): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-03-07T07:28:37.982 INFO:teuthology.orchestra.run.smithi160.stdout:(87/174): python3-protobuf-3.14.0-13.el9.noarch 609 kB/s | 269 kB 00:00 2024-03-07T07:28:37.985 INFO:teuthology.orchestra.run.smithi177.stdout:(92/174): python3-requests-oauthlib-1.3.0-12.el 582 kB/s | 54 kB 00:00 2024-03-07T07:28:38.035 INFO:teuthology.orchestra.run.smithi177.stdout:(93/174): python3-toml-0.10.2-6.el9.noarch.rpm 834 kB/s | 42 kB 00:00 2024-03-07T07:28:38.086 INFO:teuthology.orchestra.run.smithi177.stdout:(94/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 883 kB/s | 44 kB 00:00 2024-03-07T07:28:38.128 INFO:teuthology.orchestra.run.smithi177.stdout:(95/174): qatlib-23.11.0-1.el9.x86_64.rpm 5.1 MB/s | 216 kB 00:00 2024-03-07T07:28:38.152 INFO:teuthology.orchestra.run.smithi178.stdout:(151/174): librados2-19.0.0-1848.g8b896a9e.el9. 4.1 MB/s | 3.4 MB 00:00 2024-03-07T07:28:38.165 INFO:teuthology.orchestra.run.smithi160.stdout:(88/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 867 kB/s | 159 kB 00:00 2024-03-07T07:28:38.178 INFO:teuthology.orchestra.run.smithi177.stdout:(96/174): qatlib-service-23.11.0-1.el9.x86_64.r 697 kB/s | 35 kB 00:00 2024-03-07T07:28:38.228 INFO:teuthology.orchestra.run.smithi177.stdout:(97/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 892 kB/s | 44 kB 00:00 2024-03-07T07:28:38.279 INFO:teuthology.orchestra.run.smithi177.stdout:(98/174): socat-1.7.4.1-5.el9.x86_64.rpm 6.0 MB/s | 305 kB 00:00 2024-03-07T07:28:38.337 INFO:teuthology.orchestra.run.smithi177.stdout:(99/174): uuid-1.6.2-55.el9.x86_64.rpm 993 kB/s | 58 kB 00:00 2024-03-07T07:28:38.387 INFO:teuthology.orchestra.run.smithi177.stdout:(100/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-03-07T07:28:38.507 INFO:teuthology.orchestra.run.smithi160.stdout:(89/174): python3-pyasn1-modules-0.4.8-6.el9.no 816 kB/s | 279 kB 00:00 2024-03-07T07:28:38.530 INFO:teuthology.orchestra.run.smithi178.stdout:(152/174): NetworkManager-1.46.0-1.el9.x86_64.r 2.6 MB/s | 2.3 MB 00:00 2024-03-07T07:28:38.587 INFO:teuthology.orchestra.run.smithi178.stdout:(153/174): librbd1-19.0.0-1848.g8b896a9e.el9.x8 2.6 MB/s | 3.1 MB 00:01 2024-03-07T07:28:38.624 INFO:teuthology.orchestra.run.smithi160.stdout:(90/174): python3-pytz-2021.1-5.el9.noarch.rpm 437 kB/s | 51 kB 00:00 2024-03-07T07:28:38.636 INFO:teuthology.orchestra.run.smithi178.stdout:(154/174): NetworkManager-team-1.46.0-1.el9.x86 391 kB/s | 42 kB 00:00 2024-03-07T07:28:38.728 INFO:teuthology.orchestra.run.smithi160.stdout:(91/174): python3-requests-oauthlib-1.3.0-12.el 536 kB/s | 54 kB 00:00 2024-03-07T07:28:38.745 INFO:teuthology.orchestra.run.smithi178.stdout:(155/174): acl-2.3.1-4.el9.x86_64.rpm 659 kB/s | 71 kB 00:00 2024-03-07T07:28:38.796 INFO:teuthology.orchestra.run.smithi177.stdout:(101/174): ceph-test-19.0.0-1848.g8b896a9e.el9. 6.5 MB/s | 47 MB 00:07 2024-03-07T07:28:38.822 INFO:teuthology.orchestra.run.smithi177.stdout:(102/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 147 kB/s | 64 kB 00:00 2024-03-07T07:28:38.854 INFO:teuthology.orchestra.run.smithi178.stdout:(156/174): elfutils-debuginfod-client-0.190-2.e 336 kB/s | 36 kB 00:00 2024-03-07T07:28:38.895 INFO:teuthology.orchestra.run.smithi178.stdout:(157/174): NetworkManager-libnm-1.46.0-1.el9.x8 2.4 MB/s | 1.8 MB 00:00 2024-03-07T07:28:38.914 INFO:teuthology.orchestra.run.smithi177.stdout:(103/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 387 kB/s | 45 kB 00:00 2024-03-07T07:28:38.962 INFO:teuthology.orchestra.run.smithi178.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.7 MB/s | 192 kB 00:00 2024-03-07T07:28:39.004 INFO:teuthology.orchestra.run.smithi178.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.3 MB/s | 256 kB 00:00 2024-03-07T07:28:39.062 INFO:teuthology.orchestra.run.smithi178.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 229 kB/s | 23 kB 00:00 2024-03-07T07:28:39.088 INFO:teuthology.orchestra.run.smithi178.stdout:(161/174): NetworkManager-tui-1.46.0-1.el9.x86_ 494 kB/s | 248 kB 00:00 2024-03-07T07:28:39.113 INFO:teuthology.orchestra.run.smithi178.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 994 kB/s | 108 kB 00:00 2024-03-07T07:28:39.172 INFO:teuthology.orchestra.run.smithi178.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.4 MB/s | 155 kB 00:00 2024-03-07T07:28:39.197 INFO:teuthology.orchestra.run.smithi178.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 1.2 MB/s | 136 kB 00:00 2024-03-07T07:28:39.214 INFO:teuthology.orchestra.run.smithi178.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 636 kB/s | 64 kB 00:00 2024-03-07T07:28:39.264 INFO:teuthology.orchestra.run.smithi178.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 311 kB/s | 29 kB 00:00 2024-03-07T07:28:39.372 INFO:teuthology.orchestra.run.smithi177.stdout:(104/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-03-07T07:28:39.439 INFO:teuthology.orchestra.run.smithi178.stdout:(167/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 9.6 MB/s | 2.2 MB 00:00 2024-03-07T07:28:39.556 INFO:teuthology.orchestra.run.smithi178.stdout:(168/174): util-linux-core-2.37.4-18.el9.x86_64 3.9 MB/s | 465 kB 00:00 2024-03-07T07:28:39.563 INFO:teuthology.orchestra.run.smithi177.stdout:(105/174): lua-devel-5.4.4-4.el9.x86_64.rpm 30 kB/s | 22 kB 00:00 2024-03-07T07:28:39.614 INFO:teuthology.orchestra.run.smithi177.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 2.2 MB/s | 548 kB 00:00 2024-03-07T07:28:39.639 INFO:teuthology.orchestra.run.smithi177.stdout:(107/174): gperftools-libs-2.9.1-2.el9.x86_64.r 4.1 MB/s | 309 kB 00:00 2024-03-07T07:28:39.665 INFO:teuthology.orchestra.run.smithi178.stdout:(169/174): util-linux-user-2.37.4-18.el9.x86_64 287 kB/s | 31 kB 00:00 2024-03-07T07:28:39.672 INFO:teuthology.orchestra.run.smithi177.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 337 kB/s | 19 kB 00:00 2024-03-07T07:28:39.698 INFO:teuthology.orchestra.run.smithi178.stdout:(170/174): openssl-3.0.7-27.el9.x86_64.rpm 2.4 MB/s | 1.2 MB 00:00 2024-03-07T07:28:39.723 INFO:teuthology.orchestra.run.smithi177.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 526 kB/s | 26 kB 00:00 2024-03-07T07:28:39.773 INFO:teuthology.orchestra.run.smithi177.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 985 kB/s | 49 kB 00:00 2024-03-07T07:28:39.782 INFO:teuthology.orchestra.run.smithi178.stdout:(171/174): xfsdump-3.1.12-4.el9.x86_64.rpm 2.7 MB/s | 323 kB 00:00 2024-03-07T07:28:39.857 INFO:teuthology.orchestra.run.smithi177.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 818 kB/s | 67 kB 00:00 2024-03-07T07:28:39.865 INFO:teuthology.orchestra.run.smithi178.stdout:(172/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 892 kB/s | 74 kB 00:00 2024-03-07T07:28:39.899 INFO:teuthology.orchestra.run.smithi178.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 5.4 MB/s | 1.1 MB 00:00 2024-03-07T07:28:39.940 INFO:teuthology.orchestra.run.smithi177.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-03-07T07:28:40.057 INFO:teuthology.orchestra.run.smithi177.stdout:(113/174): libarrow-9.0.0-10.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-03-07T07:28:40.100 INFO:teuthology.orchestra.run.smithi177.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 5.2 MB/s | 838 kB 00:00 2024-03-07T07:28:40.141 INFO:teuthology.orchestra.run.smithi178.stdout:(174/174): util-linux-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 2.3 MB 00:00 2024-03-07T07:28:40.144 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-03-07T07:28:40.144 INFO:teuthology.orchestra.run.smithi178.stdout:Total 13 MB/s | 226 MB 00:17 2024-03-07T07:28:40.149 INFO:teuthology.orchestra.run.smithi177.stdout:(115/174): python3-asyncssh-2.13.2-4.el9.noarch 5.9 MB/s | 548 kB 00:00 2024-03-07T07:28:40.175 INFO:teuthology.orchestra.run.smithi177.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 584 kB/s | 43 kB 00:00 2024-03-07T07:28:40.200 INFO:teuthology.orchestra.run.smithi177.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 643 kB/s | 32 kB 00:00 2024-03-07T07:28:40.225 INFO:teuthology.orchestra.run.smithi177.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 300 kB/s | 15 kB 00:00 2024-03-07T07:28:40.250 INFO:teuthology.orchestra.run.smithi177.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-03-07T07:28:40.266 INFO:teuthology.orchestra.run.smithi160.stdout:(92/174): postgresql-server-13.11-1.el9.x86_64. 840 kB/s | 5.7 MB 00:07 2024-03-07T07:28:40.276 INFO:teuthology.orchestra.run.smithi177.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 7.1 MB/s | 358 kB 00:00 2024-03-07T07:28:40.301 INFO:teuthology.orchestra.run.smithi177.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 6.0 MB/s | 303 kB 00:00 2024-03-07T07:28:40.327 INFO:teuthology.orchestra.run.smithi177.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 711 kB/s | 35 kB 00:00 2024-03-07T07:28:40.375 INFO:teuthology.orchestra.run.smithi160.stdout:(93/174): python3-toml-0.10.2-6.el9.noarch.rpm 385 kB/s | 42 kB 00:00 2024-03-07T07:28:40.385 INFO:teuthology.orchestra.run.smithi177.stdout:(123/174): python3-google-auth-2.28.0-1.el9.noa 2.4 MB/s | 203 kB 00:00 2024-03-07T07:28:40.475 INFO:teuthology.orchestra.run.smithi160.stdout:(94/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 441 kB/s | 44 kB 00:00 2024-03-07T07:28:40.502 INFO:teuthology.orchestra.run.smithi177.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 1.2 MB/s | 144 kB 00:00 2024-03-07T07:28:40.591 INFO:teuthology.orchestra.run.smithi177.stdout:(125/174): python3-grpcio-1.46.7-10.el9.x86_64. 7.7 MB/s | 2.0 MB 00:00 2024-03-07T07:28:40.619 INFO:teuthology.orchestra.run.smithi177.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 92 kB/s | 11 kB 00:00 2024-03-07T07:28:40.644 INFO:teuthology.orchestra.run.smithi177.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 334 kB/s | 18 kB 00:00 2024-03-07T07:28:40.670 INFO:teuthology.orchestra.run.smithi177.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 459 kB/s | 23 kB 00:00 2024-03-07T07:28:40.695 INFO:teuthology.orchestra.run.smithi177.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-03-07T07:28:40.720 INFO:teuthology.orchestra.run.smithi177.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 394 kB/s | 20 kB 00:00 2024-03-07T07:28:40.750 INFO:teuthology.orchestra.run.smithi160.stdout:(95/174): qatlib-23.11.0-1.el9.x86_64.rpm 788 kB/s | 216 kB 00:00 2024-03-07T07:28:40.762 INFO:teuthology.orchestra.run.smithi177.stdout:(131/174): protobuf-compiler-3.14.0-13.el9.x86_ 467 kB/s | 863 kB 00:01 2024-03-07T07:28:40.804 INFO:teuthology.orchestra.run.smithi177.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 557 kB/s | 46 kB 00:00 2024-03-07T07:28:40.838 INFO:teuthology.orchestra.run.smithi177.stdout:(133/174): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-03-07T07:28:40.859 INFO:teuthology.orchestra.run.smithi160.stdout:(96/174): qatlib-service-23.11.0-1.el9.x86_64.r 322 kB/s | 35 kB 00:00 2024-03-07T07:28:40.863 INFO:teuthology.orchestra.run.smithi177.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 790 kB/s | 79 kB 00:00 2024-03-07T07:28:40.888 INFO:teuthology.orchestra.run.smithi177.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 696 kB/s | 58 kB 00:00 2024-03-07T07:28:40.913 INFO:teuthology.orchestra.run.smithi177.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 5.4 MB/s | 272 kB 00:00 2024-03-07T07:28:40.939 INFO:teuthology.orchestra.run.smithi177.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 219 kB/s | 16 kB 00:00 2024-03-07T07:28:40.964 INFO:teuthology.orchestra.run.smithi177.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-03-07T07:28:40.976 INFO:teuthology.orchestra.run.smithi160.stdout:(97/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 381 kB/s | 44 kB 00:00 2024-03-07T07:28:40.989 INFO:teuthology.orchestra.run.smithi177.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 376 kB/s | 28 kB 00:00 2024-03-07T07:28:41.015 INFO:teuthology.orchestra.run.smithi177.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 617 kB/s | 31 kB 00:00 2024-03-07T07:28:41.040 INFO:teuthology.orchestra.run.smithi177.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.5 MB/s | 188 kB 00:00 2024-03-07T07:28:41.065 INFO:teuthology.orchestra.run.smithi177.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 785 kB/s | 59 kB 00:00 2024-03-07T07:28:41.090 INFO:teuthology.orchestra.run.smithi177.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 477 kB/s | 36 kB 00:00 2024-03-07T07:28:41.115 INFO:teuthology.orchestra.run.smithi177.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-03-07T07:28:41.140 INFO:teuthology.orchestra.run.smithi177.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-03-07T07:28:41.166 INFO:teuthology.orchestra.run.smithi177.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-03-07T07:28:41.216 INFO:teuthology.orchestra.run.smithi177.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-03-07T07:28:41.241 INFO:teuthology.orchestra.run.smithi177.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 269 kB/s | 20 kB 00:00 2024-03-07T07:28:41.267 INFO:teuthology.orchestra.run.smithi177.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-03-07T07:28:41.359 INFO:teuthology.orchestra.run.smithi160.stdout:(98/174): socat-1.7.4.1-5.el9.x86_64.rpm 795 kB/s | 305 kB 00:00 2024-03-07T07:28:41.383 INFO:teuthology.orchestra.run.smithi177.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 9.5 MB/s | 1.6 MB 00:00 2024-03-07T07:28:41.468 INFO:teuthology.orchestra.run.smithi160.stdout:(99/174): uuid-1.6.2-55.el9.x86_64.rpm 534 kB/s | 58 kB 00:00 2024-03-07T07:28:41.515 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-03-07T07:28:41.567 INFO:teuthology.orchestra.run.smithi177.stdout:(151/174): NetworkManager-1.46.0-1.el9.x86_64.r 13 MB/s | 2.3 MB 00:00 2024-03-07T07:28:41.593 INFO:teuthology.orchestra.run.smithi160.stdout:(100/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 750 kB/s | 94 kB 00:00 2024-03-07T07:28:41.644 INFO:teuthology.orchestra.run.smithi177.stdout:(152/174): NetworkManager-libnm-1.46.0-1.el9.x8 24 MB/s | 1.8 MB 00:00 2024-03-07T07:28:41.684 INFO:teuthology.orchestra.run.smithi177.stdout:(153/174): NetworkManager-team-1.46.0-1.el9.x86 1.0 MB/s | 42 kB 00:00 2024-03-07T07:28:41.726 INFO:teuthology.orchestra.run.smithi177.stdout:(154/174): NetworkManager-tui-1.46.0-1.el9.x86_ 5.8 MB/s | 248 kB 00:00 2024-03-07T07:28:41.727 INFO:teuthology.orchestra.run.smithi160.stdout:(101/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 478 kB/s | 64 kB 00:00 2024-03-07T07:28:41.759 INFO:teuthology.orchestra.run.smithi177.stdout:(155/174): acl-2.3.1-4.el9.x86_64.rpm 2.1 MB/s | 71 kB 00:00 2024-03-07T07:28:41.800 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-03-07T07:28:41.800 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-03-07T07:28:41.811 INFO:teuthology.orchestra.run.smithi177.stdout:(156/174): librados2-19.0.0-1848.g8b896a9e.el9. 6.3 MB/s | 3.4 MB 00:00 2024-03-07T07:28:41.835 INFO:teuthology.orchestra.run.smithi177.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 483 kB/s | 36 kB 00:00 2024-03-07T07:28:41.835 INFO:teuthology.orchestra.run.smithi160.stdout:(102/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 417 kB/s | 45 kB 00:00 2024-03-07T07:28:41.877 INFO:teuthology.orchestra.run.smithi177.stdout:(158/174): elfutils-libs-0.190-2.el9.x86_64.rpm 6.0 MB/s | 256 kB 00:00 2024-03-07T07:28:41.885 INFO:teuthology.orchestra.run.smithi160.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 446 kB/s | 22 kB 00:00 2024-03-07T07:28:41.936 INFO:teuthology.orchestra.run.smithi177.stdout:(159/174): librbd1-19.0.0-1848.g8b896a9e.el9.x8 4.7 MB/s | 3.1 MB 00:00 2024-03-07T07:28:41.961 INFO:teuthology.orchestra.run.smithi177.stdout:(160/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.2 MB/s | 192 kB 00:00 2024-03-07T07:28:41.994 INFO:teuthology.orchestra.run.smithi177.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 196 kB/s | 23 kB 00:00 2024-03-07T07:28:42.036 INFO:teuthology.orchestra.run.smithi177.stdout:(162/174): libfdisk-2.37.4-18.el9.x86_64.rpm 3.7 MB/s | 155 kB 00:00 2024-03-07T07:28:42.062 INFO:teuthology.orchestra.run.smithi177.stdout:(163/174): libmount-2.37.4-18.el9.x86_64.rpm 2.0 MB/s | 136 kB 00:00 2024-03-07T07:28:42.087 INFO:teuthology.orchestra.run.smithi177.stdout:(164/174): libblkid-2.37.4-18.el9.x86_64.rpm 718 kB/s | 108 kB 00:00 2024-03-07T07:28:42.094 INFO:teuthology.orchestra.run.smithi160.stdout:(104/174): protobuf-compiler-3.14.0-13.el9.x86_ 4.0 MB/s | 863 kB 00:00 2024-03-07T07:28:42.104 INFO:teuthology.orchestra.run.smithi177.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 951 kB/s | 64 kB 00:00 2024-03-07T07:28:42.121 INFO:teuthology.orchestra.run.smithi177.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 490 kB/s | 29 kB 00:00 2024-03-07T07:28:42.171 INFO:teuthology.orchestra.run.smithi177.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 18 MB/s | 1.2 MB 00:00 2024-03-07T07:28:42.221 INFO:teuthology.orchestra.run.smithi177.stdout:(168/174): util-linux-core-2.37.4-18.el9.x86_64 9.2 MB/s | 465 kB 00:00 2024-03-07T07:28:42.271 INFO:teuthology.orchestra.run.smithi177.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 13 MB/s | 2.2 MB 00:00 2024-03-07T07:28:42.297 INFO:teuthology.orchestra.run.smithi177.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 412 kB/s | 31 kB 00:00 2024-03-07T07:28:42.330 INFO:teuthology.orchestra.run.smithi177.stdout:(171/174): xfsdump-3.1.12-4.el9.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2024-03-07T07:28:42.372 INFO:teuthology.orchestra.run.smithi177.stdout:(172/174): util-linux-2.37.4-18.el9.x86_64.rpm 9.1 MB/s | 2.3 MB 00:00 2024-03-07T07:28:42.386 INFO:teuthology.orchestra.run.smithi160.stdout:(105/174): python3-babel-2.9.1-2.el9.noarch.rpm 831 kB/s | 6.0 MB 00:07 2024-03-07T07:28:42.406 INFO:teuthology.orchestra.run.smithi177.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 9.9 MB/s | 1.1 MB 00:00 2024-03-07T07:28:42.431 INFO:teuthology.orchestra.run.smithi177.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 743 kB/s | 74 kB 00:00 2024-03-07T07:28:42.435 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-03-07T07:28:42.435 INFO:teuthology.orchestra.run.smithi177.stdout:Total 14 MB/s | 226 MB 00:15 2024-03-07T07:28:42.587 INFO:teuthology.orchestra.run.smithi160.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-03-07T07:28:42.645 INFO:teuthology.orchestra.run.smithi160.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 333 kB/s | 19 kB 00:00 2024-03-07T07:28:42.771 INFO:teuthology.orchestra.run.smithi160.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 805 kB/s | 309 kB 00:00 2024-03-07T07:28:42.829 INFO:teuthology.orchestra.run.smithi160.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 447 kB/s | 26 kB 00:00 2024-03-07T07:28:42.904 INFO:teuthology.orchestra.run.smithi160.stdout:(110/174): libarrow-9.0.0-10.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-03-07T07:28:42.921 INFO:teuthology.orchestra.run.smithi160.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 528 kB/s | 49 kB 00:00 2024-03-07T07:28:43.063 INFO:teuthology.orchestra.run.smithi160.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 425 kB/s | 67 kB 00:00 2024-03-07T07:28:43.089 INFO:teuthology.orchestra.run.smithi160.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 904 kB/s | 151 kB 00:00 2024-03-07T07:28:43.147 INFO:teuthology.orchestra.run.smithi160.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 9.8 MB/s | 838 kB 00:00 2024-03-07T07:28:43.206 INFO:teuthology.orchestra.run.smithi160.stdout:(115/174): python3-asyncssh-2.13.2-4.el9.noarch 4.6 MB/s | 548 kB 00:00 2024-03-07T07:28:43.222 INFO:teuthology.orchestra.run.smithi160.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 578 kB/s | 43 kB 00:00 2024-03-07T07:28:43.264 INFO:teuthology.orchestra.run.smithi160.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 550 kB/s | 32 kB 00:00 2024-03-07T07:28:43.281 INFO:teuthology.orchestra.run.smithi160.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 256 kB/s | 15 kB 00:00 2024-03-07T07:28:43.324 INFO:teuthology.orchestra.run.smithi160.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-03-07T07:28:43.365 INFO:teuthology.orchestra.run.smithi160.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-03-07T07:28:43.391 INFO:teuthology.orchestra.run.smithi160.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-03-07T07:28:43.424 INFO:teuthology.orchestra.run.smithi160.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 604 kB/s | 35 kB 00:00 2024-03-07T07:28:43.458 INFO:teuthology.orchestra.run.smithi160.stdout:(123/174): python3-google-auth-2.28.0-1.el9.noa 3.0 MB/s | 203 kB 00:00 2024-03-07T07:28:43.525 INFO:teuthology.orchestra.run.smithi160.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-03-07T07:28:43.583 INFO:teuthology.orchestra.run.smithi160.stdout:(125/174): python3-grpcio-1.46.7-10.el9.x86_64. 13 MB/s | 2.0 MB 00:00 2024-03-07T07:28:43.600 INFO:teuthology.orchestra.run.smithi160.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 142 kB/s | 11 kB 00:00 2024-03-07T07:28:43.648 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-03-07T07:28:43.649 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-03-07T07:28:43.650 INFO:teuthology.orchestra.run.smithi160.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 264 kB/s | 18 kB 00:00 2024-03-07T07:28:43.676 INFO:teuthology.orchestra.run.smithi160.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 307 kB/s | 23 kB 00:00 2024-03-07T07:28:43.718 INFO:teuthology.orchestra.run.smithi160.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 291 kB/s | 19 kB 00:00 2024-03-07T07:28:43.743 INFO:teuthology.orchestra.run.smithi160.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 294 kB/s | 20 kB 00:00 2024-03-07T07:28:43.801 INFO:teuthology.orchestra.run.smithi160.stdout:(131/174): python3-kubernetes-26.1.0-2.el9.noar 12 MB/s | 1.0 MB 00:00 2024-03-07T07:28:43.815 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-03-07T07:28:43.826 INFO:teuthology.orchestra.run.smithi160.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 554 kB/s | 46 kB 00:00 2024-03-07T07:28:43.869 INFO:teuthology.orchestra.run.smithi160.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 1.2 MB/s | 79 kB 00:00 2024-03-07T07:28:43.894 INFO:teuthology.orchestra.run.smithi160.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 861 kB/s | 58 kB 00:00 2024-03-07T07:28:43.944 INFO:teuthology.orchestra.run.smithi160.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-03-07T07:28:43.969 INFO:teuthology.orchestra.run.smithi160.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 218 kB/s | 16 kB 00:00 2024-03-07T07:28:44.020 INFO:teuthology.orchestra.run.smithi160.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-03-07T07:28:44.062 INFO:teuthology.orchestra.run.smithi160.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 306 kB/s | 28 kB 00:00 2024-03-07T07:28:44.087 INFO:teuthology.orchestra.run.smithi160.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 460 kB/s | 31 kB 00:00 2024-03-07T07:28:44.092 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-03-07T07:28:44.092 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-03-07T07:28:44.129 INFO:teuthology.orchestra.run.smithi160.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 2.8 MB/s | 188 kB 00:00 2024-03-07T07:28:44.154 INFO:teuthology.orchestra.run.smithi160.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 885 kB/s | 59 kB 00:00 2024-03-07T07:28:44.196 INFO:teuthology.orchestra.run.smithi160.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 535 kB/s | 36 kB 00:00 2024-03-07T07:28:44.222 INFO:teuthology.orchestra.run.smithi160.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 763 kB/s | 51 kB 00:00 2024-03-07T07:28:44.264 INFO:teuthology.orchestra.run.smithi160.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 3.3 MB/s | 230 kB 00:00 2024-03-07T07:28:44.289 INFO:teuthology.orchestra.run.smithi160.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-03-07T07:28:44.347 INFO:teuthology.orchestra.run.smithi160.stdout:(146/174): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-03-07T07:28:44.364 INFO:teuthology.orchestra.run.smithi160.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 266 kB/s | 20 kB 00:00 2024-03-07T07:28:44.414 INFO:teuthology.orchestra.run.smithi160.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-03-07T07:28:44.456 INFO:teuthology.orchestra.run.smithi160.stdout:(149/174): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-03-07T07:28:45.140 INFO:teuthology.orchestra.run.smithi160.stdout:(150/174): librados2-19.0.0-1848.g8b896a9e.el9. 4.7 MB/s | 3.4 MB 00:00 2024-03-07T07:28:45.273 INFO:teuthology.orchestra.run.smithi160.stdout:(151/174): librbd1-19.0.0-1848.g8b896a9e.el9.x8 3.8 MB/s | 3.1 MB 00:00 2024-03-07T07:28:45.466 INFO:teuthology.orchestra.run.smithi160.stdout:(152/174): NetworkManager-1.46.0-1.el9.x86_64.r 7.0 MB/s | 2.3 MB 00:00 2024-03-07T07:28:45.516 INFO:teuthology.orchestra.run.smithi160.stdout:(153/174): NetworkManager-team-1.46.0-1.el9.x86 850 kB/s | 42 kB 00:00 2024-03-07T07:28:45.574 INFO:teuthology.orchestra.run.smithi160.stdout:(154/174): NetworkManager-tui-1.46.0-1.el9.x86_ 4.2 MB/s | 248 kB 00:00 2024-03-07T07:28:45.625 INFO:teuthology.orchestra.run.smithi160.stdout:(155/174): acl-2.3.1-4.el9.x86_64.rpm 1.4 MB/s | 71 kB 00:00 2024-03-07T07:28:45.666 INFO:teuthology.orchestra.run.smithi160.stdout:(156/174): elfutils-debuginfod-client-0.190-2.e 877 kB/s | 36 kB 00:00 2024-03-07T07:28:45.725 INFO:teuthology.orchestra.run.smithi160.stdout:(157/174): elfutils-libelf-0.190-2.el9.x86_64.r 3.2 MB/s | 192 kB 00:00 2024-03-07T07:28:45.784 INFO:teuthology.orchestra.run.smithi160.stdout:(158/174): elfutils-libs-0.190-2.el9.x86_64.rpm 4.3 MB/s | 256 kB 00:00 2024-03-07T07:28:45.842 INFO:teuthology.orchestra.run.smithi160.stdout:(159/174): NetworkManager-libnm-1.46.0-1.el9.x8 3.2 MB/s | 1.8 MB 00:00 2024-03-07T07:28:45.859 INFO:teuthology.orchestra.run.smithi160.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 306 kB/s | 23 kB 00:00 2024-03-07T07:28:45.895 INFO:teuthology.orchestra.run.smithi160.stdout:(161/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.1 MB/s | 108 kB 00:00 2024-03-07T07:28:45.910 INFO:teuthology.orchestra.run.smithi160.stdout:(162/174): libfdisk-2.37.4-18.el9.x86_64.rpm 3.0 MB/s | 155 kB 00:00 2024-03-07T07:28:45.934 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-03-07T07:28:45.935 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-03-07T07:28:45.951 INFO:teuthology.orchestra.run.smithi160.stdout:(163/174): libmount-2.37.4-18.el9.x86_64.rpm 2.3 MB/s | 136 kB 00:00 2024-03-07T07:28:45.968 INFO:teuthology.orchestra.run.smithi160.stdout:(164/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.1 MB/s | 64 kB 00:00 2024-03-07T07:28:46.069 INFO:teuthology.orchestra.run.smithi160.stdout:(165/174): openssl-3.0.7-27.el9.x86_64.rpm 12 MB/s | 1.2 MB 00:00 2024-03-07T07:28:46.094 INFO:teuthology.orchestra.run.smithi160.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 202 kB/s | 29 kB 00:00 2024-03-07T07:28:46.194 INFO:teuthology.orchestra.run.smithi160.stdout:(167/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 17 MB/s | 2.2 MB 00:00 2024-03-07T07:28:46.253 INFO:teuthology.orchestra.run.smithi160.stdout:(168/174): util-linux-core-2.37.4-18.el9.x86_64 7.7 MB/s | 465 kB 00:00 2024-03-07T07:28:46.304 INFO:teuthology.orchestra.run.smithi160.stdout:(169/174): util-linux-user-2.37.4-18.el9.x86_64 625 kB/s | 31 kB 00:00 2024-03-07T07:28:46.362 INFO:teuthology.orchestra.run.smithi160.stdout:(170/174): xfsdump-3.1.12-4.el9.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2024-03-07T07:28:46.454 INFO:teuthology.orchestra.run.smithi160.stdout:(171/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 12 MB/s | 1.1 MB 00:00 2024-03-07T07:28:46.729 INFO:teuthology.orchestra.run.smithi160.stdout:(172/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 270 kB/s | 74 kB 00:00 2024-03-07T07:28:46.804 INFO:teuthology.orchestra.run.smithi160.stdout:(173/174): util-linux-2.37.4-18.el9.x86_64.rpm 3.2 MB/s | 2.3 MB 00:00 2024-03-07T07:28:50.588 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-03-07T07:28:50.713 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-07T07:28:50.765 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-07T07:28:50.905 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-07T07:28:50.914 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-07T07:28:51.159 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-07T07:28:51.214 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-07T07:28:51.246 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-07T07:28:51.280 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-07T07:28:51.322 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-07T07:28:51.654 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-07T07:28:51.714 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : librados2-2:19.0.0-1848.g8b896a9e.el9.x86_64 10/198 2024-03-07T07:28:51.836 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librados2-2:19.0.0-1848.g8b896a9e.el9.x86_64 10/198 2024-03-07T07:28:51.866 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephfs2-2:19.0.0-1848.g8b896a9e.el9.x86_64 11/198 2024-03-07T07:28:51.965 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libcephfs2-2:19.0.0-1848.g8b896a9e.el9.x86_64 11/198 2024-03-07T07:28:51.995 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rados-2:19.0.0-1848.g8b896a9e.el9.x86_64 12/198 2024-03-07T07:28:52.033 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-07T07:28:52.210 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-07T07:28:52.259 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-07T07:28:52.287 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephsqlite-2:19.0.0-1848.g8b896a9e.el9.x86_64 16/198 2024-03-07T07:28:52.391 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1848.g8b896a9e.el9.x86_64 16/198 2024-03-07T07:28:52.413 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libradosstriper1-2:19.0.0-1848.g8b896a9e.el9.x86 17/198 2024-03-07T07:28:52.477 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1848.g8b896a9e.el9.x86 17/198 2024-03-07T07:28:52.517 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-07T07:28:52.540 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-07T07:28:52.555 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-07T07:28:52.747 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-07T07:28:52.779 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-07T07:28:52.810 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-07T07:28:52.888 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-03-07T07:28:52.913 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-07T07:28:53.001 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-07T07:28:53.027 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-07T07:28:53.031 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-07T07:28:53.065 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-07T07:28:53.076 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-07T07:28:53.090 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-07T07:28:53.115 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-07T07:28:53.162 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-07T07:28:53.168 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-07T07:28:53.203 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-07T07:28:53.216 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-07T07:28:53.228 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-07T07:28:53.270 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-07T07:28:53.423 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-07T07:28:53.529 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-07T07:28:53.590 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-07T07:28:53.624 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-07T07:28:53.646 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : librados2-2:19.0.0-1848.g8b896a9e.el9.x86_64 10/198 2024-03-07T07:28:53.654 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-07T07:28:53.697 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-07T07:28:53.761 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 29/198 2024-03-07T07:28:53.767 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librados2-2:19.0.0-1848.g8b896a9e.el9.x86_64 10/198 2024-03-07T07:28:53.796 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 30/198 2024-03-07T07:28:53.796 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libcephfs2-2:19.0.0-1848.g8b896a9e.el9.x86_64 11/198 2024-03-07T07:28:53.839 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 31/198 2024-03-07T07:28:53.872 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 32/198 2024-03-07T07:28:53.887 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libcephfs2-2:19.0.0-1848.g8b896a9e.el9.x86_64 11/198 2024-03-07T07:28:53.900 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : lua-5.4.4-4.el9.x86_64 33/198 2024-03-07T07:28:53.917 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rados-2:19.0.0-1848.g8b896a9e.el9.x86_64 12/198 2024-03-07T07:28:53.958 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-07T07:28:53.968 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 34/198 2024-03-07T07:28:54.022 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 35/198 2024-03-07T07:28:54.061 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 36/198 2024-03-07T07:28:54.096 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 37/198 2024-03-07T07:28:54.194 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 38/198 2024-03-07T07:28:54.195 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-07T07:28:54.195 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-07T07:28:54.212 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libcephsqlite-2:19.0.0-1848.g8b896a9e.el9.x86_64 16/198 2024-03-07T07:28:54.215 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-ceph-common-2:19.0.0-1848.g8b896a9e.el9. 39/198 2024-03-07T07:28:54.259 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-ceph-argparse-2:19.0.0-1848.g8b896a9e.el 40/198 2024-03-07T07:28:54.288 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cephfs-2:19.0.0-1848.g8b896a9e.el9.x86_6 41/198 2024-03-07T07:28:54.313 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1848.g8b896a9e.el9.x86_64 16/198 2024-03-07T07:28:54.336 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libradosstriper1-2:19.0.0-1848.g8b896a9e.el9.x86 17/198 2024-03-07T07:28:54.356 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-07T07:28:54.383 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1848.g8b896a9e.el9.x86 17/198 2024-03-07T07:28:54.402 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-07T07:28:54.423 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-07T07:28:54.446 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-07T07:28:54.462 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-07T07:28:54.476 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-07T07:28:54.619 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-07T07:28:54.620 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-07T07:28:54.651 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-07T07:28:54.679 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-07T07:28:54.691 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-07T07:28:54.796 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-07T07:28:54.918 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-07T07:28:54.962 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-07T07:28:55.036 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-07T07:28:55.096 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-07T07:28:55.198 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-07T07:28:55.282 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-07T07:28:55.336 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-07T07:28:55.367 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-07T07:28:55.420 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-07T07:28:55.428 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-07T07:28:55.460 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-07T07:28:55.518 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-07T07:28:55.555 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librados-devel-2:19.0.0-1848.g8b896a9e.el9.x86_6 53/198 2024-03-07T07:28:55.584 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-07T07:28:55.595 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-07T07:28:55.747 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-07T07:28:55.787 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-07T07:28:55.798 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-07T07:28:55.819 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-07T07:28:55.892 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 29/198 2024-03-07T07:28:55.934 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 30/198 2024-03-07T07:28:55.978 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 31/198 2024-03-07T07:28:55.986 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-07T07:28:56.013 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 32/198 2024-03-07T07:28:56.041 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : lua-5.4.4-4.el9.x86_64 33/198 2024-03-07T07:28:56.045 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-07T07:28:56.089 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-07T07:28:56.109 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 34/198 2024-03-07T07:28:56.162 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 35/198 2024-03-07T07:28:56.193 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 36/198 2024-03-07T07:28:56.216 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-07T07:28:56.228 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 37/198 2024-03-07T07:28:56.259 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-07T07:28:56.303 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 38/198 2024-03-07T07:28:56.313 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-07T07:28:56.339 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-ceph-common-2:19.0.0-1848.g8b896a9e.el9. 39/198 2024-03-07T07:28:56.386 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-07T07:28:56.391 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-ceph-argparse-2:19.0.0-1848.g8b896a9e.el 40/198 2024-03-07T07:28:56.417 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-07T07:28:56.419 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cephfs-2:19.0.0-1848.g8b896a9e.el9.x86_6 41/198 2024-03-07T07:28:56.454 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-07T07:28:56.490 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-07T07:28:56.509 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-07T07:28:56.534 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-07T07:28:56.545 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-07T07:28:56.588 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-07T07:28:56.608 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-07T07:28:56.630 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-07T07:28:56.743 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-07T07:28:56.764 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-07T07:28:56.795 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-07T07:28:56.802 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-07T07:28:56.832 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-07T07:28:56.874 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-07T07:28:56.968 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-07T07:28:57.016 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-07T07:28:57.056 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-07T07:28:57.082 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-07T07:28:57.106 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-07T07:28:57.179 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-07T07:28:57.307 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-07T07:28:57.324 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-07T07:28:57.338 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-07T07:28:57.366 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-03-07T07:28:57.399 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-07T07:28:57.400 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 82/198 2024-03-07T07:28:57.440 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-07T07:28:57.471 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 84/198 2024-03-07T07:28:57.477 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-07T07:28:57.501 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 85/198 2024-03-07T07:28:57.526 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-07T07:28:57.534 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-07T07:28:57.535 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 86/198 2024-03-07T07:28:57.566 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-07T07:28:57.569 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 87/198 2024-03-07T07:28:57.607 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-07T07:28:57.610 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 88/198 2024-03-07T07:28:57.645 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librados-devel-2:19.0.0-1848.g8b896a9e.el9.x86_6 53/198 2024-03-07T07:28:57.646 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 89/198 2024-03-07T07:28:57.674 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-07T07:28:57.731 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 90/198 2024-03-07T07:28:57.774 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 91/198 2024-03-07T07:28:57.806 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 92/198 2024-03-07T07:28:57.853 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 93/198 2024-03-07T07:28:57.919 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-07T07:28:57.951 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-packaging-20.9-5.el9.noarch 94/198 2024-03-07T07:28:58.000 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 95/198 2024-03-07T07:28:58.097 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-07T07:28:58.160 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-07T07:28:58.196 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-07T07:28:58.333 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-07T07:28:58.374 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-07T07:28:58.428 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-07T07:28:58.646 INFO:teuthology.orchestra.run.smithi160.stdout:(174/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 835 kB/s | 16 MB 00:19 2024-03-07T07:28:58.650 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2024-03-07T07:28:58.651 INFO:teuthology.orchestra.run.smithi160.stdout:Total 5.8 MB/s | 226 MB 00:39 2024-03-07T07:28:58.696 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-07T07:28:58.720 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 96/198 2024-03-07T07:28:58.732 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-07T07:28:58.769 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-07T07:28:58.771 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 97/198 2024-03-07T07:28:58.805 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 98/198 2024-03-07T07:28:58.842 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-07T07:28:58.893 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-07T07:28:58.903 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 99/198 2024-03-07T07:28:58.937 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-07T07:28:58.963 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 100/198 2024-03-07T07:28:58.980 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-07T07:28:59.050 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 101/198 2024-03-07T07:28:59.082 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 102/198 2024-03-07T07:28:59.114 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-07T07:28:59.158 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-07T07:28:59.189 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-07T07:28:59.230 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-07T07:28:59.252 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : openblas-0.3.21-2.el9.x86_64 103/198 2024-03-07T07:28:59.288 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 104/198 2024-03-07T07:28:59.324 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-07T07:28:59.364 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-07T07:28:59.365 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 105/198 2024-03-07T07:28:59.404 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-07T07:28:59.429 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-07T07:28:59.454 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-07T07:28:59.526 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-07T07:28:59.647 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-07T07:28:59.678 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-07T07:28:59.706 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-03-07T07:28:59.740 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 82/198 2024-03-07T07:28:59.780 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-07T07:28:59.820 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 84/198 2024-03-07T07:28:59.849 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 85/198 2024-03-07T07:28:59.883 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 86/198 2024-03-07T07:28:59.917 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 87/198 2024-03-07T07:28:59.958 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 88/198 2024-03-07T07:28:59.993 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 89/198 2024-03-07T07:29:00.006 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 106/198 2024-03-07T07:29:00.014 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2024-03-07T07:29:00.081 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 90/198 2024-03-07T07:29:00.119 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 107/198 2024-03-07T07:29:00.124 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 91/198 2024-03-07T07:29:00.155 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 92/198 2024-03-07T07:29:00.202 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 93/198 2024-03-07T07:29:00.300 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-packaging-20.9-5.el9.noarch 94/198 2024-03-07T07:29:00.335 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction check succeeded. 2024-03-07T07:29:00.336 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2024-03-07T07:29:00.357 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 95/198 2024-03-07T07:29:01.090 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 96/198 2024-03-07T07:29:01.145 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 97/198 2024-03-07T07:29:01.178 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 98/198 2024-03-07T07:29:01.279 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 99/198 2024-03-07T07:29:01.336 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 100/198 2024-03-07T07:29:01.342 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 108/198 2024-03-07T07:29:01.424 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 101/198 2024-03-07T07:29:01.424 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 109/198 2024-03-07T07:29:01.451 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 110/198 2024-03-07T07:29:01.455 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 102/198 2024-03-07T07:29:01.632 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : openblas-0.3.21-2.el9.x86_64 103/198 2024-03-07T07:29:01.662 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 104/198 2024-03-07T07:29:01.739 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 105/198 2024-03-07T07:29:01.764 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 111/198 2024-03-07T07:29:01.790 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : librbd1-2:19.0.0-1848.g8b896a9e.el9.x86_64 112/198 2024-03-07T07:29:01.863 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librbd1-2:19.0.0-1848.g8b896a9e.el9.x86_64 112/198 2024-03-07T07:29:01.894 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 113/198 2024-03-07T07:29:01.930 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 114/198 2024-03-07T07:29:02.372 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 106/198 2024-03-07T07:29:02.380 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-07T07:29:02.409 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librgw2-2:19.0.0-1848.g8b896a9e.el9.x86_64 116/198 2024-03-07T07:29:02.480 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librgw2-2:19.0.0-1848.g8b896a9e.el9.x86_64 116/198 2024-03-07T07:29:02.480 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded. 2024-03-07T07:29:02.481 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2024-03-07T07:29:02.492 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 107/198 2024-03-07T07:29:02.502 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 117/198 2024-03-07T07:29:03.713 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 108/198 2024-03-07T07:29:03.790 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 109/198 2024-03-07T07:29:03.817 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 110/198 2024-03-07T07:29:04.127 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 111/198 2024-03-07T07:29:04.155 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : librbd1-2:19.0.0-1848.g8b896a9e.el9.x86_64 112/198 2024-03-07T07:29:04.228 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librbd1-2:19.0.0-1848.g8b896a9e.el9.x86_64 112/198 2024-03-07T07:29:04.260 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 113/198 2024-03-07T07:29:04.296 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 114/198 2024-03-07T07:29:04.670 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 118/198 2024-03-07T07:29:04.707 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 118/198 2024-03-07T07:29:04.746 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-07T07:29:04.775 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librgw2-2:19.0.0-1848.g8b896a9e.el9.x86_64 116/198 2024-03-07T07:29:04.828 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librgw2-2:19.0.0-1848.g8b896a9e.el9.x86_64 116/198 2024-03-07T07:29:04.851 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 117/198 2024-03-07T07:29:05.244 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 118/198 2024-03-07T07:29:05.285 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-base-2:19.0.0-1848.g8b896a9e.el9.x86_64 119/198 2024-03-07T07:29:05.375 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-base-2:19.0.0-1848.g8b896a9e.el9.x86_64 119/198 2024-03-07T07:29:05.375 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-07T07:29:05.375 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-07T07:29:05.375 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-03-07T07:29:05.403 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-selinux-2:19.0.0-1848.g8b896a9e.el9.x86_64 120/198 2024-03-07T07:29:07.081 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 118/198 2024-03-07T07:29:07.114 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 118/198 2024-03-07T07:29:07.652 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 118/198 2024-03-07T07:29:07.692 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-base-2:19.0.0-1848.g8b896a9e.el9.x86_64 119/198 2024-03-07T07:29:07.773 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-base-2:19.0.0-1848.g8b896a9e.el9.x86_64 119/198 2024-03-07T07:29:07.774 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-07T07:29:07.774 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-07T07:29:07.774 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-07T07:29:07.802 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-selinux-2:19.0.0-1848.g8b896a9e.el9.x86_64 120/198 2024-03-07T07:29:09.637 INFO:teuthology.orchestra.run.smithi160.stdout: Preparing : 1/1 2024-03-07T07:29:09.751 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-07T07:29:09.784 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-07T07:29:09.830 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-07T07:29:09.841 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-07T07:29:09.885 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-07T07:29:09.943 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-07T07:29:09.976 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-07T07:29:10.001 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-07T07:29:10.052 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-07T07:29:10.391 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-07T07:29:10.442 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : librados2-2:19.0.0-1848.g8b896a9e.el9.x86_64 10/198 2024-03-07T07:29:10.564 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librados2-2:19.0.0-1848.g8b896a9e.el9.x86_64 10/198 2024-03-07T07:29:10.601 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libcephfs2-2:19.0.0-1848.g8b896a9e.el9.x86_64 11/198 2024-03-07T07:29:10.693 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libcephfs2-2:19.0.0-1848.g8b896a9e.el9.x86_64 11/198 2024-03-07T07:29:10.731 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-rados-2:19.0.0-1848.g8b896a9e.el9.x86_64 12/198 2024-03-07T07:29:10.779 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-07T07:29:10.965 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-07T07:29:11.011 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-07T07:29:11.039 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libcephsqlite-2:19.0.0-1848.g8b896a9e.el9.x86_64 16/198 2024-03-07T07:29:11.144 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1848.g8b896a9e.el9.x86_64 16/198 2024-03-07T07:29:11.173 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libradosstriper1-2:19.0.0-1848.g8b896a9e.el9.x86 17/198 2024-03-07T07:29:11.239 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1848.g8b896a9e.el9.x86 17/198 2024-03-07T07:29:11.277 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-07T07:29:11.300 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-07T07:29:11.316 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-07T07:29:11.508 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-07T07:29:11.540 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-07T07:29:11.579 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-07T07:29:11.913 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-07T07:29:12.180 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-07T07:29:12.240 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-07T07:29:12.365 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-07T07:29:12.416 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-07T07:29:12.635 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-07T07:29:12.745 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-07T07:29:12.841 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-07T07:29:12.871 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-07T07:29:12.905 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-07T07:29:12.969 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 29/198 2024-03-07T07:29:13.004 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 30/198 2024-03-07T07:29:13.048 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 31/198 2024-03-07T07:29:13.081 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 32/198 2024-03-07T07:29:13.108 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : lua-5.4.4-4.el9.x86_64 33/198 2024-03-07T07:29:13.178 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 34/198 2024-03-07T07:29:13.231 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 35/198 2024-03-07T07:29:13.261 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 36/198 2024-03-07T07:29:13.305 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 37/198 2024-03-07T07:29:13.381 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 38/198 2024-03-07T07:29:13.415 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-ceph-common-2:19.0.0-1848.g8b896a9e.el9. 39/198 2024-03-07T07:29:13.458 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-ceph-argparse-2:19.0.0-1848.g8b896a9e.el 40/198 2024-03-07T07:29:13.487 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cephfs-2:19.0.0-1848.g8b896a9e.el9.x86_6 41/198 2024-03-07T07:29:13.557 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-07T07:29:13.602 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-07T07:29:13.676 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-07T07:29:13.812 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-07T07:29:13.873 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-07T07:29:14.402 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-07T07:29:14.494 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-07T07:29:14.597 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-07T07:29:14.654 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-07T07:29:14.663 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-07T07:29:14.705 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-07T07:29:14.752 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-07T07:29:14.790 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : librados-devel-2:19.0.0-1848.g8b896a9e.el9.x86_6 53/198 2024-03-07T07:29:14.820 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-07T07:29:15.095 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-07T07:29:15.266 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-07T07:29:15.363 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-07T07:29:15.408 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-07T07:29:15.551 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-07T07:29:15.593 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-07T07:29:15.648 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-07T07:29:15.721 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-07T07:29:15.752 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-07T07:29:15.789 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-07T07:29:15.836 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-07T07:29:15.871 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-07T07:29:15.906 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-07T07:29:15.949 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-07T07:29:16.085 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-07T07:29:16.129 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-07T07:29:16.159 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-07T07:29:16.200 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-07T07:29:16.297 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-07T07:29:16.334 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-07T07:29:16.374 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-07T07:29:16.408 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-07T07:29:16.432 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-07T07:29:16.530 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-07T07:29:16.661 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-07T07:29:16.698 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-07T07:29:16.726 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-03-07T07:29:16.764 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 82/198 2024-03-07T07:29:16.799 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-07T07:29:16.831 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 84/198 2024-03-07T07:29:16.860 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 85/198 2024-03-07T07:29:16.895 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 86/198 2024-03-07T07:29:16.928 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 87/198 2024-03-07T07:29:16.961 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 88/198 2024-03-07T07:29:16.996 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 89/198 2024-03-07T07:29:17.065 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1848.g8b896a9e.el9.x86_64 120/198 2024-03-07T07:29:17.066 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /sys 2024-03-07T07:29:17.066 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /proc 2024-03-07T07:29:17.066 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /mnt 2024-03-07T07:29:17.066 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /var/tmp 2024-03-07T07:29:17.066 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /home 2024-03-07T07:29:17.066 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /root 2024-03-07T07:29:17.067 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /tmp 2024-03-07T07:29:17.067 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-03-07T07:29:17.084 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 90/198 2024-03-07T07:29:17.134 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 91/198 2024-03-07T07:29:17.168 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 92/198 2024-03-07T07:29:17.214 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 93/198 2024-03-07T07:29:17.222 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-volume-2:19.0.0-1848.g8b896a9e.el9.noarch 121/198 2024-03-07T07:29:17.236 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-volume-2:19.0.0-1848.g8b896a9e.el9.noarch 121/198 2024-03-07T07:29:17.237 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T07:29:17.237 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-07T07:29:17.237 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-03-07T07:29:17.312 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-packaging-20.9-5.el9.noarch 94/198 2024-03-07T07:29:17.369 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 95/198 2024-03-07T07:29:18.105 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 96/198 2024-03-07T07:29:18.156 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 97/198 2024-03-07T07:29:18.190 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 98/198 2024-03-07T07:29:18.291 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 99/198 2024-03-07T07:29:18.348 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 100/198 2024-03-07T07:29:18.435 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 101/198 2024-03-07T07:29:18.466 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 102/198 2024-03-07T07:29:18.635 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : openblas-0.3.21-2.el9.x86_64 103/198 2024-03-07T07:29:18.672 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 104/198 2024-03-07T07:29:18.707 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-osd-2:19.0.0-1848.g8b896a9e.el9.x86_64 122/198 2024-03-07T07:29:18.733 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-osd-2:19.0.0-1848.g8b896a9e.el9.x86_64 122/198 2024-03-07T07:29:18.734 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T07:29:18.734 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-07T07:29:18.734 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-07T07:29:18.734 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-07T07:29:18.734 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-03-07T07:29:18.751 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 105/198 2024-03-07T07:29:18.975 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mds-2:19.0.0-1848.g8b896a9e.el9.x86_64 123/198 2024-03-07T07:29:18.997 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mds-2:19.0.0-1848.g8b896a9e.el9.x86_64 123/198 2024-03-07T07:29:18.997 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T07:29:18.997 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-07T07:29:18.997 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-07T07:29:18.997 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-07T07:29:18.998 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-03-07T07:29:19.251 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1848.g8b896a9e.el9.x86_64 120/198 2024-03-07T07:29:19.252 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /sys 2024-03-07T07:29:19.252 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /proc 2024-03-07T07:29:19.252 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /mnt 2024-03-07T07:29:19.252 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /var/tmp 2024-03-07T07:29:19.252 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /home 2024-03-07T07:29:19.252 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /root 2024-03-07T07:29:19.252 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /tmp 2024-03-07T07:29:19.252 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-07T07:29:19.408 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 106/198 2024-03-07T07:29:19.413 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-volume-2:19.0.0-1848.g8b896a9e.el9.noarch 121/198 2024-03-07T07:29:19.429 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-volume-2:19.0.0-1848.g8b896a9e.el9.noarch 121/198 2024-03-07T07:29:19.429 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T07:29:19.429 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-07T07:29:19.429 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-07T07:29:19.444 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mon-2:19.0.0-1848.g8b896a9e.el9.x86_64 124/198 2024-03-07T07:29:19.466 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mon-2:19.0.0-1848.g8b896a9e.el9.x86_64 124/198 2024-03-07T07:29:19.466 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T07:29:19.466 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-07T07:29:19.467 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-07T07:29:19.467 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-07T07:29:19.467 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-03-07T07:29:19.538 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 107/198 2024-03-07T07:29:19.585 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : automake-1.16.2-8.el9.noarch 125/198 2024-03-07T07:29:19.638 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-ply-3.11-14.el9.noarch 126/198 2024-03-07T07:29:19.694 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 127/198 2024-03-07T07:29:19.887 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 128/198 2024-03-07T07:29:19.935 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 129/198 2024-03-07T07:29:20.004 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 130/198 2024-03-07T07:29:20.092 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 131/198 2024-03-07T07:29:20.232 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 132/198 2024-03-07T07:29:20.285 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 133/198 2024-03-07T07:29:20.318 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 134/198 2024-03-07T07:29:20.350 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 135/198 2024-03-07T07:29:20.385 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : jq-1.6-16.el9.x86_64 136/198 2024-03-07T07:29:20.421 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : fuse-2.9.9-15.el9.x86_64 137/198 2024-03-07T07:29:20.786 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 108/198 2024-03-07T07:29:20.865 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: cephadm-2:19.0.0-1848.g8b896a9e.el9.noarch 138/198 2024-03-07T07:29:20.874 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 109/198 2024-03-07T07:29:20.891 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-osd-2:19.0.0-1848.g8b896a9e.el9.x86_64 122/198 2024-03-07T07:29:20.894 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephadm-2:19.0.0-1848.g8b896a9e.el9.noarch 138/198 2024-03-07T07:29:20.901 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 110/198 2024-03-07T07:29:20.917 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-osd-2:19.0.0-1848.g8b896a9e.el9.x86_64 122/198 2024-03-07T07:29:20.917 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T07:29:20.917 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-07T07:29:20.918 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-07T07:29:20.918 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-07T07:29:20.918 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-07T07:29:20.926 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1848.g8b896a9e.e 139/198 2024-03-07T07:29:20.986 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1848.g8b896a9e. 140/198 2024-03-07T07:29:21.018 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1848.g8b896a9e.el9.noa 141/198 2024-03-07T07:29:21.142 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mds-2:19.0.0-1848.g8b896a9e.el9.x86_64 123/198 2024-03-07T07:29:21.165 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mds-2:19.0.0-1848.g8b896a9e.el9.x86_64 123/198 2024-03-07T07:29:21.165 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T07:29:21.165 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-07T07:29:21.165 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-07T07:29:21.165 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-07T07:29:21.165 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-07T07:29:21.232 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 111/198 2024-03-07T07:29:21.246 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1848.g8b896a9e.el9.noa 141/198 2024-03-07T07:29:21.268 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : librbd1-2:19.0.0-1848.g8b896a9e.el9.x86_64 112/198 2024-03-07T07:29:21.277 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1848.g8b896a9e.el9.n 142/198 2024-03-07T07:29:21.338 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librbd1-2:19.0.0-1848.g8b896a9e.el9.x86_64 112/198 2024-03-07T07:29:21.369 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-rbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 113/198 2024-03-07T07:29:21.406 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 114/198 2024-03-07T07:29:21.609 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mon-2:19.0.0-1848.g8b896a9e.el9.x86_64 124/198 2024-03-07T07:29:21.632 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mon-2:19.0.0-1848.g8b896a9e.el9.x86_64 124/198 2024-03-07T07:29:21.632 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T07:29:21.632 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-07T07:29:21.632 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-07T07:29:21.632 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-07T07:29:21.632 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-07T07:29:21.739 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : automake-1.16.2-8.el9.noarch 125/198 2024-03-07T07:29:21.796 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-ply-3.11-14.el9.noarch 126/198 2024-03-07T07:29:21.851 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 127/198 2024-03-07T07:29:21.861 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-07T07:29:21.892 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : librgw2-2:19.0.0-1848.g8b896a9e.el9.x86_64 116/198 2024-03-07T07:29:21.964 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librgw2-2:19.0.0-1848.g8b896a9e.el9.x86_64 116/198 2024-03-07T07:29:21.986 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-rgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 117/198 2024-03-07T07:29:22.035 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 128/198 2024-03-07T07:29:22.077 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 129/198 2024-03-07T07:29:22.154 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 130/198 2024-03-07T07:29:22.242 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1848.g8b896a9e.el9.n 142/198 2024-03-07T07:29:22.243 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 131/198 2024-03-07T07:29:22.269 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1848.g8b8 143/198 2024-03-07T07:29:22.370 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1848.g8b8 143/198 2024-03-07T07:29:22.392 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 132/198 2024-03-07T07:29:22.434 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 133/198 2024-03-07T07:29:22.468 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 134/198 2024-03-07T07:29:22.499 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 135/198 2024-03-07T07:29:22.532 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1848.g8b896a9e.el 144/198 2024-03-07T07:29:22.534 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : jq-1.6-16.el9.x86_64 136/198 2024-03-07T07:29:22.562 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : fuse-2.9.9-15.el9.x86_64 137/198 2024-03-07T07:29:22.563 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-2:19.0.0-1848.g8b896a9e.el9.x86_64 145/198 2024-03-07T07:29:22.586 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1848.g8b896a9e.el9.x86_64 145/198 2024-03-07T07:29:22.586 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T07:29:22.587 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-07T07:29:22.587 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-07T07:29:22.587 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-07T07:29:22.587 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-03-07T07:29:22.630 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-rook-2:19.0.0-1848.g8b896a9e.el9.noarch 146/198 2024-03-07T07:29:22.641 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1848.g8b896a9e.el9.noarch 146/198 2024-03-07T07:29:22.738 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-2:19.0.0-1848.g8b896a9e.el9.x86_64 147/198 2024-03-07T07:29:23.013 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: cephadm-2:19.0.0-1848.g8b896a9e.el9.noarch 138/198 2024-03-07T07:29:23.044 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : cephadm-2:19.0.0-1848.g8b896a9e.el9.noarch 138/198 2024-03-07T07:29:23.083 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1848.g8b896a9e.e 139/198 2024-03-07T07:29:23.144 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1848.g8b896a9e. 140/198 2024-03-07T07:29:23.167 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1848.g8b896a9e.el9.noa 141/198 2024-03-07T07:29:23.396 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1848.g8b896a9e.el9.noa 141/198 2024-03-07T07:29:23.434 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1848.g8b896a9e.el9.n 142/198 2024-03-07T07:29:24.259 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 118/198 2024-03-07T07:29:24.291 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 118/198 2024-03-07T07:29:24.399 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1848.g8b896a9e.el9.n 142/198 2024-03-07T07:29:24.426 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1848.g8b8 143/198 2024-03-07T07:29:24.530 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1848.g8b8 143/198 2024-03-07T07:29:24.689 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1848.g8b896a9e.el 144/198 2024-03-07T07:29:24.720 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-2:19.0.0-1848.g8b896a9e.el9.x86_64 145/198 2024-03-07T07:29:24.743 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1848.g8b896a9e.el9.x86_64 145/198 2024-03-07T07:29:24.743 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T07:29:24.743 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-07T07:29:24.743 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-07T07:29:24.743 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-07T07:29:24.743 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-07T07:29:24.787 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-rook-2:19.0.0-1848.g8b896a9e.el9.noarch 146/198 2024-03-07T07:29:24.798 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1848.g8b896a9e.el9.noarch 146/198 2024-03-07T07:29:24.854 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 118/198 2024-03-07T07:29:24.896 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-2:19.0.0-1848.g8b896a9e.el9.x86_64 147/198 2024-03-07T07:29:24.903 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-base-2:19.0.0-1848.g8b896a9e.el9.x86_64 119/198 2024-03-07T07:29:25.000 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-base-2:19.0.0-1848.g8b896a9e.el9.x86_64 119/198 2024-03-07T07:29:25.000 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-07T07:29:25.000 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-07T07:29:25.000 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:29:25.029 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-selinux-2:19.0.0-1848.g8b896a9e.el9.x86_64 120/198 2024-03-07T07:29:27.139 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 148/198 2024-03-07T07:29:27.209 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-test-2:19.0.0-1848.g8b896a9e.el9.x86_64 149/198 2024-03-07T07:29:28.063 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libtool-2.4.6-45.el9.x86_64 150/198 2024-03-07T07:29:28.097 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-radosgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 151/198 2024-03-07T07:29:28.120 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 151/198 2024-03-07T07:29:28.120 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T07:29:28.120 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-07T07:29:28.120 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-07T07:29:28.120 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-07T07:29:28.121 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-03-07T07:29:28.164 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1848.g8b896 152/198 2024-03-07T07:29:28.186 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1848.g8b896 152/198 2024-03-07T07:29:28.187 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T07:29:28.187 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-07T07:29:28.187 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-03-07T07:29:28.231 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephfs-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 153/198 2024-03-07T07:29:28.254 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 153/198 2024-03-07T07:29:28.254 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T07:29:28.254 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-07T07:29:28.254 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-07T07:29:28.254 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-07T07:29:28.254 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-03-07T07:29:28.566 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 154/198 2024-03-07T07:29:28.588 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 154/198 2024-03-07T07:29:28.588 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T07:29:28.588 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-07T07:29:28.588 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-07T07:29:28.588 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-07T07:29:28.588 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-03-07T07:29:28.640 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 155/198 2024-03-07T07:29:28.670 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-nbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 156/198 2024-03-07T07:29:28.697 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 157/198 2024-03-07T07:29:28.878 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephfs-shell-2:19.0.0-1848.g8b896a9e.el9.x86_64 158/198 2024-03-07T07:29:28.937 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 159/198 2024-03-07T07:29:28.975 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-07T07:29:29.036 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-07T07:29:29.080 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-07T07:29:29.112 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephfs-devel-2:19.0.0-1848.g8b896a9e.el9.x86_ 163/198 2024-03-07T07:29:29.153 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-07T07:29:29.182 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-07T07:29:29.237 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-07T07:29:29.317 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 148/198 2024-03-07T07:29:29.320 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-07T07:29:29.357 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-07T07:29:29.381 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-test-2:19.0.0-1848.g8b896a9e.el9.x86_64 149/198 2024-03-07T07:29:29.951 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-07T07:29:29.987 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-07T07:29:30.007 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-07T07:29:30.241 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libtool-2.4.6-45.el9.x86_64 150/198 2024-03-07T07:29:30.269 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-radosgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 151/198 2024-03-07T07:29:30.292 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 151/198 2024-03-07T07:29:30.292 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T07:29:30.292 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-07T07:29:30.292 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-07T07:29:30.292 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-07T07:29:30.293 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-07T07:29:30.345 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1848.g8b896 152/198 2024-03-07T07:29:30.366 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1848.g8b896 152/198 2024-03-07T07:29:30.367 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T07:29:30.367 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-07T07:29:30.367 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-07T07:29:30.412 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : cephfs-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 153/198 2024-03-07T07:29:30.435 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 153/198 2024-03-07T07:29:30.435 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T07:29:30.435 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-07T07:29:30.435 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-07T07:29:30.435 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-07T07:29:30.435 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-07T07:29:30.738 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : rbd-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 154/198 2024-03-07T07:29:30.761 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 154/198 2024-03-07T07:29:30.761 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T07:29:30.761 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-07T07:29:30.761 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-07T07:29:30.761 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-07T07:29:30.761 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-07T07:29:30.812 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : rbd-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 155/198 2024-03-07T07:29:30.817 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephfs-top-2:19.0.0-1848.g8b896a9e.el9.noarch 170/198 2024-03-07T07:29:30.842 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : rbd-nbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 156/198 2024-03-07T07:29:30.870 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 157/198 2024-03-07T07:29:30.915 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-07T07:29:30.996 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flex-2.6.4-9.el9.x86_64 172/198 2024-03-07T07:29:31.035 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : bison-3.7.4-5.el9.x86_64 173/198 2024-03-07T07:29:31.049 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : cephfs-shell-2:19.0.0-1848.g8b896a9e.el9.x86_64 158/198 2024-03-07T07:29:31.060 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 174/198 2024-03-07T07:29:31.086 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-07T07:29:31.100 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 159/198 2024-03-07T07:29:31.111 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-07T07:29:31.137 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-07T07:29:31.138 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-07T07:29:31.139 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-07T07:29:31.192 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-07T07:29:31.227 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-07T07:29:31.233 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-07T07:29:31.251 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libcephfs-devel-2:19.0.0-1848.g8b896a9e.el9.x86_ 163/198 2024-03-07T07:29:31.263 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-07T07:29:31.263 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-07T07:29:31.275 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-07T07:29:31.313 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-07T07:29:31.348 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-07T07:29:31.367 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-07T07:29:31.374 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-07T07:29:31.407 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-07T07:29:31.427 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-07T07:29:31.433 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-07T07:29:31.458 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-07T07:29:31.470 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-07T07:29:31.490 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-07T07:29:31.530 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-07T07:29:31.562 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-07T07:29:31.588 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-07T07:29:31.613 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-07T07:29:31.638 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-07T07:29:31.663 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-07T07:29:31.670 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-07T07:29:31.670 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-07T07:29:31.858 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-07T07:29:31.858 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-07T07:29:31.941 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-07T07:29:31.966 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-07T07:29:31.991 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-07T07:29:32.017 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-07T07:29:32.039 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-07T07:29:32.049 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-07T07:29:32.076 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-07T07:29:32.097 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-07T07:29:32.114 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-07T07:29:32.909 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : cephfs-top-2:19.0.0-1848.g8b896a9e.el9.noarch 170/198 2024-03-07T07:29:33.003 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-07T07:29:33.083 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : flex-2.6.4-9.el9.x86_64 172/198 2024-03-07T07:29:33.116 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : bison-3.7.4-5.el9.x86_64 173/198 2024-03-07T07:29:33.141 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 174/198 2024-03-07T07:29:33.166 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-07T07:29:33.191 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-07T07:29:33.218 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-07T07:29:33.218 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-07T07:29:33.306 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-07T07:29:33.335 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-07T07:29:33.335 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-07T07:29:33.428 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-07T07:29:33.455 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-07T07:29:33.487 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-07T07:29:33.513 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-07T07:29:33.538 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-07T07:29:33.571 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-07T07:29:33.612 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-07T07:29:33.653 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-07T07:29:33.678 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-07T07:29:33.703 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-07T07:29:33.737 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-07T07:29:33.762 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-07T07:29:33.769 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-07T07:29:33.769 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-07T07:29:33.948 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-07T07:29:33.949 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-07T07:29:34.023 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-07T07:29:34.048 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-07T07:29:34.064 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-07T07:29:34.090 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-07T07:29:34.123 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-07T07:29:34.188 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-07T07:29:36.462 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1848.g8b896a9e.el9.x86_64 120/198 2024-03-07T07:29:36.462 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /sys 2024-03-07T07:29:36.462 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /proc 2024-03-07T07:29:36.463 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /mnt 2024-03-07T07:29:36.463 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /var/tmp 2024-03-07T07:29:36.463 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /home 2024-03-07T07:29:36.463 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /root 2024-03-07T07:29:36.463 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /tmp 2024-03-07T07:29:36.463 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:29:36.639 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-volume-2:19.0.0-1848.g8b896a9e.el9.noarch 121/198 2024-03-07T07:29:36.654 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-volume-2:19.0.0-1848.g8b896a9e.el9.noarch 121/198 2024-03-07T07:29:36.655 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T07:29:36.655 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-07T07:29:36.655 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:29:37.164 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-07T07:29:37.164 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-2:19.0.0-1848.g8b896a9e.el9.x86_64 1/198 2024-03-07T07:29:37.164 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-base-2:19.0.0-1848.g8b896a9e.el9.x86_64 2/198 2024-03-07T07:29:37.164 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 3/198 2024-03-07T07:29:37.164 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 4/198 2024-03-07T07:29:37.164 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1848.g8b896 5/198 2024-03-07T07:29:37.164 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mds-2:19.0.0-1848.g8b896a9e.el9.x86_64 6/198 2024-03-07T07:29:37.164 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-2:19.0.0-1848.g8b896a9e.el9.x86_64 7/198 2024-03-07T07:29:37.164 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mon-2:19.0.0-1848.g8b896a9e.el9.x86_64 8/198 2024-03-07T07:29:37.164 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-osd-2:19.0.0-1848.g8b896a9e.el9.x86_64 9/198 2024-03-07T07:29:37.165 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-radosgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 10/198 2024-03-07T07:29:37.165 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-selinux-2:19.0.0-1848.g8b896a9e.el9.x86_64 11/198 2024-03-07T07:29:37.165 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-test-2:19.0.0-1848.g8b896a9e.el9.x86_64 12/198 2024-03-07T07:29:37.165 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephfs-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 13/198 2024-03-07T07:29:37.165 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephfs-shell-2:19.0.0-1848.g8b896a9e.el9.x86_64 14/198 2024-03-07T07:29:37.165 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephfs-devel-2:19.0.0-1848.g8b896a9e.el9.x86_ 15/198 2024-03-07T07:29:37.165 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephfs2-2:19.0.0-1848.g8b896a9e.el9.x86_64 16/198 2024-03-07T07:29:37.165 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephsqlite-2:19.0.0-1848.g8b896a9e.el9.x86_64 17/198 2024-03-07T07:29:37.165 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados-devel-2:19.0.0-1848.g8b896a9e.el9.x86_6 18/198 2024-03-07T07:29:37.165 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libradosstriper1-2:19.0.0-1848.g8b896a9e.el9.x86 19/198 2024-03-07T07:29:37.165 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librgw2-2:19.0.0-1848.g8b896a9e.el9.x86_64 20/198 2024-03-07T07:29:37.165 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1848.g8b896a9e.el 21/198 2024-03-07T07:29:37.165 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-ceph-common-2:19.0.0-1848.g8b896a9e.el9. 22/198 2024-03-07T07:29:37.166 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cephfs-2:19.0.0-1848.g8b896a9e.el9.x86_6 23/198 2024-03-07T07:29:37.166 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rados-2:19.0.0-1848.g8b896a9e.el9.x86_64 24/198 2024-03-07T07:29:37.166 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 25/198 2024-03-07T07:29:37.166 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 26/198 2024-03-07T07:29:37.166 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 27/198 2024-03-07T07:29:37.166 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 28/198 2024-03-07T07:29:37.166 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-nbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 29/198 2024-03-07T07:29:37.166 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1848.g8b896a9e. 30/198 2024-03-07T07:29:37.166 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1848.g8b896a9e.el9.noa 31/198 2024-03-07T07:29:37.166 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1848.g8b896a9e.el9.n 32/198 2024-03-07T07:29:37.166 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1848.g8b8 33/198 2024-03-07T07:29:37.166 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1848.g8b896a9e.el 34/198 2024-03-07T07:29:37.166 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1848.g8b896a9e.el9.noarch 35/198 2024-03-07T07:29:37.167 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1848.g8b896a9e.e 36/198 2024-03-07T07:29:37.167 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-volume-2:19.0.0-1848.g8b896a9e.el9.noarch 37/198 2024-03-07T07:29:37.167 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephadm-2:19.0.0-1848.g8b896a9e.el9.noarch 38/198 2024-03-07T07:29:37.167 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephfs-top-2:19.0.0-1848.g8b896a9e.el9.noarch 39/198 2024-03-07T07:29:37.167 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 40/198 2024-03-07T07:29:37.167 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 41/198 2024-03-07T07:29:37.167 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : jq-1.6-16.el9.x86_64 42/198 2024-03-07T07:29:37.167 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 43/198 2024-03-07T07:29:37.167 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 44/198 2024-03-07T07:29:37.167 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 45/198 2024-03-07T07:29:37.167 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 46/198 2024-03-07T07:29:37.167 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 47/198 2024-03-07T07:29:37.167 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-ply-3.11-14.el9.noarch 48/198 2024-03-07T07:29:37.167 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 49/198 2024-03-07T07:29:37.168 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 50/198 2024-03-07T07:29:37.168 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 51/198 2024-03-07T07:29:37.168 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : automake-1.16.2-8.el9.noarch 52/198 2024-03-07T07:29:37.168 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : bison-3.7.4-5.el9.x86_64 53/198 2024-03-07T07:29:37.168 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 54/198 2024-03-07T07:29:37.168 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 55/198 2024-03-07T07:29:37.168 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flex-2.6.4-9.el9.x86_64 56/198 2024-03-07T07:29:37.168 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 57/198 2024-03-07T07:29:37.168 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 58/198 2024-03-07T07:29:37.168 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 59/198 2024-03-07T07:29:37.168 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 60/198 2024-03-07T07:29:37.168 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 61/198 2024-03-07T07:29:37.168 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 62/198 2024-03-07T07:29:37.169 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 63/198 2024-03-07T07:29:37.169 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 64/198 2024-03-07T07:29:37.169 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 65/198 2024-03-07T07:29:37.169 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 66/198 2024-03-07T07:29:37.169 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : lua-5.4.4-4.el9.x86_64 67/198 2024-03-07T07:29:37.169 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 68/198 2024-03-07T07:29:37.169 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 69/198 2024-03-07T07:29:37.169 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 70/198 2024-03-07T07:29:37.169 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : postgresql-13.11-1.el9.x86_64 71/198 2024-03-07T07:29:37.169 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 72/198 2024-03-07T07:29:37.169 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 73/198 2024-03-07T07:29:37.169 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 74/198 2024-03-07T07:29:37.169 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 75/198 2024-03-07T07:29:37.170 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 76/198 2024-03-07T07:29:37.170 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 77/198 2024-03-07T07:29:37.170 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 78/198 2024-03-07T07:29:37.170 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 79/198 2024-03-07T07:29:37.170 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/198 2024-03-07T07:29:37.170 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/198 2024-03-07T07:29:37.170 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/198 2024-03-07T07:29:37.170 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/198 2024-03-07T07:29:37.170 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/198 2024-03-07T07:29:37.170 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/198 2024-03-07T07:29:37.170 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/198 2024-03-07T07:29:37.170 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/198 2024-03-07T07:29:37.170 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/198 2024-03-07T07:29:37.170 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/198 2024-03-07T07:29:37.171 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/198 2024-03-07T07:29:37.171 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/198 2024-03-07T07:29:37.171 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/198 2024-03-07T07:29:37.171 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/198 2024-03-07T07:29:37.171 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/198 2024-03-07T07:29:37.171 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/198 2024-03-07T07:29:37.171 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/198 2024-03-07T07:29:37.171 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 97/198 2024-03-07T07:29:37.171 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 98/198 2024-03-07T07:29:37.171 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 99/198 2024-03-07T07:29:37.171 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 100/198 2024-03-07T07:29:37.171 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/198 2024-03-07T07:29:37.171 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 102/198 2024-03-07T07:29:37.171 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/198 2024-03-07T07:29:37.172 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 104/198 2024-03-07T07:29:37.172 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-07T07:29:37.172 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-07T07:29:37.172 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-07T07:29:37.172 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-07T07:29:37.172 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-07T07:29:37.172 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-07T07:29:37.172 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-07T07:29:37.172 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-07T07:29:37.172 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-07T07:29:37.172 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-07T07:29:37.172 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-07T07:29:37.172 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-07T07:29:37.172 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-07T07:29:37.173 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-07T07:29:37.173 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-07T07:29:37.173 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-07T07:29:37.173 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-07T07:29:37.173 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-07T07:29:37.173 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-07T07:29:37.173 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-07T07:29:37.173 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-07T07:29:37.173 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-07T07:29:37.173 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-07T07:29:37.173 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-07T07:29:37.173 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-07T07:29:37.173 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-07T07:29:37.174 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-07T07:29:37.174 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-07T07:29:37.174 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-07T07:29:37.174 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-07T07:29:37.174 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-07T07:29:37.174 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-07T07:29:37.174 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-07T07:29:37.174 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-07T07:29:37.174 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-07T07:29:37.174 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-07T07:29:37.174 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-07T07:29:37.174 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-07T07:29:37.174 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-07T07:29:37.174 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-07T07:29:37.175 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-07T07:29:37.175 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-07T07:29:37.175 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-07T07:29:37.175 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-07T07:29:37.175 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-07T07:29:37.175 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-07T07:29:37.175 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados2-2:19.0.0-1848.g8b896a9e.el9.x86_64 151/198 2024-03-07T07:29:37.175 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-07T07:29:37.175 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librbd1-2:19.0.0-1848.g8b896a9e.el9.x86_64 153/198 2024-03-07T07:29:37.175 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-07T07:29:37.175 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 155/198 2024-03-07T07:29:37.175 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-03-07T07:29:37.175 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 157/198 2024-03-07T07:29:37.176 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-03-07T07:29:37.176 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 159/198 2024-03-07T07:29:37.176 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-03-07T07:29:37.176 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 161/198 2024-03-07T07:29:37.176 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-03-07T07:29:37.176 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : acl-2.3.1-4.el9.x86_64 163/198 2024-03-07T07:29:37.176 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : acl-2.3.1-3.el9.x86_64 164/198 2024-03-07T07:29:37.176 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/198 2024-03-07T07:29:37.176 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 166/198 2024-03-07T07:29:37.176 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 167/198 2024-03-07T07:29:37.176 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 168/198 2024-03-07T07:29:37.176 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 169/198 2024-03-07T07:29:37.176 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 170/198 2024-03-07T07:29:37.177 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 171/198 2024-03-07T07:29:37.177 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 172/198 2024-03-07T07:29:37.177 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 173/198 2024-03-07T07:29:37.177 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 174/198 2024-03-07T07:29:37.177 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 175/198 2024-03-07T07:29:37.177 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 176/198 2024-03-07T07:29:37.177 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 177/198 2024-03-07T07:29:37.177 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 178/198 2024-03-07T07:29:37.177 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 179/198 2024-03-07T07:29:37.177 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 180/198 2024-03-07T07:29:37.178 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 181/198 2024-03-07T07:29:37.178 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 182/198 2024-03-07T07:29:37.178 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 183/198 2024-03-07T07:29:37.178 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 184/198 2024-03-07T07:29:37.178 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 185/198 2024-03-07T07:29:37.178 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 186/198 2024-03-07T07:29:37.178 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 187/198 2024-03-07T07:29:37.178 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 188/198 2024-03-07T07:29:37.178 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 189/198 2024-03-07T07:29:37.178 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 190/198 2024-03-07T07:29:37.178 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 191/198 2024-03-07T07:29:37.178 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 192/198 2024-03-07T07:29:37.179 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 193/198 2024-03-07T07:29:37.179 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 194/198 2024-03-07T07:29:37.179 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-07T07:29:37.179 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-07T07:29:37.179 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-07T07:29:38.142 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-osd-2:19.0.0-1848.g8b896a9e.el9.x86_64 122/198 2024-03-07T07:29:38.167 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-osd-2:19.0.0-1848.g8b896a9e.el9.x86_64 122/198 2024-03-07T07:29:38.168 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T07:29:38.168 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-07T07:29:38.168 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-07T07:29:38.168 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-07T07:29:38.168 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:29:38.384 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mds-2:19.0.0-1848.g8b896a9e.el9.x86_64 123/198 2024-03-07T07:29:38.407 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mds-2:19.0.0-1848.g8b896a9e.el9.x86_64 123/198 2024-03-07T07:29:38.407 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T07:29:38.407 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-07T07:29:38.407 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-07T07:29:38.407 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-07T07:29:38.408 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:29:38.859 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mon-2:19.0.0-1848.g8b896a9e.el9.x86_64 124/198 2024-03-07T07:29:38.882 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mon-2:19.0.0-1848.g8b896a9e.el9.x86_64 124/198 2024-03-07T07:29:38.882 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T07:29:38.882 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-07T07:29:38.882 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-07T07:29:38.882 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-07T07:29:38.882 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:29:38.998 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : automake-1.16.2-8.el9.noarch 125/198 2024-03-07T07:29:39.054 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-ply-3.11-14.el9.noarch 126/198 2024-03-07T07:29:39.111 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 127/198 2024-03-07T07:29:39.296 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-07T07:29:39.296 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-2:19.0.0-1848.g8b896a9e.el9.x86_64 1/198 2024-03-07T07:29:39.296 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-base-2:19.0.0-1848.g8b896a9e.el9.x86_64 2/198 2024-03-07T07:29:39.296 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 3/198 2024-03-07T07:29:39.296 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 4/198 2024-03-07T07:29:39.297 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1848.g8b896 5/198 2024-03-07T07:29:39.297 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mds-2:19.0.0-1848.g8b896a9e.el9.x86_64 6/198 2024-03-07T07:29:39.297 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-2:19.0.0-1848.g8b896a9e.el9.x86_64 7/198 2024-03-07T07:29:39.297 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mon-2:19.0.0-1848.g8b896a9e.el9.x86_64 8/198 2024-03-07T07:29:39.297 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-osd-2:19.0.0-1848.g8b896a9e.el9.x86_64 9/198 2024-03-07T07:29:39.297 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-radosgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 10/198 2024-03-07T07:29:39.297 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-selinux-2:19.0.0-1848.g8b896a9e.el9.x86_64 11/198 2024-03-07T07:29:39.297 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-test-2:19.0.0-1848.g8b896a9e.el9.x86_64 12/198 2024-03-07T07:29:39.297 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : cephfs-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 13/198 2024-03-07T07:29:39.297 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : cephfs-shell-2:19.0.0-1848.g8b896a9e.el9.x86_64 14/198 2024-03-07T07:29:39.297 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcephfs-devel-2:19.0.0-1848.g8b896a9e.el9.x86_ 15/198 2024-03-07T07:29:39.297 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcephfs2-2:19.0.0-1848.g8b896a9e.el9.x86_64 16/198 2024-03-07T07:29:39.298 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcephsqlite-2:19.0.0-1848.g8b896a9e.el9.x86_64 17/198 2024-03-07T07:29:39.298 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librados-devel-2:19.0.0-1848.g8b896a9e.el9.x86_6 18/198 2024-03-07T07:29:39.298 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libradosstriper1-2:19.0.0-1848.g8b896a9e.el9.x86 19/198 2024-03-07T07:29:39.298 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librgw2-2:19.0.0-1848.g8b896a9e.el9.x86_64 20/198 2024-03-07T07:29:39.298 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1848.g8b896a9e.el 21/198 2024-03-07T07:29:39.298 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-ceph-common-2:19.0.0-1848.g8b896a9e.el9. 22/198 2024-03-07T07:29:39.298 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cephfs-2:19.0.0-1848.g8b896a9e.el9.x86_6 23/198 2024-03-07T07:29:39.298 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rados-2:19.0.0-1848.g8b896a9e.el9.x86_64 24/198 2024-03-07T07:29:39.298 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 25/198 2024-03-07T07:29:39.298 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 26/198 2024-03-07T07:29:39.298 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : rbd-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 27/198 2024-03-07T07:29:39.298 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : rbd-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 28/198 2024-03-07T07:29:39.298 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : rbd-nbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 29/198 2024-03-07T07:29:39.298 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1848.g8b896a9e. 30/198 2024-03-07T07:29:39.306 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 128/198 2024-03-07T07:29:39.316 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1848.g8b896a9e.el9.noa 31/198 2024-03-07T07:29:39.316 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1848.g8b896a9e.el9.n 32/198 2024-03-07T07:29:39.316 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1848.g8b8 33/198 2024-03-07T07:29:39.316 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1848.g8b896a9e.el 34/198 2024-03-07T07:29:39.316 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1848.g8b896a9e.el9.noarch 35/198 2024-03-07T07:29:39.316 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1848.g8b896a9e.e 36/198 2024-03-07T07:29:39.316 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-volume-2:19.0.0-1848.g8b896a9e.el9.noarch 37/198 2024-03-07T07:29:39.316 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : cephadm-2:19.0.0-1848.g8b896a9e.el9.noarch 38/198 2024-03-07T07:29:39.316 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : cephfs-top-2:19.0.0-1848.g8b896a9e.el9.noarch 39/198 2024-03-07T07:29:39.316 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 40/198 2024-03-07T07:29:39.316 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 41/198 2024-03-07T07:29:39.317 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : jq-1.6-16.el9.x86_64 42/198 2024-03-07T07:29:39.317 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 43/198 2024-03-07T07:29:39.317 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 44/198 2024-03-07T07:29:39.317 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 45/198 2024-03-07T07:29:39.317 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 46/198 2024-03-07T07:29:39.317 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 47/198 2024-03-07T07:29:39.317 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-ply-3.11-14.el9.noarch 48/198 2024-03-07T07:29:39.317 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 49/198 2024-03-07T07:29:39.317 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 50/198 2024-03-07T07:29:39.317 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 51/198 2024-03-07T07:29:39.317 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : automake-1.16.2-8.el9.noarch 52/198 2024-03-07T07:29:39.317 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : bison-3.7.4-5.el9.x86_64 53/198 2024-03-07T07:29:39.317 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 54/198 2024-03-07T07:29:39.318 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 55/198 2024-03-07T07:29:39.318 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : flex-2.6.4-9.el9.x86_64 56/198 2024-03-07T07:29:39.318 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 57/198 2024-03-07T07:29:39.318 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 58/198 2024-03-07T07:29:39.318 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 59/198 2024-03-07T07:29:39.318 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 60/198 2024-03-07T07:29:39.318 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 61/198 2024-03-07T07:29:39.318 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 62/198 2024-03-07T07:29:39.318 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 63/198 2024-03-07T07:29:39.318 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 64/198 2024-03-07T07:29:39.318 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 65/198 2024-03-07T07:29:39.318 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 66/198 2024-03-07T07:29:39.318 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : lua-5.4.4-4.el9.x86_64 67/198 2024-03-07T07:29:39.318 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 68/198 2024-03-07T07:29:39.319 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 69/198 2024-03-07T07:29:39.319 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 70/198 2024-03-07T07:29:39.319 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : postgresql-13.11-1.el9.x86_64 71/198 2024-03-07T07:29:39.319 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 72/198 2024-03-07T07:29:39.319 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 73/198 2024-03-07T07:29:39.319 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 74/198 2024-03-07T07:29:39.319 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 75/198 2024-03-07T07:29:39.319 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 76/198 2024-03-07T07:29:39.319 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 77/198 2024-03-07T07:29:39.319 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 78/198 2024-03-07T07:29:39.319 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 79/198 2024-03-07T07:29:39.319 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/198 2024-03-07T07:29:39.319 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/198 2024-03-07T07:29:39.320 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/198 2024-03-07T07:29:39.320 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/198 2024-03-07T07:29:39.320 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/198 2024-03-07T07:29:39.320 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/198 2024-03-07T07:29:39.320 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/198 2024-03-07T07:29:39.320 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/198 2024-03-07T07:29:39.320 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/198 2024-03-07T07:29:39.320 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/198 2024-03-07T07:29:39.320 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/198 2024-03-07T07:29:39.320 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/198 2024-03-07T07:29:39.320 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/198 2024-03-07T07:29:39.320 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/198 2024-03-07T07:29:39.320 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/198 2024-03-07T07:29:39.321 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/198 2024-03-07T07:29:39.321 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/198 2024-03-07T07:29:39.321 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 97/198 2024-03-07T07:29:39.321 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 98/198 2024-03-07T07:29:39.321 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 99/198 2024-03-07T07:29:39.321 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 100/198 2024-03-07T07:29:39.321 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/198 2024-03-07T07:29:39.321 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 102/198 2024-03-07T07:29:39.321 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/198 2024-03-07T07:29:39.321 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 104/198 2024-03-07T07:29:39.321 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-07T07:29:39.321 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-07T07:29:39.321 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-07T07:29:39.321 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-07T07:29:39.322 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-07T07:29:39.322 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-07T07:29:39.322 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-07T07:29:39.322 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-07T07:29:39.322 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-07T07:29:39.322 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-07T07:29:39.322 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-07T07:29:39.322 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-07T07:29:39.322 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-07T07:29:39.322 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-07T07:29:39.322 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-07T07:29:39.322 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-07T07:29:39.322 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-07T07:29:39.322 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-07T07:29:39.323 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-07T07:29:39.323 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-07T07:29:39.323 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-07T07:29:39.323 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-07T07:29:39.323 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-07T07:29:39.323 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-07T07:29:39.323 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-07T07:29:39.323 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-07T07:29:39.323 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-07T07:29:39.323 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-07T07:29:39.323 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-07T07:29:39.323 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-07T07:29:39.323 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-07T07:29:39.323 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-07T07:29:39.324 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-07T07:29:39.324 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-07T07:29:39.324 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-07T07:29:39.324 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-07T07:29:39.324 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-07T07:29:39.324 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-07T07:29:39.324 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-07T07:29:39.324 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-07T07:29:39.324 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-07T07:29:39.324 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-07T07:29:39.324 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-07T07:29:39.324 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-07T07:29:39.325 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-07T07:29:39.325 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-07T07:29:39.325 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librados2-2:19.0.0-1848.g8b896a9e.el9.x86_64 151/198 2024-03-07T07:29:39.325 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-07T07:29:39.325 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librbd1-2:19.0.0-1848.g8b896a9e.el9.x86_64 153/198 2024-03-07T07:29:39.325 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-07T07:29:39.325 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 155/198 2024-03-07T07:29:39.325 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-03-07T07:29:39.325 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 157/198 2024-03-07T07:29:39.325 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-03-07T07:29:39.325 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 159/198 2024-03-07T07:29:39.325 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-03-07T07:29:39.325 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 161/198 2024-03-07T07:29:39.325 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-03-07T07:29:39.326 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : acl-2.3.1-4.el9.x86_64 163/198 2024-03-07T07:29:39.326 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : acl-2.3.1-3.el9.x86_64 164/198 2024-03-07T07:29:39.326 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/198 2024-03-07T07:29:39.326 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 166/198 2024-03-07T07:29:39.326 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 167/198 2024-03-07T07:29:39.326 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 168/198 2024-03-07T07:29:39.326 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 169/198 2024-03-07T07:29:39.326 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 170/198 2024-03-07T07:29:39.326 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 171/198 2024-03-07T07:29:39.326 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 172/198 2024-03-07T07:29:39.326 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 173/198 2024-03-07T07:29:39.326 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 174/198 2024-03-07T07:29:39.326 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 175/198 2024-03-07T07:29:39.326 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 176/198 2024-03-07T07:29:39.327 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 177/198 2024-03-07T07:29:39.327 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 178/198 2024-03-07T07:29:39.327 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 179/198 2024-03-07T07:29:39.327 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 180/198 2024-03-07T07:29:39.327 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 181/198 2024-03-07T07:29:39.327 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 182/198 2024-03-07T07:29:39.327 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 183/198 2024-03-07T07:29:39.327 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 184/198 2024-03-07T07:29:39.327 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 185/198 2024-03-07T07:29:39.327 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 186/198 2024-03-07T07:29:39.327 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 187/198 2024-03-07T07:29:39.327 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 188/198 2024-03-07T07:29:39.327 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 189/198 2024-03-07T07:29:39.327 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 190/198 2024-03-07T07:29:39.328 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 191/198 2024-03-07T07:29:39.328 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 192/198 2024-03-07T07:29:39.328 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 193/198 2024-03-07T07:29:39.328 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 194/198 2024-03-07T07:29:39.328 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-07T07:29:39.328 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-07T07:29:39.328 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-07T07:29:39.360 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 129/198 2024-03-07T07:29:39.429 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 130/198 2024-03-07T07:29:39.519 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 131/198 2024-03-07T07:29:39.659 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 132/198 2024-03-07T07:29:39.701 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 133/198 2024-03-07T07:29:39.735 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 134/198 2024-03-07T07:29:39.766 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 135/198 2024-03-07T07:29:39.801 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : jq-1.6-16.el9.x86_64 136/198 2024-03-07T07:29:39.829 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : fuse-2.9.9-15.el9.x86_64 137/198 2024-03-07T07:29:40.046 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-07T07:29:40.046 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-03-07T07:29:40.046 INFO:teuthology.orchestra.run.smithi178.stdout:Upgraded: 2024-03-07T07:29:40.046 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-07T07:29:40.046 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-07T07:29:40.046 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-07T07:29:40.046 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-07T07:29:40.046 INFO:teuthology.orchestra.run.smithi178.stdout: acl-2.3.1-4.el9.x86_64 2024-03-07T07:29:40.046 INFO:teuthology.orchestra.run.smithi178.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-07T07:29:40.046 INFO:teuthology.orchestra.run.smithi178.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-07T07:29:40.047 INFO:teuthology.orchestra.run.smithi178.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-07T07:29:40.047 INFO:teuthology.orchestra.run.smithi178.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-07T07:29:40.047 INFO:teuthology.orchestra.run.smithi178.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-07T07:29:40.047 INFO:teuthology.orchestra.run.smithi178.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-07T07:29:40.047 INFO:teuthology.orchestra.run.smithi178.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-07T07:29:40.047 INFO:teuthology.orchestra.run.smithi178.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-07T07:29:40.047 INFO:teuthology.orchestra.run.smithi178.stdout: librados2-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:40.047 INFO:teuthology.orchestra.run.smithi178.stdout: librbd1-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:40.047 INFO:teuthology.orchestra.run.smithi178.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-07T07:29:40.047 INFO:teuthology.orchestra.run.smithi178.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-07T07:29:40.047 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-07T07:29:40.047 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-07T07:29:40.047 INFO:teuthology.orchestra.run.smithi178.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-07T07:29:40.047 INFO:teuthology.orchestra.run.smithi178.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-07T07:29:40.048 INFO:teuthology.orchestra.run.smithi178.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-07T07:29:40.048 INFO:teuthology.orchestra.run.smithi178.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-07T07:29:40.048 INFO:teuthology.orchestra.run.smithi178.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-07T07:29:40.048 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-03-07T07:29:40.048 INFO:teuthology.orchestra.run.smithi178.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-07T07:29:40.048 INFO:teuthology.orchestra.run.smithi178.stdout: automake-1.16.2-8.el9.noarch 2024-03-07T07:29:40.048 INFO:teuthology.orchestra.run.smithi178.stdout: bison-3.7.4-5.el9.x86_64 2024-03-07T07:29:40.048 INFO:teuthology.orchestra.run.smithi178.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-07T07:29:40.048 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:40.048 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-base-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:40.048 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:40.048 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:40.048 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-grafana-dashboards-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:29:40.049 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-immutable-object-cache-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:40.049 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mds-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:40.049 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:40.049 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-cephadm-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:29:40.049 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-dashboard-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:29:40.049 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:29:40.049 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-modules-core-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:29:40.049 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-rook-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:29:40.049 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mon-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:40.049 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-osd-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:40.049 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-prometheus-alerts-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:29:40.049 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-radosgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:40.049 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-selinux-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:40.049 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-test-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:40.050 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-volume-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:29:40.050 INFO:teuthology.orchestra.run.smithi178.stdout: cephadm-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:29:40.050 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:40.050 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-shell-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:40.050 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-top-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:29:40.050 INFO:teuthology.orchestra.run.smithi178.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-07T07:29:40.050 INFO:teuthology.orchestra.run.smithi178.stdout: flex-2.6.4-9.el9.x86_64 2024-03-07T07:29:40.050 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-07T07:29:40.050 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-07T07:29:40.050 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-07T07:29:40.050 INFO:teuthology.orchestra.run.smithi178.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-07T07:29:40.050 INFO:teuthology.orchestra.run.smithi178.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-07T07:29:40.050 INFO:teuthology.orchestra.run.smithi178.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-07T07:29:40.050 INFO:teuthology.orchestra.run.smithi178.stdout: jq-1.6-16.el9.x86_64 2024-03-07T07:29:40.051 INFO:teuthology.orchestra.run.smithi178.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-03-07T07:29:40.051 INFO:teuthology.orchestra.run.smithi178.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-07T07:29:40.051 INFO:teuthology.orchestra.run.smithi178.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-07T07:29:40.051 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs-devel-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:40.051 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs2-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:40.051 INFO:teuthology.orchestra.run.smithi178.stdout: libcephsqlite-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:40.051 INFO:teuthology.orchestra.run.smithi178.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-07T07:29:40.051 INFO:teuthology.orchestra.run.smithi178.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-07T07:29:40.051 INFO:teuthology.orchestra.run.smithi178.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-07T07:29:40.051 INFO:teuthology.orchestra.run.smithi178.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-07T07:29:40.051 INFO:teuthology.orchestra.run.smithi178.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-07T07:29:40.051 INFO:teuthology.orchestra.run.smithi178.stdout: librados-devel-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:40.051 INFO:teuthology.orchestra.run.smithi178.stdout: libradosstriper1-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:40.051 INFO:teuthology.orchestra.run.smithi178.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-07T07:29:40.052 INFO:teuthology.orchestra.run.smithi178.stdout: librgw2-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:40.052 INFO:teuthology.orchestra.run.smithi178.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-07T07:29:40.052 INFO:teuthology.orchestra.run.smithi178.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-07T07:29:40.052 INFO:teuthology.orchestra.run.smithi178.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-07T07:29:40.052 INFO:teuthology.orchestra.run.smithi178.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-07T07:29:40.052 INFO:teuthology.orchestra.run.smithi178.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-07T07:29:40.052 INFO:teuthology.orchestra.run.smithi178.stdout: lua-5.4.4-4.el9.x86_64 2024-03-07T07:29:40.052 INFO:teuthology.orchestra.run.smithi178.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-07T07:29:40.052 INFO:teuthology.orchestra.run.smithi178.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-07T07:29:40.052 INFO:teuthology.orchestra.run.smithi178.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-07T07:29:40.052 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-07T07:29:40.052 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-07T07:29:40.052 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-07T07:29:40.053 INFO:teuthology.orchestra.run.smithi178.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-07T07:29:40.053 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-13.11-1.el9.x86_64 2024-03-07T07:29:40.053 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-07T07:29:40.053 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-07T07:29:40.053 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-07T07:29:40.053 INFO:teuthology.orchestra.run.smithi178.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-07T07:29:40.053 INFO:teuthology.orchestra.run.smithi178.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-07T07:29:40.053 INFO:teuthology.orchestra.run.smithi178.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-07T07:29:40.053 INFO:teuthology.orchestra.run.smithi178.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-07T07:29:40.053 INFO:teuthology.orchestra.run.smithi178.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-07T07:29:40.053 INFO:teuthology.orchestra.run.smithi178.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-07T07:29:40.053 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-07T07:29:40.053 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-argparse-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:40.054 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:40.054 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cephfs-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:40.054 INFO:teuthology.orchestra.run.smithi178.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-07T07:29:40.054 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-07T07:29:40.054 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-07T07:29:40.054 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-07T07:29:40.054 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-07T07:29:40.054 INFO:teuthology.orchestra.run.smithi178.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-07T07:29:40.054 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-07T07:29:40.054 INFO:teuthology.orchestra.run.smithi178.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-07T07:29:40.054 INFO:teuthology.orchestra.run.smithi178.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-07T07:29:40.054 INFO:teuthology.orchestra.run.smithi178.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-07T07:29:40.054 INFO:teuthology.orchestra.run.smithi178.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-07T07:29:40.054 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-07T07:29:40.055 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-07T07:29:40.055 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-07T07:29:40.055 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-07T07:29:40.055 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-07T07:29:40.055 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-07T07:29:40.055 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-07T07:29:40.055 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-07T07:29:40.055 INFO:teuthology.orchestra.run.smithi178.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-07T07:29:40.055 INFO:teuthology.orchestra.run.smithi178.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-07T07:29:40.055 INFO:teuthology.orchestra.run.smithi178.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-07T07:29:40.055 INFO:teuthology.orchestra.run.smithi178.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-07T07:29:40.055 INFO:teuthology.orchestra.run.smithi178.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-07T07:29:40.055 INFO:teuthology.orchestra.run.smithi178.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-07T07:29:40.055 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-07T07:29:40.055 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-07T07:29:40.056 INFO:teuthology.orchestra.run.smithi178.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-07T07:29:40.056 INFO:teuthology.orchestra.run.smithi178.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-07T07:29:40.056 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-07T07:29:40.056 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ply-3.11-14.el9.noarch 2024-03-07T07:29:40.056 INFO:teuthology.orchestra.run.smithi178.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-07T07:29:40.056 INFO:teuthology.orchestra.run.smithi178.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-07T07:29:40.056 INFO:teuthology.orchestra.run.smithi178.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-07T07:29:40.056 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-07T07:29:40.056 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-07T07:29:40.056 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-07T07:29:40.056 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-07T07:29:40.056 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-07T07:29:40.056 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-07T07:29:40.056 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rados-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:40.057 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:40.057 INFO:teuthology.orchestra.run.smithi178.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-07T07:29:40.057 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-07T07:29:40.057 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-07T07:29:40.057 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:40.057 INFO:teuthology.orchestra.run.smithi178.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-07T07:29:40.057 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-07T07:29:40.057 INFO:teuthology.orchestra.run.smithi178.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-07T07:29:40.057 INFO:teuthology.orchestra.run.smithi178.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-07T07:29:40.057 INFO:teuthology.orchestra.run.smithi178.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-07T07:29:40.057 INFO:teuthology.orchestra.run.smithi178.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-07T07:29:40.057 INFO:teuthology.orchestra.run.smithi178.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-07T07:29:40.057 INFO:teuthology.orchestra.run.smithi178.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-07T07:29:40.058 INFO:teuthology.orchestra.run.smithi178.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-07T07:29:40.058 INFO:teuthology.orchestra.run.smithi178.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-07T07:29:40.058 INFO:teuthology.orchestra.run.smithi178.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-07T07:29:40.058 INFO:teuthology.orchestra.run.smithi178.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-07T07:29:40.058 INFO:teuthology.orchestra.run.smithi178.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-07T07:29:40.058 INFO:teuthology.orchestra.run.smithi178.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-07T07:29:40.058 INFO:teuthology.orchestra.run.smithi178.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-07T07:29:40.058 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:40.058 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:40.058 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-nbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:40.058 INFO:teuthology.orchestra.run.smithi178.stdout: re2-1:20211101-3.el9.x86_64 2024-03-07T07:29:40.058 INFO:teuthology.orchestra.run.smithi178.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-07T07:29:40.058 INFO:teuthology.orchestra.run.smithi178.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-07T07:29:40.058 INFO:teuthology.orchestra.run.smithi178.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-07T07:29:40.059 INFO:teuthology.orchestra.run.smithi178.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-07T07:29:40.059 INFO:teuthology.orchestra.run.smithi178.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-07T07:29:40.059 INFO:teuthology.orchestra.run.smithi178.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-07T07:29:40.059 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-03-07T07:29:40.059 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-03-07T07:29:40.256 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: cephadm-2:19.0.0-1848.g8b896a9e.el9.noarch 138/198 2024-03-07T07:29:40.286 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : cephadm-2:19.0.0-1848.g8b896a9e.el9.noarch 138/198 2024-03-07T07:29:40.317 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1848.g8b896a9e.e 139/198 2024-03-07T07:29:40.378 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1848.g8b896a9e. 140/198 2024-03-07T07:29:40.409 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1848.g8b896a9e.el9.noa 141/198 2024-03-07T07:29:40.574 DEBUG:teuthology.parallel:result is None 2024-03-07T07:29:40.638 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1848.g8b896a9e.el9.noa 141/198 2024-03-07T07:29:40.668 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1848.g8b896a9e.el9.n 142/198 2024-03-07T07:29:41.638 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1848.g8b896a9e.el9.n 142/198 2024-03-07T07:29:41.660 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1848.g8b8 143/198 2024-03-07T07:29:41.761 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1848.g8b8 143/198 2024-03-07T07:29:41.931 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1848.g8b896a9e.el 144/198 2024-03-07T07:29:41.971 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-2:19.0.0-1848.g8b896a9e.el9.x86_64 145/198 2024-03-07T07:29:41.993 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1848.g8b896a9e.el9.x86_64 145/198 2024-03-07T07:29:41.993 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T07:29:41.993 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-07T07:29:41.993 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-07T07:29:41.993 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-07T07:29:41.994 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:29:42.037 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-rook-2:19.0.0-1848.g8b896a9e.el9.noarch 146/198 2024-03-07T07:29:42.049 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1848.g8b896a9e.el9.noarch 146/198 2024-03-07T07:29:42.154 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-2:19.0.0-1848.g8b896a9e.el9.x86_64 147/198 2024-03-07T07:29:42.473 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-07T07:29:42.474 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-07T07:29:42.474 INFO:teuthology.orchestra.run.smithi177.stdout:Upgraded: 2024-03-07T07:29:42.474 INFO:teuthology.orchestra.run.smithi177.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-07T07:29:42.474 INFO:teuthology.orchestra.run.smithi177.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-07T07:29:42.474 INFO:teuthology.orchestra.run.smithi177.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-07T07:29:42.474 INFO:teuthology.orchestra.run.smithi177.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-07T07:29:42.474 INFO:teuthology.orchestra.run.smithi177.stdout: acl-2.3.1-4.el9.x86_64 2024-03-07T07:29:42.474 INFO:teuthology.orchestra.run.smithi177.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-07T07:29:42.474 INFO:teuthology.orchestra.run.smithi177.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-07T07:29:42.474 INFO:teuthology.orchestra.run.smithi177.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-07T07:29:42.475 INFO:teuthology.orchestra.run.smithi177.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-07T07:29:42.475 INFO:teuthology.orchestra.run.smithi177.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-07T07:29:42.475 INFO:teuthology.orchestra.run.smithi177.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-07T07:29:42.475 INFO:teuthology.orchestra.run.smithi177.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-07T07:29:42.475 INFO:teuthology.orchestra.run.smithi177.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-07T07:29:42.475 INFO:teuthology.orchestra.run.smithi177.stdout: librados2-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:42.475 INFO:teuthology.orchestra.run.smithi177.stdout: librbd1-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:42.475 INFO:teuthology.orchestra.run.smithi177.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-07T07:29:42.475 INFO:teuthology.orchestra.run.smithi177.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-07T07:29:42.475 INFO:teuthology.orchestra.run.smithi177.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-07T07:29:42.475 INFO:teuthology.orchestra.run.smithi177.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-07T07:29:42.475 INFO:teuthology.orchestra.run.smithi177.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-07T07:29:42.475 INFO:teuthology.orchestra.run.smithi177.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-07T07:29:42.476 INFO:teuthology.orchestra.run.smithi177.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-07T07:29:42.476 INFO:teuthology.orchestra.run.smithi177.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-07T07:29:42.476 INFO:teuthology.orchestra.run.smithi177.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-07T07:29:42.476 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-03-07T07:29:42.476 INFO:teuthology.orchestra.run.smithi177.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-07T07:29:42.476 INFO:teuthology.orchestra.run.smithi177.stdout: automake-1.16.2-8.el9.noarch 2024-03-07T07:29:42.476 INFO:teuthology.orchestra.run.smithi177.stdout: bison-3.7.4-5.el9.x86_64 2024-03-07T07:29:42.476 INFO:teuthology.orchestra.run.smithi177.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-07T07:29:42.476 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:42.476 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-base-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:42.476 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:42.476 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:42.476 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-grafana-dashboards-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:29:42.476 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-immutable-object-cache-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:42.477 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mds-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:42.477 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:42.477 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-cephadm-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:29:42.477 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-dashboard-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:29:42.477 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:29:42.477 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-modules-core-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:29:42.477 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-rook-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:29:42.477 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mon-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:42.477 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-osd-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:42.477 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-prometheus-alerts-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:29:42.477 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-radosgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:42.477 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-selinux-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:42.477 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-test-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:42.477 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-volume-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:29:42.477 INFO:teuthology.orchestra.run.smithi177.stdout: cephadm-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:29:42.478 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:42.478 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-shell-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:42.478 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-top-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:29:42.478 INFO:teuthology.orchestra.run.smithi177.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-07T07:29:42.478 INFO:teuthology.orchestra.run.smithi177.stdout: flex-2.6.4-9.el9.x86_64 2024-03-07T07:29:42.478 INFO:teuthology.orchestra.run.smithi177.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-07T07:29:42.478 INFO:teuthology.orchestra.run.smithi177.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-07T07:29:42.478 INFO:teuthology.orchestra.run.smithi177.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-07T07:29:42.478 INFO:teuthology.orchestra.run.smithi177.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-07T07:29:42.478 INFO:teuthology.orchestra.run.smithi177.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-07T07:29:42.478 INFO:teuthology.orchestra.run.smithi177.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-07T07:29:42.478 INFO:teuthology.orchestra.run.smithi177.stdout: jq-1.6-16.el9.x86_64 2024-03-07T07:29:42.478 INFO:teuthology.orchestra.run.smithi177.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-03-07T07:29:42.478 INFO:teuthology.orchestra.run.smithi177.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-07T07:29:42.478 INFO:teuthology.orchestra.run.smithi177.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-07T07:29:42.478 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs-devel-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:42.479 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs2-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:42.479 INFO:teuthology.orchestra.run.smithi177.stdout: libcephsqlite-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:42.479 INFO:teuthology.orchestra.run.smithi177.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-07T07:29:42.479 INFO:teuthology.orchestra.run.smithi177.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-07T07:29:42.479 INFO:teuthology.orchestra.run.smithi177.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-07T07:29:42.479 INFO:teuthology.orchestra.run.smithi177.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-07T07:29:42.479 INFO:teuthology.orchestra.run.smithi177.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-07T07:29:42.479 INFO:teuthology.orchestra.run.smithi177.stdout: librados-devel-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:42.479 INFO:teuthology.orchestra.run.smithi177.stdout: libradosstriper1-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:42.479 INFO:teuthology.orchestra.run.smithi177.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-07T07:29:42.479 INFO:teuthology.orchestra.run.smithi177.stdout: librgw2-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:42.479 INFO:teuthology.orchestra.run.smithi177.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-07T07:29:42.479 INFO:teuthology.orchestra.run.smithi177.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-07T07:29:42.479 INFO:teuthology.orchestra.run.smithi177.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-07T07:29:42.479 INFO:teuthology.orchestra.run.smithi177.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-07T07:29:42.479 INFO:teuthology.orchestra.run.smithi177.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-07T07:29:42.480 INFO:teuthology.orchestra.run.smithi177.stdout: lua-5.4.4-4.el9.x86_64 2024-03-07T07:29:42.480 INFO:teuthology.orchestra.run.smithi177.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-07T07:29:42.480 INFO:teuthology.orchestra.run.smithi177.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-07T07:29:42.480 INFO:teuthology.orchestra.run.smithi177.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-07T07:29:42.480 INFO:teuthology.orchestra.run.smithi177.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-07T07:29:42.480 INFO:teuthology.orchestra.run.smithi177.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-07T07:29:42.480 INFO:teuthology.orchestra.run.smithi177.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-07T07:29:42.480 INFO:teuthology.orchestra.run.smithi177.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-07T07:29:42.480 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-13.11-1.el9.x86_64 2024-03-07T07:29:42.480 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-07T07:29:42.480 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-07T07:29:42.481 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-07T07:29:42.481 INFO:teuthology.orchestra.run.smithi177.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-07T07:29:42.481 INFO:teuthology.orchestra.run.smithi177.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-07T07:29:42.481 INFO:teuthology.orchestra.run.smithi177.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-07T07:29:42.481 INFO:teuthology.orchestra.run.smithi177.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-07T07:29:42.481 INFO:teuthology.orchestra.run.smithi177.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-07T07:29:42.481 INFO:teuthology.orchestra.run.smithi177.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-07T07:29:42.481 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-07T07:29:42.481 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-argparse-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:42.481 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:42.481 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cephfs-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:42.481 INFO:teuthology.orchestra.run.smithi177.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-07T07:29:42.482 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-07T07:29:42.482 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-07T07:29:42.482 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-07T07:29:42.482 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-07T07:29:42.482 INFO:teuthology.orchestra.run.smithi177.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-07T07:29:42.482 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-07T07:29:42.482 INFO:teuthology.orchestra.run.smithi177.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-07T07:29:42.483 INFO:teuthology.orchestra.run.smithi177.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-07T07:29:42.483 INFO:teuthology.orchestra.run.smithi177.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-07T07:29:42.483 INFO:teuthology.orchestra.run.smithi177.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-07T07:29:42.483 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-07T07:29:42.483 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-07T07:29:42.483 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-07T07:29:42.483 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-07T07:29:42.483 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-07T07:29:42.483 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-07T07:29:42.483 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-07T07:29:42.483 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-07T07:29:42.483 INFO:teuthology.orchestra.run.smithi177.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-07T07:29:42.483 INFO:teuthology.orchestra.run.smithi177.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-07T07:29:42.483 INFO:teuthology.orchestra.run.smithi177.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-07T07:29:42.484 INFO:teuthology.orchestra.run.smithi177.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-07T07:29:42.484 INFO:teuthology.orchestra.run.smithi177.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-07T07:29:42.484 INFO:teuthology.orchestra.run.smithi177.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-07T07:29:42.484 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-07T07:29:42.484 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-07T07:29:42.484 INFO:teuthology.orchestra.run.smithi177.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-07T07:29:42.484 INFO:teuthology.orchestra.run.smithi177.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-07T07:29:42.484 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-07T07:29:42.484 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ply-3.11-14.el9.noarch 2024-03-07T07:29:42.484 INFO:teuthology.orchestra.run.smithi177.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-07T07:29:42.484 INFO:teuthology.orchestra.run.smithi177.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-07T07:29:42.484 INFO:teuthology.orchestra.run.smithi177.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-07T07:29:42.484 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-07T07:29:42.484 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-07T07:29:42.484 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-07T07:29:42.485 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-07T07:29:42.485 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-07T07:29:42.485 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-07T07:29:42.485 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rados-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:42.485 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:42.485 INFO:teuthology.orchestra.run.smithi177.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-07T07:29:42.485 INFO:teuthology.orchestra.run.smithi177.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-07T07:29:42.485 INFO:teuthology.orchestra.run.smithi177.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-07T07:29:42.485 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:42.485 INFO:teuthology.orchestra.run.smithi177.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-07T07:29:42.485 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-07T07:29:42.485 INFO:teuthology.orchestra.run.smithi177.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-07T07:29:42.485 INFO:teuthology.orchestra.run.smithi177.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-07T07:29:42.485 INFO:teuthology.orchestra.run.smithi177.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-07T07:29:42.486 INFO:teuthology.orchestra.run.smithi177.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-07T07:29:42.486 INFO:teuthology.orchestra.run.smithi177.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-07T07:29:42.486 INFO:teuthology.orchestra.run.smithi177.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-07T07:29:42.486 INFO:teuthology.orchestra.run.smithi177.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-07T07:29:42.486 INFO:teuthology.orchestra.run.smithi177.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-07T07:29:42.486 INFO:teuthology.orchestra.run.smithi177.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-07T07:29:42.486 INFO:teuthology.orchestra.run.smithi177.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-07T07:29:42.486 INFO:teuthology.orchestra.run.smithi177.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-07T07:29:42.486 INFO:teuthology.orchestra.run.smithi177.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-07T07:29:42.486 INFO:teuthology.orchestra.run.smithi177.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-07T07:29:42.486 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:42.486 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:42.486 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-nbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:42.486 INFO:teuthology.orchestra.run.smithi177.stdout: re2-1:20211101-3.el9.x86_64 2024-03-07T07:29:42.486 INFO:teuthology.orchestra.run.smithi177.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-07T07:29:42.487 INFO:teuthology.orchestra.run.smithi177.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-07T07:29:42.487 INFO:teuthology.orchestra.run.smithi177.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-07T07:29:42.487 INFO:teuthology.orchestra.run.smithi177.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-07T07:29:42.487 INFO:teuthology.orchestra.run.smithi177.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-07T07:29:42.487 INFO:teuthology.orchestra.run.smithi177.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-07T07:29:42.487 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-07T07:29:42.487 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-03-07T07:29:42.994 DEBUG:teuthology.parallel:result is None 2024-03-07T07:29:46.594 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 148/198 2024-03-07T07:29:46.666 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-test-2:19.0.0-1848.g8b896a9e.el9.x86_64 149/198 2024-03-07T07:29:47.528 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libtool-2.4.6-45.el9.x86_64 150/198 2024-03-07T07:29:47.553 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-radosgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 151/198 2024-03-07T07:29:47.576 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 151/198 2024-03-07T07:29:47.576 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T07:29:47.576 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-07T07:29:47.576 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-07T07:29:47.576 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-07T07:29:47.576 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:29:47.620 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1848.g8b896 152/198 2024-03-07T07:29:47.642 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1848.g8b896 152/198 2024-03-07T07:29:47.642 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T07:29:47.642 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-07T07:29:47.642 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:29:47.704 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : cephfs-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 153/198 2024-03-07T07:29:47.726 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 153/198 2024-03-07T07:29:47.726 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T07:29:47.726 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-07T07:29:47.727 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-07T07:29:47.727 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-07T07:29:47.727 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:29:48.030 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : rbd-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 154/198 2024-03-07T07:29:48.053 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 154/198 2024-03-07T07:29:48.053 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T07:29:48.053 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-07T07:29:48.053 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-07T07:29:48.053 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-07T07:29:48.053 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:29:48.104 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : rbd-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 155/198 2024-03-07T07:29:48.135 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : rbd-nbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 156/198 2024-03-07T07:29:48.162 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 157/198 2024-03-07T07:29:48.349 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : cephfs-shell-2:19.0.0-1848.g8b896a9e.el9.x86_64 158/198 2024-03-07T07:29:48.418 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 159/198 2024-03-07T07:29:48.466 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-07T07:29:48.534 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-07T07:29:48.577 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-07T07:29:48.610 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libcephfs-devel-2:19.0.0-1848.g8b896a9e.el9.x86_ 163/198 2024-03-07T07:29:48.651 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-07T07:29:48.697 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-07T07:29:48.767 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-07T07:29:48.831 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-07T07:29:48.871 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-07T07:29:49.450 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-07T07:29:49.493 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-07T07:29:49.513 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-07T07:29:50.357 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : cephfs-top-2:19.0.0-1848.g8b896a9e.el9.noarch 170/198 2024-03-07T07:29:50.438 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-07T07:29:50.519 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : flex-2.6.4-9.el9.x86_64 172/198 2024-03-07T07:29:50.557 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : bison-3.7.4-5.el9.x86_64 173/198 2024-03-07T07:29:50.583 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 174/198 2024-03-07T07:29:50.608 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-07T07:29:50.633 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-07T07:29:50.659 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-07T07:29:50.659 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-07T07:29:50.741 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-07T07:29:50.771 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-07T07:29:50.771 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-07T07:29:50.855 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-07T07:29:50.882 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-07T07:29:50.914 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-07T07:29:50.940 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-07T07:29:50.965 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-07T07:29:50.999 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-07T07:29:51.047 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-07T07:29:51.094 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-07T07:29:51.120 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-07T07:29:51.145 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-07T07:29:51.170 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-07T07:29:51.196 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-07T07:29:51.202 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-07T07:29:51.202 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-07T07:29:51.382 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-07T07:29:51.382 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-07T07:29:51.465 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-07T07:29:51.489 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-07T07:29:51.515 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-07T07:29:51.540 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-07T07:29:51.581 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-07T07:29:51.648 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-07T07:29:56.680 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-07T07:29:56.680 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-2:19.0.0-1848.g8b896a9e.el9.x86_64 1/198 2024-03-07T07:29:56.681 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-base-2:19.0.0-1848.g8b896a9e.el9.x86_64 2/198 2024-03-07T07:29:56.681 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 3/198 2024-03-07T07:29:56.681 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 4/198 2024-03-07T07:29:56.681 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1848.g8b896 5/198 2024-03-07T07:29:56.681 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mds-2:19.0.0-1848.g8b896a9e.el9.x86_64 6/198 2024-03-07T07:29:56.681 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-2:19.0.0-1848.g8b896a9e.el9.x86_64 7/198 2024-03-07T07:29:56.681 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mon-2:19.0.0-1848.g8b896a9e.el9.x86_64 8/198 2024-03-07T07:29:56.681 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-osd-2:19.0.0-1848.g8b896a9e.el9.x86_64 9/198 2024-03-07T07:29:56.681 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-radosgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 10/198 2024-03-07T07:29:56.681 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-selinux-2:19.0.0-1848.g8b896a9e.el9.x86_64 11/198 2024-03-07T07:29:56.681 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-test-2:19.0.0-1848.g8b896a9e.el9.x86_64 12/198 2024-03-07T07:29:56.681 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : cephfs-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 13/198 2024-03-07T07:29:56.682 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : cephfs-shell-2:19.0.0-1848.g8b896a9e.el9.x86_64 14/198 2024-03-07T07:29:56.682 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libcephfs-devel-2:19.0.0-1848.g8b896a9e.el9.x86_ 15/198 2024-03-07T07:29:56.682 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libcephfs2-2:19.0.0-1848.g8b896a9e.el9.x86_64 16/198 2024-03-07T07:29:56.682 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libcephsqlite-2:19.0.0-1848.g8b896a9e.el9.x86_64 17/198 2024-03-07T07:29:56.682 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librados-devel-2:19.0.0-1848.g8b896a9e.el9.x86_6 18/198 2024-03-07T07:29:56.682 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libradosstriper1-2:19.0.0-1848.g8b896a9e.el9.x86 19/198 2024-03-07T07:29:56.682 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librgw2-2:19.0.0-1848.g8b896a9e.el9.x86_64 20/198 2024-03-07T07:29:56.682 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1848.g8b896a9e.el 21/198 2024-03-07T07:29:56.682 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-ceph-common-2:19.0.0-1848.g8b896a9e.el9. 22/198 2024-03-07T07:29:56.682 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cephfs-2:19.0.0-1848.g8b896a9e.el9.x86_6 23/198 2024-03-07T07:29:56.682 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-rados-2:19.0.0-1848.g8b896a9e.el9.x86_64 24/198 2024-03-07T07:29:56.682 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-rbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 25/198 2024-03-07T07:29:56.682 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-rgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 26/198 2024-03-07T07:29:56.683 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : rbd-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 27/198 2024-03-07T07:29:56.683 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : rbd-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 28/198 2024-03-07T07:29:56.683 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : rbd-nbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 29/198 2024-03-07T07:29:56.683 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1848.g8b896a9e. 30/198 2024-03-07T07:29:56.683 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1848.g8b896a9e.el9.noa 31/198 2024-03-07T07:29:56.683 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1848.g8b896a9e.el9.n 32/198 2024-03-07T07:29:56.683 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1848.g8b8 33/198 2024-03-07T07:29:56.683 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1848.g8b896a9e.el 34/198 2024-03-07T07:29:56.683 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1848.g8b896a9e.el9.noarch 35/198 2024-03-07T07:29:56.683 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1848.g8b896a9e.e 36/198 2024-03-07T07:29:56.683 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-volume-2:19.0.0-1848.g8b896a9e.el9.noarch 37/198 2024-03-07T07:29:56.684 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : cephadm-2:19.0.0-1848.g8b896a9e.el9.noarch 38/198 2024-03-07T07:29:56.684 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : cephfs-top-2:19.0.0-1848.g8b896a9e.el9.noarch 39/198 2024-03-07T07:29:56.684 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 40/198 2024-03-07T07:29:56.684 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 41/198 2024-03-07T07:29:56.684 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : jq-1.6-16.el9.x86_64 42/198 2024-03-07T07:29:56.684 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 43/198 2024-03-07T07:29:56.684 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 44/198 2024-03-07T07:29:56.684 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 45/198 2024-03-07T07:29:56.684 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 46/198 2024-03-07T07:29:56.684 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 47/198 2024-03-07T07:29:56.684 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-ply-3.11-14.el9.noarch 48/198 2024-03-07T07:29:56.684 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 49/198 2024-03-07T07:29:56.685 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 50/198 2024-03-07T07:29:56.685 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 51/198 2024-03-07T07:29:56.685 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : automake-1.16.2-8.el9.noarch 52/198 2024-03-07T07:29:56.685 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : bison-3.7.4-5.el9.x86_64 53/198 2024-03-07T07:29:56.685 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 54/198 2024-03-07T07:29:56.685 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 55/198 2024-03-07T07:29:56.685 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : flex-2.6.4-9.el9.x86_64 56/198 2024-03-07T07:29:56.685 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 57/198 2024-03-07T07:29:56.685 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 58/198 2024-03-07T07:29:56.685 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 59/198 2024-03-07T07:29:56.686 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 60/198 2024-03-07T07:29:56.686 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 61/198 2024-03-07T07:29:56.686 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 62/198 2024-03-07T07:29:56.686 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 63/198 2024-03-07T07:29:56.686 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 64/198 2024-03-07T07:29:56.686 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 65/198 2024-03-07T07:29:56.686 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 66/198 2024-03-07T07:29:56.686 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : lua-5.4.4-4.el9.x86_64 67/198 2024-03-07T07:29:56.686 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 68/198 2024-03-07T07:29:56.686 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 69/198 2024-03-07T07:29:56.686 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 70/198 2024-03-07T07:29:56.687 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : postgresql-13.11-1.el9.x86_64 71/198 2024-03-07T07:29:56.687 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 72/198 2024-03-07T07:29:56.687 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 73/198 2024-03-07T07:29:56.687 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 74/198 2024-03-07T07:29:56.687 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 75/198 2024-03-07T07:29:56.687 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 76/198 2024-03-07T07:29:56.687 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 77/198 2024-03-07T07:29:56.687 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 78/198 2024-03-07T07:29:56.687 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 79/198 2024-03-07T07:29:56.687 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/198 2024-03-07T07:29:56.687 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/198 2024-03-07T07:29:56.687 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/198 2024-03-07T07:29:56.688 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/198 2024-03-07T07:29:56.688 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/198 2024-03-07T07:29:56.688 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/198 2024-03-07T07:29:56.688 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/198 2024-03-07T07:29:56.688 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/198 2024-03-07T07:29:56.688 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/198 2024-03-07T07:29:56.688 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/198 2024-03-07T07:29:56.688 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/198 2024-03-07T07:29:56.688 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/198 2024-03-07T07:29:56.688 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/198 2024-03-07T07:29:56.688 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/198 2024-03-07T07:29:56.688 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/198 2024-03-07T07:29:56.689 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/198 2024-03-07T07:29:56.689 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/198 2024-03-07T07:29:56.689 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 97/198 2024-03-07T07:29:56.689 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 98/198 2024-03-07T07:29:56.689 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 99/198 2024-03-07T07:29:56.689 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 100/198 2024-03-07T07:29:56.689 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/198 2024-03-07T07:29:56.689 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 102/198 2024-03-07T07:29:56.689 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/198 2024-03-07T07:29:56.689 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 104/198 2024-03-07T07:29:56.689 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-07T07:29:56.689 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-07T07:29:56.689 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-07T07:29:56.690 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-07T07:29:56.690 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-07T07:29:56.690 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-07T07:29:56.690 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-07T07:29:56.690 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-07T07:29:56.690 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-07T07:29:56.690 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-07T07:29:56.690 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-07T07:29:56.690 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-07T07:29:56.690 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-07T07:29:56.690 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-07T07:29:56.690 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-07T07:29:56.690 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-07T07:29:56.690 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-07T07:29:56.690 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-07T07:29:56.691 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-07T07:29:56.691 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-07T07:29:56.691 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-07T07:29:56.691 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-07T07:29:56.691 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-07T07:29:56.691 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-07T07:29:56.691 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-07T07:29:56.691 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-07T07:29:56.691 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-07T07:29:56.691 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-07T07:29:56.691 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-07T07:29:56.691 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-07T07:29:56.691 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-07T07:29:56.692 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-07T07:29:56.692 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-07T07:29:56.692 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-07T07:29:56.692 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-07T07:29:56.692 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-07T07:29:56.692 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-07T07:29:56.692 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-07T07:29:56.692 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-07T07:29:56.692 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-07T07:29:56.692 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-07T07:29:56.692 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-07T07:29:56.693 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-07T07:29:56.693 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-07T07:29:56.693 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-07T07:29:56.693 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-07T07:29:56.693 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librados2-2:19.0.0-1848.g8b896a9e.el9.x86_64 151/198 2024-03-07T07:29:56.693 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-07T07:29:56.693 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librbd1-2:19.0.0-1848.g8b896a9e.el9.x86_64 153/198 2024-03-07T07:29:56.693 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-07T07:29:56.693 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 155/198 2024-03-07T07:29:56.693 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-03-07T07:29:56.693 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 157/198 2024-03-07T07:29:56.693 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-03-07T07:29:56.694 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 159/198 2024-03-07T07:29:56.694 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-03-07T07:29:56.694 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 161/198 2024-03-07T07:29:56.694 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-03-07T07:29:56.694 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : acl-2.3.1-4.el9.x86_64 163/198 2024-03-07T07:29:56.694 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : acl-2.3.1-3.el9.x86_64 164/198 2024-03-07T07:29:56.694 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/198 2024-03-07T07:29:56.694 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 166/198 2024-03-07T07:29:56.694 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 167/198 2024-03-07T07:29:56.694 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 168/198 2024-03-07T07:29:56.694 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 169/198 2024-03-07T07:29:56.694 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 170/198 2024-03-07T07:29:56.694 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 171/198 2024-03-07T07:29:56.694 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 172/198 2024-03-07T07:29:56.694 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 173/198 2024-03-07T07:29:56.695 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 174/198 2024-03-07T07:29:56.695 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 175/198 2024-03-07T07:29:56.695 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 176/198 2024-03-07T07:29:56.695 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 177/198 2024-03-07T07:29:56.695 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 178/198 2024-03-07T07:29:56.695 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 179/198 2024-03-07T07:29:56.695 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 180/198 2024-03-07T07:29:56.695 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 181/198 2024-03-07T07:29:56.695 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 182/198 2024-03-07T07:29:56.695 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 183/198 2024-03-07T07:29:56.695 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 184/198 2024-03-07T07:29:56.695 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 185/198 2024-03-07T07:29:56.695 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 186/198 2024-03-07T07:29:56.696 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 187/198 2024-03-07T07:29:56.696 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 188/198 2024-03-07T07:29:56.696 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 189/198 2024-03-07T07:29:56.696 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 190/198 2024-03-07T07:29:56.696 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 191/198 2024-03-07T07:29:56.696 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 192/198 2024-03-07T07:29:56.696 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 193/198 2024-03-07T07:29:56.696 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 194/198 2024-03-07T07:29:56.696 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-07T07:29:56.696 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-07T07:29:56.696 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-07T07:29:59.649 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-07T07:29:59.649 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:29:59.649 INFO:teuthology.orchestra.run.smithi160.stdout:Upgraded: 2024-03-07T07:29:59.650 INFO:teuthology.orchestra.run.smithi160.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-07T07:29:59.650 INFO:teuthology.orchestra.run.smithi160.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-07T07:29:59.650 INFO:teuthology.orchestra.run.smithi160.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-07T07:29:59.650 INFO:teuthology.orchestra.run.smithi160.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-07T07:29:59.650 INFO:teuthology.orchestra.run.smithi160.stdout: acl-2.3.1-4.el9.x86_64 2024-03-07T07:29:59.650 INFO:teuthology.orchestra.run.smithi160.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-07T07:29:59.650 INFO:teuthology.orchestra.run.smithi160.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-07T07:29:59.650 INFO:teuthology.orchestra.run.smithi160.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-07T07:29:59.650 INFO:teuthology.orchestra.run.smithi160.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-07T07:29:59.650 INFO:teuthology.orchestra.run.smithi160.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-07T07:29:59.650 INFO:teuthology.orchestra.run.smithi160.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-07T07:29:59.650 INFO:teuthology.orchestra.run.smithi160.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-07T07:29:59.651 INFO:teuthology.orchestra.run.smithi160.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-07T07:29:59.651 INFO:teuthology.orchestra.run.smithi160.stdout: librados2-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:59.651 INFO:teuthology.orchestra.run.smithi160.stdout: librbd1-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:59.651 INFO:teuthology.orchestra.run.smithi160.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-07T07:29:59.651 INFO:teuthology.orchestra.run.smithi160.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-07T07:29:59.651 INFO:teuthology.orchestra.run.smithi160.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-07T07:29:59.651 INFO:teuthology.orchestra.run.smithi160.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-07T07:29:59.651 INFO:teuthology.orchestra.run.smithi160.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-07T07:29:59.652 INFO:teuthology.orchestra.run.smithi160.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-07T07:29:59.652 INFO:teuthology.orchestra.run.smithi160.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-07T07:29:59.652 INFO:teuthology.orchestra.run.smithi160.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-07T07:29:59.652 INFO:teuthology.orchestra.run.smithi160.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-07T07:29:59.652 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2024-03-07T07:29:59.652 INFO:teuthology.orchestra.run.smithi160.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-07T07:29:59.652 INFO:teuthology.orchestra.run.smithi160.stdout: automake-1.16.2-8.el9.noarch 2024-03-07T07:29:59.652 INFO:teuthology.orchestra.run.smithi160.stdout: bison-3.7.4-5.el9.x86_64 2024-03-07T07:29:59.652 INFO:teuthology.orchestra.run.smithi160.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-07T07:29:59.652 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:59.652 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-base-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:59.652 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:59.652 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:59.653 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-grafana-dashboards-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:29:59.653 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-immutable-object-cache-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:59.653 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mds-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:59.653 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:59.653 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-cephadm-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:29:59.653 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-dashboard-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:29:59.653 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:29:59.653 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-modules-core-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:29:59.654 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-rook-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:29:59.654 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mon-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:59.654 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-osd-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:59.654 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-prometheus-alerts-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:29:59.654 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-radosgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:59.654 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-selinux-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:59.654 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-test-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:59.654 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-volume-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:29:59.654 INFO:teuthology.orchestra.run.smithi160.stdout: cephadm-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:29:59.654 INFO:teuthology.orchestra.run.smithi160.stdout: cephfs-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:59.654 INFO:teuthology.orchestra.run.smithi160.stdout: cephfs-shell-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:59.654 INFO:teuthology.orchestra.run.smithi160.stdout: cephfs-top-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:29:59.654 INFO:teuthology.orchestra.run.smithi160.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-07T07:29:59.655 INFO:teuthology.orchestra.run.smithi160.stdout: flex-2.6.4-9.el9.x86_64 2024-03-07T07:29:59.655 INFO:teuthology.orchestra.run.smithi160.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-07T07:29:59.655 INFO:teuthology.orchestra.run.smithi160.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-07T07:29:59.655 INFO:teuthology.orchestra.run.smithi160.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-07T07:29:59.655 INFO:teuthology.orchestra.run.smithi160.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-07T07:29:59.655 INFO:teuthology.orchestra.run.smithi160.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-07T07:29:59.655 INFO:teuthology.orchestra.run.smithi160.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-07T07:29:59.655 INFO:teuthology.orchestra.run.smithi160.stdout: jq-1.6-16.el9.x86_64 2024-03-07T07:29:59.655 INFO:teuthology.orchestra.run.smithi160.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-03-07T07:29:59.656 INFO:teuthology.orchestra.run.smithi160.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-07T07:29:59.656 INFO:teuthology.orchestra.run.smithi160.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-07T07:29:59.656 INFO:teuthology.orchestra.run.smithi160.stdout: libcephfs-devel-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:59.656 INFO:teuthology.orchestra.run.smithi160.stdout: libcephfs2-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:59.656 INFO:teuthology.orchestra.run.smithi160.stdout: libcephsqlite-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:59.656 INFO:teuthology.orchestra.run.smithi160.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-07T07:29:59.656 INFO:teuthology.orchestra.run.smithi160.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-07T07:29:59.656 INFO:teuthology.orchestra.run.smithi160.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-07T07:29:59.657 INFO:teuthology.orchestra.run.smithi160.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-07T07:29:59.657 INFO:teuthology.orchestra.run.smithi160.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-07T07:29:59.657 INFO:teuthology.orchestra.run.smithi160.stdout: librados-devel-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:59.657 INFO:teuthology.orchestra.run.smithi160.stdout: libradosstriper1-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:59.657 INFO:teuthology.orchestra.run.smithi160.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-07T07:29:59.657 INFO:teuthology.orchestra.run.smithi160.stdout: librgw2-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:59.657 INFO:teuthology.orchestra.run.smithi160.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-07T07:29:59.657 INFO:teuthology.orchestra.run.smithi160.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-07T07:29:59.657 INFO:teuthology.orchestra.run.smithi160.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-07T07:29:59.657 INFO:teuthology.orchestra.run.smithi160.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-07T07:29:59.657 INFO:teuthology.orchestra.run.smithi160.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-07T07:29:59.658 INFO:teuthology.orchestra.run.smithi160.stdout: lua-5.4.4-4.el9.x86_64 2024-03-07T07:29:59.658 INFO:teuthology.orchestra.run.smithi160.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-07T07:29:59.658 INFO:teuthology.orchestra.run.smithi160.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-07T07:29:59.658 INFO:teuthology.orchestra.run.smithi160.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-07T07:29:59.658 INFO:teuthology.orchestra.run.smithi160.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-07T07:29:59.658 INFO:teuthology.orchestra.run.smithi160.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-07T07:29:59.658 INFO:teuthology.orchestra.run.smithi160.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-07T07:29:59.658 INFO:teuthology.orchestra.run.smithi160.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-07T07:29:59.658 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-13.11-1.el9.x86_64 2024-03-07T07:29:59.658 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-07T07:29:59.659 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-07T07:29:59.659 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-07T07:29:59.659 INFO:teuthology.orchestra.run.smithi160.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-07T07:29:59.659 INFO:teuthology.orchestra.run.smithi160.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-07T07:29:59.659 INFO:teuthology.orchestra.run.smithi160.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-07T07:29:59.659 INFO:teuthology.orchestra.run.smithi160.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-07T07:29:59.659 INFO:teuthology.orchestra.run.smithi160.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-07T07:29:59.659 INFO:teuthology.orchestra.run.smithi160.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-07T07:29:59.659 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-07T07:29:59.659 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ceph-argparse-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:59.659 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:59.659 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cephfs-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:59.660 INFO:teuthology.orchestra.run.smithi160.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-07T07:29:59.660 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-07T07:29:59.660 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-07T07:29:59.660 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-07T07:29:59.660 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-07T07:29:59.660 INFO:teuthology.orchestra.run.smithi160.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-07T07:29:59.660 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-07T07:29:59.660 INFO:teuthology.orchestra.run.smithi160.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-07T07:29:59.660 INFO:teuthology.orchestra.run.smithi160.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-07T07:29:59.661 INFO:teuthology.orchestra.run.smithi160.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-07T07:29:59.661 INFO:teuthology.orchestra.run.smithi160.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-07T07:29:59.661 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-07T07:29:59.661 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-07T07:29:59.661 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-07T07:29:59.661 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-07T07:29:59.661 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-07T07:29:59.661 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-07T07:29:59.661 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-07T07:29:59.661 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-07T07:29:59.661 INFO:teuthology.orchestra.run.smithi160.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-07T07:29:59.661 INFO:teuthology.orchestra.run.smithi160.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-07T07:29:59.661 INFO:teuthology.orchestra.run.smithi160.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-07T07:29:59.662 INFO:teuthology.orchestra.run.smithi160.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-07T07:29:59.662 INFO:teuthology.orchestra.run.smithi160.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-07T07:29:59.662 INFO:teuthology.orchestra.run.smithi160.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-07T07:29:59.662 INFO:teuthology.orchestra.run.smithi160.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-07T07:29:59.662 INFO:teuthology.orchestra.run.smithi160.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-07T07:29:59.662 INFO:teuthology.orchestra.run.smithi160.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-07T07:29:59.662 INFO:teuthology.orchestra.run.smithi160.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-07T07:29:59.662 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-07T07:29:59.662 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ply-3.11-14.el9.noarch 2024-03-07T07:29:59.662 INFO:teuthology.orchestra.run.smithi160.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-07T07:29:59.662 INFO:teuthology.orchestra.run.smithi160.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-07T07:29:59.663 INFO:teuthology.orchestra.run.smithi160.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-07T07:29:59.663 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-07T07:29:59.663 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-07T07:29:59.663 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-07T07:29:59.663 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-07T07:29:59.663 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-07T07:29:59.663 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-07T07:29:59.663 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rados-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:59.663 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:59.663 INFO:teuthology.orchestra.run.smithi160.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-07T07:29:59.663 INFO:teuthology.orchestra.run.smithi160.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-07T07:29:59.663 INFO:teuthology.orchestra.run.smithi160.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-07T07:29:59.663 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:59.664 INFO:teuthology.orchestra.run.smithi160.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-07T07:29:59.664 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-07T07:29:59.664 INFO:teuthology.orchestra.run.smithi160.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-07T07:29:59.664 INFO:teuthology.orchestra.run.smithi160.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-07T07:29:59.664 INFO:teuthology.orchestra.run.smithi160.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-07T07:29:59.664 INFO:teuthology.orchestra.run.smithi160.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-07T07:29:59.664 INFO:teuthology.orchestra.run.smithi160.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-07T07:29:59.664 INFO:teuthology.orchestra.run.smithi160.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-07T07:29:59.664 INFO:teuthology.orchestra.run.smithi160.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-07T07:29:59.664 INFO:teuthology.orchestra.run.smithi160.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-07T07:29:59.664 INFO:teuthology.orchestra.run.smithi160.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-07T07:29:59.665 INFO:teuthology.orchestra.run.smithi160.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-07T07:29:59.665 INFO:teuthology.orchestra.run.smithi160.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-07T07:29:59.665 INFO:teuthology.orchestra.run.smithi160.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-07T07:29:59.665 INFO:teuthology.orchestra.run.smithi160.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-07T07:29:59.665 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:59.665 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:59.665 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-nbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:29:59.665 INFO:teuthology.orchestra.run.smithi160.stdout: re2-1:20211101-3.el9.x86_64 2024-03-07T07:29:59.665 INFO:teuthology.orchestra.run.smithi160.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-07T07:29:59.665 INFO:teuthology.orchestra.run.smithi160.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-07T07:29:59.665 INFO:teuthology.orchestra.run.smithi160.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-07T07:29:59.665 INFO:teuthology.orchestra.run.smithi160.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-07T07:29:59.666 INFO:teuthology.orchestra.run.smithi160.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-07T07:29:59.666 INFO:teuthology.orchestra.run.smithi160.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-07T07:29:59.666 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:29:59.666 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-03-07T07:30:00.154 DEBUG:teuthology.parallel:result is None 2024-03-07T07:30:00.155 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8b896a9e145796119f0451201dc4d53ddac97db0 2024-03-07T07:30:00.306 DEBUG:teuthology.orchestra.run.smithi160:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-07T07:30:00.328 INFO:teuthology.orchestra.run.smithi160.stdout:19.0.0-1848.g8b896a9e.el9 2024-03-07T07:30:00.329 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1848.g8b896a9e.el9 2024-03-07T07:30:00.329 INFO:teuthology.task.install:The correct ceph version 19.0.0-1848.g8b896a9e is installed. 2024-03-07T07:30:00.331 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8b896a9e145796119f0451201dc4d53ddac97db0 2024-03-07T07:30:00.554 DEBUG:teuthology.orchestra.run.smithi177:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-07T07:30:00.579 INFO:teuthology.orchestra.run.smithi177.stdout:19.0.0-1848.g8b896a9e.el9 2024-03-07T07:30:00.580 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1848.g8b896a9e.el9 2024-03-07T07:30:00.580 INFO:teuthology.task.install:The correct ceph version 19.0.0-1848.g8b896a9e is installed. 2024-03-07T07:30:00.582 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8b896a9e145796119f0451201dc4d53ddac97db0 2024-03-07T07:30:00.729 DEBUG:teuthology.orchestra.run.smithi178:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-07T07:30:00.755 INFO:teuthology.orchestra.run.smithi178.stdout:19.0.0-1848.g8b896a9e.el9 2024-03-07T07:30:00.755 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1848.g8b896a9e.el9 2024-03-07T07:30:00.755 INFO:teuthology.task.install:The correct ceph version 19.0.0-1848.g8b896a9e is installed. 2024-03-07T07:30:00.757 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-07T07:30:00.758 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-03-07T07:30:00.758 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-07T07:30:00.806 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-03-07T07:30:00.806 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-07T07:30:00.842 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-03-07T07:30:00.842 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-07T07:30:00.878 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-07T07:30:00.878 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-03-07T07:30:00.878 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/usr/bin/daemon-helper 2024-03-07T07:30:00.906 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-07T07:30:00.972 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-03-07T07:30:00.973 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/usr/bin/daemon-helper 2024-03-07T07:30:01.005 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-07T07:30:01.074 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-03-07T07:30:01.075 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/daemon-helper 2024-03-07T07:30:01.103 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-07T07:30:01.172 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-07T07:30:01.205 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-03-07T07:30:01.206 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-07T07:30:01.232 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-07T07:30:01.300 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-03-07T07:30:01.300 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-07T07:30:01.327 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-07T07:30:01.395 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-03-07T07:30:01.395 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-07T07:30:01.426 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-07T07:30:01.493 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-07T07:30:01.493 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-03-07T07:30:01.493 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/usr/bin/stdin-killer 2024-03-07T07:30:01.520 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-07T07:30:01.588 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-03-07T07:30:01.588 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/usr/bin/stdin-killer 2024-03-07T07:30:01.615 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-07T07:30:01.682 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-03-07T07:30:01.682 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/stdin-killer 2024-03-07T07:30:01.708 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-07T07:30:01.775 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-07T07:30:01.876 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '8b896a9e145796119f0451201dc4d53ddac97db0', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-03-07T07:30:01.876 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 2024-03-07T07:30:01.876 INFO:tasks.cephadm:Cluster fsid is 826f6582-dc54-11ee-95c6-87774f69a715 2024-03-07T07:30:01.876 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-07T07:30:01.877 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.160', 'mon.b': '172.21.15.177', 'mon.c': '172.21.15.178'} 2024-03-07T07:30:01.877 INFO:tasks.cephadm:First mon is mon.a on smithi160 2024-03-07T07:30:01.877 INFO:tasks.cephadm:First mgr is x 2024-03-07T07:30:01.877 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-07T07:30:01.877 DEBUG:teuthology.orchestra.run.smithi160:> sudo hostname $(hostname -s) 2024-03-07T07:30:01.905 DEBUG:teuthology.orchestra.run.smithi177:> sudo hostname $(hostname -s) 2024-03-07T07:30:01.932 DEBUG:teuthology.orchestra.run.smithi178:> sudo hostname $(hostname -s) 2024-03-07T07:30:01.959 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-03-07T07:30:01.959 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8b896a9e145796119f0451201dc4d53ddac97db0 2024-03-07T07:30:02.115 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '8b896a9e145796119f0451201dc4d53ddac97db0', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77380/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-1848-g8b896a9e', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1848.g8b896a9e'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-lusov-qdb/8b896a9e145796119f0451201dc4d53ddac97db0/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-06 23:30:28.927100', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-lusov-qdb', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-lusov-qdb/8b896a9e145796119f0451201dc4d53ddac97db0/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-03-07T07:30:02.261 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-lusov-qdb, sha1 8b896a9e145796119f0451201dc4d53ddac97db0 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=8b896a9e145796119f0451201dc4d53ddac97db0 2024-03-07T07:30:02.264 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-lusov-qdb/8b896a9e145796119f0451201dc4d53ddac97db0/centos/9/x86_64/flavors/default/cephadm 2024-03-07T07:30:02.264 DEBUG:teuthology.orchestra.run.smithi160:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-lusov-qdb/8b896a9e145796119f0451201dc4d53ddac97db0/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-07T07:30:03.072 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r--. 1 ubuntu ubuntu 767460 Mar 7 07:30 /home/ubuntu/cephtest/cephadm 2024-03-07T07:30:03.072 DEBUG:teuthology.orchestra.run.smithi177:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-lusov-qdb/8b896a9e145796119f0451201dc4d53ddac97db0/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-07T07:30:03.872 INFO:teuthology.orchestra.run.smithi177.stdout:-rw-r--r--. 1 ubuntu ubuntu 767460 Mar 7 07:30 /home/ubuntu/cephtest/cephadm 2024-03-07T07:30:03.872 DEBUG:teuthology.orchestra.run.smithi178:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-lusov-qdb/8b896a9e145796119f0451201dc4d53ddac97db0/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-07T07:30:04.537 INFO:teuthology.orchestra.run.smithi178.stdout:-rw-r--r--. 1 ubuntu ubuntu 767460 Mar 7 07:30 /home/ubuntu/cephtest/cephadm 2024-03-07T07:30:04.537 DEBUG:teuthology.orchestra.run.smithi160:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-07T07:30:04.556 DEBUG:teuthology.orchestra.run.smithi177:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-07T07:30:04.575 DEBUG:teuthology.orchestra.run.smithi178:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-07T07:30:04.612 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 on all hosts... 2024-03-07T07:30:04.613 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 pull 2024-03-07T07:30:04.615 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 pull 2024-03-07T07:30:04.618 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 pull 2024-03-07T07:30:05.464 INFO:teuthology.orchestra.run.smithi178.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0... 2024-03-07T07:30:05.517 INFO:teuthology.orchestra.run.smithi160.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0... 2024-03-07T07:30:05.548 INFO:teuthology.orchestra.run.smithi177.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0... 2024-03-07T07:30:33.149 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-03-07T07:30:33.149 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_version": "ceph version 19.0.0-1848-g8b896a9e (8b896a9e145796119f0451201dc4d53ddac97db0) squid (dev)", 2024-03-07T07:30:33.150 INFO:teuthology.orchestra.run.smithi178.stdout: "image_id": "e362cd7cebfee51077fb90b7f01895811140c9a4272edd83ad9910b05e3a46f6", 2024-03-07T07:30:33.150 INFO:teuthology.orchestra.run.smithi178.stdout: "repo_digests": [ 2024-03-07T07:30:33.150 INFO:teuthology.orchestra.run.smithi178.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2" 2024-03-07T07:30:33.150 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-03-07T07:30:33.150 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-03-07T07:30:58.730 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-03-07T07:30:58.731 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_version": "ceph version 19.0.0-1848-g8b896a9e (8b896a9e145796119f0451201dc4d53ddac97db0) squid (dev)", 2024-03-07T07:30:58.731 INFO:teuthology.orchestra.run.smithi177.stdout: "image_id": "e362cd7cebfee51077fb90b7f01895811140c9a4272edd83ad9910b05e3a46f6", 2024-03-07T07:30:58.731 INFO:teuthology.orchestra.run.smithi177.stdout: "repo_digests": [ 2024-03-07T07:30:58.731 INFO:teuthology.orchestra.run.smithi177.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2" 2024-03-07T07:30:58.731 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-03-07T07:30:58.731 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-03-07T07:31:01.897 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-03-07T07:31:01.897 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_version": "ceph version 19.0.0-1848-g8b896a9e (8b896a9e145796119f0451201dc4d53ddac97db0) squid (dev)", 2024-03-07T07:31:01.897 INFO:teuthology.orchestra.run.smithi160.stdout: "image_id": "e362cd7cebfee51077fb90b7f01895811140c9a4272edd83ad9910b05e3a46f6", 2024-03-07T07:31:01.897 INFO:teuthology.orchestra.run.smithi160.stdout: "repo_digests": [ 2024-03-07T07:31:01.897 INFO:teuthology.orchestra.run.smithi160.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2" 2024-03-07T07:31:01.897 INFO:teuthology.orchestra.run.smithi160.stdout: ] 2024-03-07T07:31:01.897 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-03-07T07:31:01.927 DEBUG:teuthology.orchestra.run.smithi160:> sudo mkdir -p /etc/ceph 2024-03-07T07:31:01.983 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /etc/ceph 2024-03-07T07:31:02.018 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /etc/ceph 2024-03-07T07:31:02.054 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod 777 /etc/ceph 2024-03-07T07:31:02.083 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 777 /etc/ceph 2024-03-07T07:31:02.113 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 777 /etc/ceph 2024-03-07T07:31:02.144 INFO:tasks.cephadm:Writing seed config... 2024-03-07T07:31:02.145 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-07T07:31:02.145 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-07T07:31:02.146 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-07T07:31:02.146 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-07T07:31:02.146 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-07T07:31:02.146 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-03-07T07:31:02.146 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-07T07:31:02.146 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-07T07:31:02.146 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-07T07:31:02.146 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-03-07T07:31:02.146 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-03-07T07:31:02.146 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-03-07T07:31:02.146 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-03-07T07:31:02.146 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-07T07:31:02.146 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-07T07:31:02.146 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-07T07:31:02.147 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-07T07:31:02.147 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-03-07T07:31:02.147 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-03-07T07:31:02.147 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-07T07:31:02.147 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-07T07:31:02.147 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-07T07:31:02.147 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-03-07T07:31:02.147 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-07T07:31:02.147 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-07T07:31:02.147 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-07T07:31:02.147 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-07T07:31:02.147 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-07T07:31:02.147 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-07T07:31:02.148 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-07T07:31:02.148 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-03-07T07:31:02.148 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-07T07:31:02.148 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-03-07T07:31:02.148 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-03-07T07:31:02.148 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-07T07:31:02.148 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-07T07:31:02.148 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-03-07T07:31:02.148 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-07T07:31:02.148 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-07T07:31:02.148 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-07T07:31:02.148 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-07T07:31:02.148 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-07T07:31:02.148 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-07T07:31:02.149 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-07T07:31:02.149 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-03-07T07:31:02.149 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-03-07T07:31:02.149 DEBUG:teuthology.orchestra.run.smithi160:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-07T07:31:02.166 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 826f6582-dc54-11ee-95c6-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-07T07:31:02.166 DEBUG:teuthology.orchestra.run.smithi160:mon.a> sudo journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@mon.a.service 2024-03-07T07:31:02.209 DEBUG:teuthology.orchestra.run.smithi160:mgr.x> sudo journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@mgr.x.service 2024-03-07T07:31:02.252 INFO:tasks.cephadm:Bootstrapping... 2024-03-07T07:31:02.252 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 -v bootstrap --fsid 826f6582-dc54-11ee-95c6-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.160 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-07T07:31:02.444 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2024-03-07T07:31:02.444 INFO:teuthology.orchestra.run.smithi160.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0', '-v', 'bootstrap', '--fsid', '826f6582-dc54-11ee-95c6-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.160', '--skip-admin-label'] 2024-03-07T07:31:02.469 INFO:teuthology.orchestra.run.smithi160.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-07T07:31:02.470 INFO:teuthology.orchestra.run.smithi160.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-07T07:31:02.470 INFO:teuthology.orchestra.run.smithi160.stdout:Verifying podman|docker is present... 2024-03-07T07:31:02.494 INFO:teuthology.orchestra.run.smithi160.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-07T07:31:02.494 INFO:teuthology.orchestra.run.smithi160.stdout:Verifying lvm2 is present... 2024-03-07T07:31:02.494 INFO:teuthology.orchestra.run.smithi160.stdout:Verifying time synchronization is in place... 2024-03-07T07:31:02.501 INFO:teuthology.orchestra.run.smithi160.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-07T07:31:02.501 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-07T07:31:02.507 INFO:teuthology.orchestra.run.smithi160.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-07T07:31:02.507 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stdout inactive 2024-03-07T07:31:02.514 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stdout enabled 2024-03-07T07:31:02.520 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stdout active 2024-03-07T07:31:02.520 INFO:teuthology.orchestra.run.smithi160.stdout:Unit chronyd.service is enabled and running 2024-03-07T07:31:02.520 INFO:teuthology.orchestra.run.smithi160.stdout:Repeating the final host check... 2024-03-07T07:31:02.543 INFO:teuthology.orchestra.run.smithi160.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-07T07:31:02.543 INFO:teuthology.orchestra.run.smithi160.stdout:podman (/bin/podman) version 4.9.3 is present 2024-03-07T07:31:02.543 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl is present 2024-03-07T07:31:02.544 INFO:teuthology.orchestra.run.smithi160.stdout:lvcreate is present 2024-03-07T07:31:02.549 INFO:teuthology.orchestra.run.smithi160.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-07T07:31:02.549 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-07T07:31:02.555 INFO:teuthology.orchestra.run.smithi160.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-07T07:31:02.555 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stdout inactive 2024-03-07T07:31:02.561 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stdout enabled 2024-03-07T07:31:02.568 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stdout active 2024-03-07T07:31:02.568 INFO:teuthology.orchestra.run.smithi160.stdout:Unit chronyd.service is enabled and running 2024-03-07T07:31:02.568 INFO:teuthology.orchestra.run.smithi160.stdout:Host looks OK 2024-03-07T07:31:02.568 INFO:teuthology.orchestra.run.smithi160.stdout:Cluster fsid: 826f6582-dc54-11ee-95c6-87774f69a715 2024-03-07T07:31:02.568 INFO:teuthology.orchestra.run.smithi160.stdout:Acquiring lock 140009494994368 on /run/cephadm/826f6582-dc54-11ee-95c6-87774f69a715.lock 2024-03-07T07:31:02.568 INFO:teuthology.orchestra.run.smithi160.stdout:Lock 140009494994368 acquired on /run/cephadm/826f6582-dc54-11ee-95c6-87774f69a715.lock 2024-03-07T07:31:02.568 INFO:teuthology.orchestra.run.smithi160.stdout:Verifying IP 172.21.15.160 port 3300 ... 2024-03-07T07:31:02.569 INFO:teuthology.orchestra.run.smithi160.stdout:Verifying IP 172.21.15.160 port 6789 ... 2024-03-07T07:31:02.569 INFO:teuthology.orchestra.run.smithi160.stdout:Base mon IP(s) is [172.21.15.160:3300, 172.21.15.160:6789], mon addrv is [v2:172.21.15.160:3300,v1:172.21.15.160:6789] 2024-03-07T07:31:02.572 INFO:teuthology.orchestra.run.smithi160.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.160 metric 100 2024-03-07T07:31:02.572 INFO:teuthology.orchestra.run.smithi160.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.160 metric 100 2024-03-07T07:31:02.574 INFO:teuthology.orchestra.run.smithi160.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-07T07:31:02.574 INFO:teuthology.orchestra.run.smithi160.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-03-07T07:31:02.574 INFO:teuthology.orchestra.run.smithi160.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1796sec hoplimit 64 pref medium 2024-03-07T07:31:02.577 INFO:teuthology.orchestra.run.smithi160.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-07T07:31:02.577 INFO:teuthology.orchestra.run.smithi160.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-07T07:31:02.577 INFO:teuthology.orchestra.run.smithi160.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-07T07:31:02.577 INFO:teuthology.orchestra.run.smithi160.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-03-07T07:31:02.577 INFO:teuthology.orchestra.run.smithi160.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:7f5b/64 scope link 2024-03-07T07:31:02.578 INFO:teuthology.orchestra.run.smithi160.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-07T07:31:02.578 INFO:teuthology.orchestra.run.smithi160.stdout:Mon IP `172.21.15.160` is in CIDR network `172.21.0.0/20` 2024-03-07T07:31:02.578 INFO:teuthology.orchestra.run.smithi160.stdout:Mon IP `172.21.15.160` is in CIDR network `172.21.0.0/20` 2024-03-07T07:31:02.578 INFO:teuthology.orchestra.run.smithi160.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-07T07:31:02.578 INFO:teuthology.orchestra.run.smithi160.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-07T07:31:02.579 INFO:teuthology.orchestra.run.smithi160.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0... 2024-03-07T07:31:03.781 INFO:teuthology.orchestra.run.smithi160.stdout:/bin/podman: stdout e362cd7cebfee51077fb90b7f01895811140c9a4272edd83ad9910b05e3a46f6 2024-03-07T07:31:03.781 INFO:teuthology.orchestra.run.smithi160.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0... 2024-03-07T07:31:03.781 INFO:teuthology.orchestra.run.smithi160.stdout:/bin/podman: stderr Getting image source signatures 2024-03-07T07:31:03.782 INFO:teuthology.orchestra.run.smithi160.stdout:/bin/podman: stderr Copying blob sha256:2daaaa8f94553300cdc74c38bac2741a5998248e1125c1b65b21a272e0fd7dd5 2024-03-07T07:31:03.782 INFO:teuthology.orchestra.run.smithi160.stdout:/bin/podman: stderr Copying blob sha256:7feca07754707458c3945cf0062cf4dabc512f6d90fe1a9a1370b362b6011124 2024-03-07T07:31:03.782 INFO:teuthology.orchestra.run.smithi160.stdout:/bin/podman: stderr Copying config sha256:e362cd7cebfee51077fb90b7f01895811140c9a4272edd83ad9910b05e3a46f6 2024-03-07T07:31:03.782 INFO:teuthology.orchestra.run.smithi160.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-07T07:31:04.691 INFO:teuthology.orchestra.run.smithi160.stdout:ceph: stdout ceph version 19.0.0-1848-g8b896a9e (8b896a9e145796119f0451201dc4d53ddac97db0) squid (dev) 2024-03-07T07:31:04.692 INFO:teuthology.orchestra.run.smithi160.stdout:Ceph version: ceph version 19.0.0-1848-g8b896a9e (8b896a9e145796119f0451201dc4d53ddac97db0) squid (dev) 2024-03-07T07:31:04.692 INFO:teuthology.orchestra.run.smithi160.stdout:Extracting ceph user uid/gid from container image... 2024-03-07T07:31:05.524 INFO:teuthology.orchestra.run.smithi160.stdout:stat: stdout 167 167 2024-03-07T07:31:05.524 INFO:teuthology.orchestra.run.smithi160.stdout:Creating initial keys... 2024-03-07T07:31:06.421 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph-authtool: stdout AQA5bellk6enMhAAY+vrMxIGGWbDkkwT8dGwSA== 2024-03-07T07:31:07.644 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph-authtool: stdout AQA6bell0dgsLhAAlWrsYQ1YtagclFYvmjVnOg== 2024-03-07T07:31:08.608 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph-authtool: stdout AQA7bellgsVgOBAA139KUWn93RLcqJFIgpDhug== 2024-03-07T07:31:08.609 INFO:teuthology.orchestra.run.smithi160.stdout:Creating initial monmap... 2024-03-07T07:31:11.859 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-07T07:31:11.860 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-03-07T07:31:11.860 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 826f6582-dc54-11ee-95c6-87774f69a715 2024-03-07T07:31:11.860 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-07T07:31:11.860 INFO:teuthology.orchestra.run.smithi160.stdout:monmaptool for a [v2:172.21.15.160:3300,v1:172.21.15.160:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-07T07:31:11.860 INFO:teuthology.orchestra.run.smithi160.stdout:setting min_mon_release = quincy 2024-03-07T07:31:11.860 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/monmaptool: set fsid to 826f6582-dc54-11ee-95c6-87774f69a715 2024-03-07T07:31:11.860 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-07T07:31:11.860 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:31:11.860 INFO:teuthology.orchestra.run.smithi160.stdout:Creating mon... 2024-03-07T07:31:12.823 INFO:teuthology.orchestra.run.smithi160.stdout:create mon.a on 2024-03-07T07:31:13.031 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-07T07:31:13.174 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-07T07:31:13.344 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-826f6582-dc54-11ee-95c6-87774f69a715.target → /etc/systemd/system/ceph-826f6582-dc54-11ee-95c6-87774f69a715.target. 2024-03-07T07:31:13.344 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-826f6582-dc54-11ee-95c6-87774f69a715.target → /etc/systemd/system/ceph-826f6582-dc54-11ee-95c6-87774f69a715.target. 2024-03-07T07:31:13.591 INFO:teuthology.orchestra.run.smithi160.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-826f6582-dc54-11ee-95c6-87774f69a715@mon.a 2024-03-07T07:31:13.591 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stderr Failed to reset failed state of unit ceph-826f6582-dc54-11ee-95c6-87774f69a715@mon.a.service: Unit ceph-826f6582-dc54-11ee-95c6-87774f69a715@mon.a.service not loaded. 2024-03-07T07:31:13.755 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-826f6582-dc54-11ee-95c6-87774f69a715.target.wants/ceph-826f6582-dc54-11ee-95c6-87774f69a715@mon.a.service → /etc/systemd/system/ceph-826f6582-dc54-11ee-95c6-87774f69a715@.service. 2024-03-07T07:31:13.776 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:13 smithi160 systemd[1]: Starting Ceph mon.a for 826f6582-dc54-11ee-95c6-87774f69a715... 2024-03-07T07:31:14.095 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:13 smithi160 podman[31570]: 2024-03-07 07:31:13.98992453 +0000 UTC m=+0.111428465 container create 2131c11f11b898ddcd57ff87d6514d916476ac0cf32216e22b1aa10fe9d49cab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mon-a, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-8b896a9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-07T07:31:14.095 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:13 smithi160 podman[31570]: 2024-03-07 07:31:13.896023864 +0000 UTC m=+0.017527807 image pull e362cd7cebfee51077fb90b7f01895811140c9a4272edd83ad9910b05e3a46f6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 2024-03-07T07:31:14.230 INFO:teuthology.orchestra.run.smithi160.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-07T07:31:14.230 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stdout disabled 2024-03-07T07:31:14.237 INFO:teuthology.orchestra.run.smithi160.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-07T07:31:14.237 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stdout inactive 2024-03-07T07:31:14.237 INFO:teuthology.orchestra.run.smithi160.stdout:firewalld.service is not enabled 2024-03-07T07:31:14.237 INFO:teuthology.orchestra.run.smithi160.stdout:Not possible to enable service . firewalld.service is not available 2024-03-07T07:31:14.238 INFO:teuthology.orchestra.run.smithi160.stdout:Waiting for mon to start... 2024-03-07T07:31:14.238 INFO:teuthology.orchestra.run.smithi160.stdout:Waiting for mon... 2024-03-07T07:31:14.363 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:14 smithi160 podman[31570]: 2024-03-07 07:31:14.158740291 +0000 UTC m=+0.280244225 container init 2131c11f11b898ddcd57ff87d6514d916476ac0cf32216e22b1aa10fe9d49cab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mon-a, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-07T07:31:14.363 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:14 smithi160 podman[31570]: 2024-03-07 07:31:14.161560857 +0000 UTC m=+0.283064787 container start 2131c11f11b898ddcd57ff87d6514d916476ac0cf32216e22b1aa10fe9d49cab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mon-a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-8b896a9, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-07T07:31:14.364 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:14 smithi160 bash[31570]: 2131c11f11b898ddcd57ff87d6514d916476ac0cf32216e22b1aa10fe9d49cab 2024-03-07T07:31:14.364 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:14 smithi160 systemd[1]: Started Ceph mon.a for 826f6582-dc54-11ee-95c6-87774f69a715. 2024-03-07T07:31:14.364 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:14 smithi160 ceph-mon[31589]: mkfs 826f6582-dc54-11ee-95c6-87774f69a715 2024-03-07T07:31:14.364 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:14 smithi160 ceph-mon[31589]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-07T07:31:15.291 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout cluster: 2024-03-07T07:31:15.291 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout id: 826f6582-dc54-11ee-95c6-87774f69a715 2024-03-07T07:31:15.291 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-07T07:31:15.291 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 2024-03-07T07:31:15.291 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout services: 2024-03-07T07:31:15.291 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.460702s) 2024-03-07T07:31:15.291 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-07T07:31:15.291 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-07T07:31:15.291 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 2024-03-07T07:31:15.291 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout data: 2024-03-07T07:31:15.291 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-07T07:31:15.292 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-07T07:31:15.292 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-07T07:31:15.292 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout pgs: 2024-03-07T07:31:15.292 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 2024-03-07T07:31:15.292 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:14.610+0000 7fdc58a67700 1 Processor -- start 2024-03-07T07:31:15.292 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:14.610+0000 7fdc58a67700 1 -- start start 2024-03-07T07:31:15.292 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:14.610+0000 7fdc58a67700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc54108520 0x7fdc5410a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:15.292 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:14.610+0000 7fdc58a67700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc540758c0 con 0x7fdc54108520 2024-03-07T07:31:15.292 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:14.611+0000 7fdc5259c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc54108520 0x7fdc5410a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:15.292 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:14.611+0000 7fdc5259c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc54108520 0x7fdc5410a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41454/0 (socket says 172.21.15.160:41454) 2024-03-07T07:31:15.292 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:14.611+0000 7fdc5259c700 1 -- 172.21.15.160:0/3528300476 learned_addr learned my addr 172.21.15.160:0/3528300476 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:31:15.292 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:14.611+0000 7fdc5259c700 1 -- 172.21.15.160:0/3528300476 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc5410ae50 con 0x7fdc54108520 2024-03-07T07:31:15.292 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:14.611+0000 7fdc5259c700 1 --2- 172.21.15.160:0/3528300476 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc54108520 0x7fdc5410a910 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdc440098a0 tx=0x7fdc44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78f2c530b116d779 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:15.293 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:14.611+0000 7fdc5159a700 1 -- 172.21.15.160:0/3528300476 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdc44004160 con 0x7fdc54108520 2024-03-07T07:31:15.293 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:14.611+0000 7fdc5159a700 1 -- 172.21.15.160:0/3528300476 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc44004300 con 0x7fdc54108520 2024-03-07T07:31:15.293 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:14.611+0000 7fdc5159a700 1 -- 172.21.15.160:0/3528300476 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdc440036a0 con 0x7fdc54108520 2024-03-07T07:31:15.293 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:14.612+0000 7fdc58a67700 1 -- 172.21.15.160:0/3528300476 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc54108520 msgr2=0x7fdc5410a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:15.293 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:14.612+0000 7fdc58a67700 1 --2- 172.21.15.160:0/3528300476 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc54108520 0x7fdc5410a910 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdc440098a0 tx=0x7fdc44009bb0 comp rx=0 tx=0).stop 2024-03-07T07:31:15.293 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:14.612+0000 7fdc58a67700 1 -- 172.21.15.160:0/3528300476 shutdown_connections 2024-03-07T07:31:15.293 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:14.612+0000 7fdc58a67700 1 --2- 172.21.15.160:0/3528300476 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc54108520 0x7fdc5410a910 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:15.293 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:14.612+0000 7fdc58a67700 1 -- 172.21.15.160:0/3528300476 >> 172.21.15.160:0/3528300476 conn(0x7fdc54101f70 msgr2=0x7fdc54104390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:31:15.293 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:14.612+0000 7fdc58a67700 1 -- 172.21.15.160:0/3528300476 shutdown_connections 2024-03-07T07:31:15.293 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:14.612+0000 7fdc58a67700 1 -- 172.21.15.160:0/3528300476 wait complete. 2024-03-07T07:31:15.293 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:14.613+0000 7fdc58a67700 1 Processor -- start 2024-03-07T07:31:15.293 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:14.613+0000 7fdc58a67700 1 -- start start 2024-03-07T07:31:15.293 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:14.613+0000 7fdc58a67700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc54108520 0x7fdc5419b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:15.294 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:14.613+0000 7fdc58a67700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5410b320 con 0x7fdc54108520 2024-03-07T07:31:15.294 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:14.613+0000 7fdc5259c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc54108520 0x7fdc5419b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:15.294 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:14.613+0000 7fdc5259c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc54108520 0x7fdc5419b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41468/0 (socket says 172.21.15.160:41468) 2024-03-07T07:31:15.294 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:14.613+0000 7fdc5259c700 1 -- 172.21.15.160:0/1363199581 learned_addr learned my addr 172.21.15.160:0/1363199581 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:31:15.294 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:14.613+0000 7fdc5259c700 1 -- 172.21.15.160:0/1363199581 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc5419b6d0 con 0x7fdc54108520 2024-03-07T07:31:15.294 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:14.614+0000 7fdc5259c700 1 --2- 172.21.15.160:0/1363199581 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc54108520 0x7fdc5419b190 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdc44000c00 tx=0x7fdc44004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:15.294 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:14.614+0000 7fdc437fe700 1 -- 172.21.15.160:0/1363199581 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdc44003db0 con 0x7fdc54108520 2024-03-07T07:31:15.294 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:14.614+0000 7fdc437fe700 1 -- 172.21.15.160:0/1363199581 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc44011630 con 0x7fdc54108520 2024-03-07T07:31:15.294 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:14.614+0000 7fdc58a67700 1 -- 172.21.15.160:0/1363199581 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc5419b960 con 0x7fdc54108520 2024-03-07T07:31:15.294 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:14.614+0000 7fdc437fe700 1 -- 172.21.15.160:0/1363199581 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdc44011950 con 0x7fdc54108520 2024-03-07T07:31:15.294 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:14.614+0000 7fdc58a67700 1 -- 172.21.15.160:0/1363199581 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc5419e650 con 0x7fdc54108520 2024-03-07T07:31:15.294 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:14.614+0000 7fdc437fe700 1 -- 172.21.15.160:0/1363199581 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fdc44011af0 con 0x7fdc54108520 2024-03-07T07:31:15.294 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:14.614+0000 7fdc437fe700 1 -- 172.21.15.160:0/1363199581 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fdc4401c450 con 0x7fdc54108520 2024-03-07T07:31:15.295 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:14.615+0000 7fdc58a67700 1 -- 172.21.15.160:0/1363199581 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc54063780 con 0x7fdc54108520 2024-03-07T07:31:15.295 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:14.618+0000 7fdc437fe700 1 -- 172.21.15.160:0/1363199581 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fdc44032020 con 0x7fdc54108520 2024-03-07T07:31:15.295 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:14.687+0000 7fdc58a67700 1 -- 172.21.15.160:0/1363199581 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fdc5402dc60 con 0x7fdc54108520 2024-03-07T07:31:15.295 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:14.688+0000 7fdc437fe700 1 -- 172.21.15.160:0/1363199581 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7fdc44004260 con 0x7fdc54108520 2024-03-07T07:31:15.295 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:14.688+0000 7fdc58a67700 1 -- 172.21.15.160:0/1363199581 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc54108520 msgr2=0x7fdc5419b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:15.295 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:14.688+0000 7fdc58a67700 1 --2- 172.21.15.160:0/1363199581 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc54108520 0x7fdc5419b190 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdc44000c00 tx=0x7fdc44004750 comp rx=0 tx=0).stop 2024-03-07T07:31:15.295 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:14.689+0000 7fdc58a67700 1 -- 172.21.15.160:0/1363199581 shutdown_connections 2024-03-07T07:31:15.295 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:14.689+0000 7fdc58a67700 1 --2- 172.21.15.160:0/1363199581 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc54108520 0x7fdc5419b190 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:15.295 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:14.689+0000 7fdc58a67700 1 -- 172.21.15.160:0/1363199581 >> 172.21.15.160:0/1363199581 conn(0x7fdc54101f70 msgr2=0x7fdc54104230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:31:15.295 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:14.689+0000 7fdc58a67700 1 -- 172.21.15.160:0/1363199581 shutdown_connections 2024-03-07T07:31:15.295 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:14.689+0000 7fdc58a67700 1 -- 172.21.15.160:0/1363199581 wait complete. 2024-03-07T07:31:15.295 INFO:teuthology.orchestra.run.smithi160.stdout:mon is available 2024-03-07T07:31:15.295 INFO:teuthology.orchestra.run.smithi160.stdout:Assimilating anything we can from ceph.conf... 2024-03-07T07:31:15.407 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:15 smithi160 ceph-mon[31589]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-07T07:31:15.407 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:15 smithi160 ceph-mon[31589]: monmap e1: 1 mons at {a=[v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-07T07:31:15.407 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:15 smithi160 ceph-mon[31589]: fsmap 2024-03-07T07:31:15.407 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:15 smithi160 ceph-mon[31589]: osdmap e1: 0 total, 0 up, 0 in 2024-03-07T07:31:15.407 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:15 smithi160 ceph-mon[31589]: mgrmap e1: no daemons active 2024-03-07T07:31:15.407 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:15 smithi160 ceph-mon[31589]: from='client.? 172.21.15.160:0/1363199581' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-07T07:31:16.287 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 2024-03-07T07:31:16.288 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout [global] 2024-03-07T07:31:16.288 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout fsid = 826f6582-dc54-11ee-95c6-87774f69a715 2024-03-07T07:31:16.288 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.160:3300,v1:172.21.15.160:6789] 2024-03-07T07:31:16.288 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-07T07:31:16.288 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-07T07:31:16.288 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-07T07:31:16.288 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-07T07:31:16.288 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 2024-03-07T07:31:16.288 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-07T07:31:16.288 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-07T07:31:16.288 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 2024-03-07T07:31:16.288 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout [osd] 2024-03-07T07:31:16.289 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-07T07:31:16.289 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-07T07:31:16.289 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-07T07:31:16.289 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-07T07:31:16.289 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-07T07:31:16.289 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:15.689+0000 7f057dc60700 1 Processor -- start 2024-03-07T07:31:16.289 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:15.689+0000 7f057dc60700 1 -- start start 2024-03-07T07:31:16.289 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:15.689+0000 7f057dc60700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0578107af0 0x7f0578107f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:16.289 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:15.689+0000 7f057dc60700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05781084e0 con 0x7f0578107af0 2024-03-07T07:31:16.289 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:15.689+0000 7f05777fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0578107af0 0x7f0578107f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:16.289 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:15.689+0000 7f05777fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0578107af0 0x7f0578107f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41476/0 (socket says 172.21.15.160:41476) 2024-03-07T07:31:16.290 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:15.689+0000 7f05777fe700 1 -- 172.21.15.160:0/2796998213 learned_addr learned my addr 172.21.15.160:0/2796998213 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:31:16.290 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:15.689+0000 7f05777fe700 1 -- 172.21.15.160:0/2796998213 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0578108cf0 con 0x7f0578107af0 2024-03-07T07:31:16.290 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:15.690+0000 7f05777fe700 1 --2- 172.21.15.160:0/2796998213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0578107af0 0x7f0578107f10 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f05680098a0 tx=0x7f0568009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c8ea2d79ec6882 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:16.290 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:15.690+0000 7f05767fc700 1 -- 172.21.15.160:0/2796998213 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0568004160 con 0x7f0578107af0 2024-03-07T07:31:16.290 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:15.690+0000 7f05767fc700 1 -- 172.21.15.160:0/2796998213 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0568004300 con 0x7f0578107af0 2024-03-07T07:31:16.290 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:15.690+0000 7f05767fc700 1 -- 172.21.15.160:0/2796998213 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f05680036a0 con 0x7f0578107af0 2024-03-07T07:31:16.290 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:15.690+0000 7f057dc60700 1 -- 172.21.15.160:0/2796998213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0578107af0 msgr2=0x7f0578107f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:16.290 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:15.690+0000 7f057dc60700 1 --2- 172.21.15.160:0/2796998213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0578107af0 0x7f0578107f10 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f05680098a0 tx=0x7f0568009bb0 comp rx=0 tx=0).stop 2024-03-07T07:31:16.290 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:15.690+0000 7f057dc60700 1 -- 172.21.15.160:0/2796998213 shutdown_connections 2024-03-07T07:31:16.290 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:15.690+0000 7f057dc60700 1 --2- 172.21.15.160:0/2796998213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0578107af0 0x7f0578107f10 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:16.290 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:15.690+0000 7f057dc60700 1 -- 172.21.15.160:0/2796998213 >> 172.21.15.160:0/2796998213 conn(0x7f0578103110 msgr2=0x7f0578105530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:31:16.290 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:15.690+0000 7f057dc60700 1 -- 172.21.15.160:0/2796998213 shutdown_connections 2024-03-07T07:31:16.290 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:15.690+0000 7f057dc60700 1 -- 172.21.15.160:0/2796998213 wait complete. 2024-03-07T07:31:16.291 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:15.691+0000 7f057dc60700 1 Processor -- start 2024-03-07T07:31:16.291 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:15.691+0000 7f057dc60700 1 -- start start 2024-03-07T07:31:16.291 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:15.691+0000 7f057dc60700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0578107af0 0x7f057807db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:16.291 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:15.691+0000 7f057dc60700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0578109220 con 0x7f0578107af0 2024-03-07T07:31:16.291 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:15.691+0000 7f05777fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0578107af0 0x7f057807db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:16.291 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:15.691+0000 7f05777fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0578107af0 0x7f057807db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41490/0 (socket says 172.21.15.160:41490) 2024-03-07T07:31:16.291 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:15.691+0000 7f05777fe700 1 -- 172.21.15.160:0/3390269681 learned_addr learned my addr 172.21.15.160:0/3390269681 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:31:16.291 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:15.692+0000 7f05777fe700 1 -- 172.21.15.160:0/3390269681 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f057807e040 con 0x7f0578107af0 2024-03-07T07:31:16.291 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:15.692+0000 7f05777fe700 1 --2- 172.21.15.160:0/3390269681 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0578107af0 0x7f057807db00 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0568000c00 tx=0x7f0568004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:16.291 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:15.692+0000 7f0574ff9700 1 -- 172.21.15.160:0/3390269681 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0568003db0 con 0x7f0578107af0 2024-03-07T07:31:16.291 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:15.692+0000 7f0574ff9700 1 -- 172.21.15.160:0/3390269681 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0568011630 con 0x7f0578107af0 2024-03-07T07:31:16.292 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:15.692+0000 7f057dc60700 1 -- 172.21.15.160:0/3390269681 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f057807c280 con 0x7f0578107af0 2024-03-07T07:31:16.292 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:15.692+0000 7f0574ff9700 1 -- 172.21.15.160:0/3390269681 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0568011950 con 0x7f0578107af0 2024-03-07T07:31:16.292 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:15.692+0000 7f057dc60700 1 -- 172.21.15.160:0/3390269681 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f057807c760 con 0x7f0578107af0 2024-03-07T07:31:16.292 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:15.692+0000 7f0574ff9700 1 -- 172.21.15.160:0/3390269681 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f056801a460 con 0x7f0578107af0 2024-03-07T07:31:16.292 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:15.692+0000 7f0574ff9700 1 -- 172.21.15.160:0/3390269681 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f056801b780 con 0x7f0578107af0 2024-03-07T07:31:16.292 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:15.693+0000 7f057dc60700 1 -- 172.21.15.160:0/3390269681 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0578063780 con 0x7f0578107af0 2024-03-07T07:31:16.292 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:15.695+0000 7f0574ff9700 1 -- 172.21.15.160:0/3390269681 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f0568031020 con 0x7f0578107af0 2024-03-07T07:31:16.292 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:15.763+0000 7f057dc60700 1 -- 172.21.15.160:0/3390269681 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f0578194aa0 con 0x7f0578107af0 2024-03-07T07:31:16.292 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:15.767+0000 7f0574ff9700 1 -- 172.21.15.160:0/3390269681 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+431 (secure 0 0 0) 0x7f056801b9d0 con 0x7f0578107af0 2024-03-07T07:31:16.292 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:15.767+0000 7f0574ff9700 1 -- 172.21.15.160:0/3390269681 <== mon.0 v2:172.21.15.160:3300/0 8 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f056801ada0 con 0x7f0578107af0 2024-03-07T07:31:16.293 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:15.768+0000 7f057dc60700 1 -- 172.21.15.160:0/3390269681 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0578107af0 msgr2=0x7f057807db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:16.293 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:15.768+0000 7f057dc60700 1 --2- 172.21.15.160:0/3390269681 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0578107af0 0x7f057807db00 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0568000c00 tx=0x7f0568004750 comp rx=0 tx=0).stop 2024-03-07T07:31:16.293 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:15.768+0000 7f057dc60700 1 -- 172.21.15.160:0/3390269681 shutdown_connections 2024-03-07T07:31:16.293 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:15.768+0000 7f057dc60700 1 --2- 172.21.15.160:0/3390269681 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0578107af0 0x7f057807db00 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:16.293 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:15.768+0000 7f057dc60700 1 -- 172.21.15.160:0/3390269681 >> 172.21.15.160:0/3390269681 conn(0x7f0578103110 msgr2=0x7f0578191ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:31:16.293 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:15.768+0000 7f057dc60700 1 -- 172.21.15.160:0/3390269681 shutdown_connections 2024-03-07T07:31:16.293 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:15.768+0000 7f057dc60700 1 -- 172.21.15.160:0/3390269681 wait complete. 2024-03-07T07:31:16.293 INFO:teuthology.orchestra.run.smithi160.stdout:Generating new minimal ceph.conf... 2024-03-07T07:31:16.405 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:16 smithi160 ceph-mon[31589]: from='client.? 172.21.15.160:0/3390269681' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-07T07:31:16.405 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:16 smithi160 ceph-mon[31589]: from='client.? 172.21.15.160:0/3390269681' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-07T07:31:17.551 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:17 smithi160 ceph-mon[31589]: from='client.? 172.21.15.160:0/3442558945' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:31:17.801 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:17.082+0000 7f0a97263700 1 Processor -- start 2024-03-07T07:31:17.801 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:17.082+0000 7f0a97263700 1 -- start start 2024-03-07T07:31:17.802 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:17.083+0000 7f0a97263700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a90107b10 0x7f0a90107f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:17.802 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:17.083+0000 7f0a97263700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a90108500 con 0x7f0a90107b10 2024-03-07T07:31:17.802 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:17.083+0000 7f0a94fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a90107b10 0x7f0a90107f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:17.802 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:17.083+0000 7f0a94fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a90107b10 0x7f0a90107f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41504/0 (socket says 172.21.15.160:41504) 2024-03-07T07:31:17.802 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:17.083+0000 7f0a94fff700 1 -- 172.21.15.160:0/3337362914 learned_addr learned my addr 172.21.15.160:0/3337362914 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:31:17.802 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:17.083+0000 7f0a94fff700 1 -- 172.21.15.160:0/3337362914 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a90108d10 con 0x7f0a90107b10 2024-03-07T07:31:17.802 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:17.083+0000 7f0a94fff700 1 --2- 172.21.15.160:0/3337362914 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a90107b10 0x7f0a90107f30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0a840098a0 tx=0x7f0a84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d91836bfa50782ad server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:17.802 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:17.083+0000 7f0a8f7fe700 1 -- 172.21.15.160:0/3337362914 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a84004160 con 0x7f0a90107b10 2024-03-07T07:31:17.802 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:17.083+0000 7f0a8f7fe700 1 -- 172.21.15.160:0/3337362914 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0a84004300 con 0x7f0a90107b10 2024-03-07T07:31:17.802 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:17.083+0000 7f0a8f7fe700 1 -- 172.21.15.160:0/3337362914 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a84011b50 con 0x7f0a90107b10 2024-03-07T07:31:17.802 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:17.084+0000 7f0a97263700 1 -- 172.21.15.160:0/3337362914 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a90107b10 msgr2=0x7f0a90107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:17.803 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:17.084+0000 7f0a97263700 1 --2- 172.21.15.160:0/3337362914 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a90107b10 0x7f0a90107f30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0a840098a0 tx=0x7f0a84009bb0 comp rx=0 tx=0).stop 2024-03-07T07:31:17.803 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:17.084+0000 7f0a97263700 1 -- 172.21.15.160:0/3337362914 shutdown_connections 2024-03-07T07:31:17.803 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:17.084+0000 7f0a97263700 1 --2- 172.21.15.160:0/3337362914 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a90107b10 0x7f0a90107f30 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:17.803 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:17.084+0000 7f0a97263700 1 -- 172.21.15.160:0/3337362914 >> 172.21.15.160:0/3337362914 conn(0x7f0a90103110 msgr2=0x7f0a90105530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:31:17.803 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:17.084+0000 7f0a97263700 1 -- 172.21.15.160:0/3337362914 shutdown_connections 2024-03-07T07:31:17.803 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:17.084+0000 7f0a97263700 1 -- 172.21.15.160:0/3337362914 wait complete. 2024-03-07T07:31:17.803 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:17.085+0000 7f0a97263700 1 Processor -- start 2024-03-07T07:31:17.803 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:17.085+0000 7f0a97263700 1 -- start start 2024-03-07T07:31:17.803 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:17.085+0000 7f0a97263700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a9019bd00 0x7f0a9019c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:17.803 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:17.085+0000 7f0a97263700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9010c0c0 con 0x7f0a9019bd00 2024-03-07T07:31:17.803 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:17.085+0000 7f0a94fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a9019bd00 0x7f0a9019c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:17.804 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:17.085+0000 7f0a94fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a9019bd00 0x7f0a9019c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41518/0 (socket says 172.21.15.160:41518) 2024-03-07T07:31:17.804 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:17.085+0000 7f0a94fff700 1 -- 172.21.15.160:0/3442558945 learned_addr learned my addr 172.21.15.160:0/3442558945 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:31:17.804 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:17.085+0000 7f0a94fff700 1 -- 172.21.15.160:0/3442558945 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a9019c660 con 0x7f0a9019bd00 2024-03-07T07:31:17.804 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:17.086+0000 7f0a94fff700 1 --2- 172.21.15.160:0/3442558945 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a9019bd00 0x7f0a9019c120 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0a84000c00 tx=0x7f0a840119e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:17.804 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:17.086+0000 7f0a8dffb700 1 -- 172.21.15.160:0/3442558945 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a840038b0 con 0x7f0a9019bd00 2024-03-07T07:31:17.804 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:17.086+0000 7f0a8dffb700 1 -- 172.21.15.160:0/3442558945 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0a84003a50 con 0x7f0a9019bd00 2024-03-07T07:31:17.804 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:17.086+0000 7f0a97263700 1 -- 172.21.15.160:0/3442558945 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a9019c8f0 con 0x7f0a9019bd00 2024-03-07T07:31:17.804 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:17.086+0000 7f0a8dffb700 1 -- 172.21.15.160:0/3442558945 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a8401b440 con 0x7f0a9019bd00 2024-03-07T07:31:17.804 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:17.086+0000 7f0a97263700 1 -- 172.21.15.160:0/3442558945 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a9019f3d0 con 0x7f0a9019bd00 2024-03-07T07:31:17.805 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:17.086+0000 7f0a8dffb700 1 -- 172.21.15.160:0/3442558945 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f0a8401ba50 con 0x7f0a9019bd00 2024-03-07T07:31:17.805 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:17.086+0000 7f0a8dffb700 1 -- 172.21.15.160:0/3442558945 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0a8402e730 con 0x7f0a9019bd00 2024-03-07T07:31:17.805 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:17.087+0000 7f0a97263700 1 -- 172.21.15.160:0/3442558945 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a90063780 con 0x7f0a9019bd00 2024-03-07T07:31:17.805 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:17.089+0000 7f0a8dffb700 1 -- 172.21.15.160:0/3442558945 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f0a8402e9d0 con 0x7f0a9019bd00 2024-03-07T07:31:17.805 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:17.157+0000 7f0a97263700 1 -- 172.21.15.160:0/3442558945 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f0a9019f740 con 0x7f0a9019bd00 2024-03-07T07:31:17.805 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:17.157+0000 7f0a8dffb700 1 -- 172.21.15.160:0/3442558945 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+177 (secure 0 0 0) 0x7f0a84033030 con 0x7f0a9019bd00 2024-03-07T07:31:17.805 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:17.158+0000 7f0a97263700 1 -- 172.21.15.160:0/3442558945 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a9019bd00 msgr2=0x7f0a9019c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:17.805 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:17.158+0000 7f0a97263700 1 --2- 172.21.15.160:0/3442558945 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a9019bd00 0x7f0a9019c120 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0a84000c00 tx=0x7f0a840119e0 comp rx=0 tx=0).stop 2024-03-07T07:31:17.805 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:17.158+0000 7f0a97263700 1 -- 172.21.15.160:0/3442558945 shutdown_connections 2024-03-07T07:31:17.805 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:17.158+0000 7f0a97263700 1 --2- 172.21.15.160:0/3442558945 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a9019bd00 0x7f0a9019c120 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:17.805 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:17.158+0000 7f0a97263700 1 -- 172.21.15.160:0/3442558945 >> 172.21.15.160:0/3442558945 conn(0x7f0a90103110 msgr2=0x7f0a90103d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:31:17.805 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:17.158+0000 7f0a97263700 1 -- 172.21.15.160:0/3442558945 shutdown_connections 2024-03-07T07:31:17.806 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:17.158+0000 7f0a97263700 1 -- 172.21.15.160:0/3442558945 wait complete. 2024-03-07T07:31:17.806 INFO:teuthology.orchestra.run.smithi160.stdout:Restarting the monitor... 2024-03-07T07:31:18.127 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:17 smithi160 systemd[1]: Stopping Ceph mon.a for 826f6582-dc54-11ee-95c6-87774f69a715... 2024-03-07T07:31:18.127 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:18 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mon-a[31585]: 2024-03-07T07:31:18.054+0000 7fa216f5c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-07T07:31:18.127 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:18 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mon-a[31585]: 2024-03-07T07:31:18.054+0000 7fa216f5c700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-07T07:31:18.405 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:18 smithi160 podman[31834]: 2024-03-07 07:31:18.127367998 +0000 UTC m=+0.159186654 container died 2131c11f11b898ddcd57ff87d6514d916476ac0cf32216e22b1aa10fe9d49cab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mon-a, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-8b896a9, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240222, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b) 2024-03-07T07:31:18.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:18 smithi160 podman[31834]: 2024-03-07 07:31:18.496662888 +0000 UTC m=+0.528481552 container cleanup 2131c11f11b898ddcd57ff87d6514d916476ac0cf32216e22b1aa10fe9d49cab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mon-a, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-qdb-8b896a9, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, CEPH_POINT_RELEASE=) 2024-03-07T07:31:18.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:18 smithi160 bash[31834]: ceph-826f6582-dc54-11ee-95c6-87774f69a715-mon-a 2024-03-07T07:31:19.106 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:18 smithi160 podman[31848]: 2024-03-07 07:31:18.706147281 +0000 UTC m=+0.577015231 container remove 2131c11f11b898ddcd57ff87d6514d916476ac0cf32216e22b1aa10fe9d49cab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mon-a, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-8b896a9, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b) 2024-03-07T07:31:19.106 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:18 smithi160 systemd[1]: ceph-826f6582-dc54-11ee-95c6-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-07T07:31:19.107 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:18 smithi160 systemd[1]: Stopped Ceph mon.a for 826f6582-dc54-11ee-95c6-87774f69a715. 2024-03-07T07:31:19.107 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:18 smithi160 systemd[1]: Starting Ceph mon.a for 826f6582-dc54-11ee-95c6-87774f69a715... 2024-03-07T07:31:19.367 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 podman[31945]: 2024-03-07 07:31:19.106923294 +0000 UTC m=+0.103222221 container create 23e49f05d5ab628af000486507c44ce0bbea95cc185028910c25c1f64531a00e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mon-a, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-8b896a9, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-07T07:31:19.367 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 podman[31945]: 2024-03-07 07:31:19.021164822 +0000 UTC m=+0.017463748 image pull e362cd7cebfee51077fb90b7f01895811140c9a4272edd83ad9910b05e3a46f6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 2024-03-07T07:31:19.367 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 podman[31945]: 2024-03-07 07:31:19.29249596 +0000 UTC m=+0.288794900 container init 23e49f05d5ab628af000486507c44ce0bbea95cc185028910c25c1f64531a00e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mon-a, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-8b896a9) 2024-03-07T07:31:19.367 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 podman[31945]: 2024-03-07 07:31:19.29679556 +0000 UTC m=+0.293094488 container start 23e49f05d5ab628af000486507c44ce0bbea95cc185028910c25c1f64531a00e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mon-a, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-8b896a9, GIT_CLEAN=True) 2024-03-07T07:31:19.368 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: set uid:gid to 167:167 (ceph:ceph) 2024-03-07T07:31:19.368 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: ceph version 19.0.0-1848-g8b896a9e (8b896a9e145796119f0451201dc4d53ddac97db0) squid (dev), process ceph-mon, pid 2 2024-03-07T07:31:19.368 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: pidfile_write: ignore empty --pid-file 2024-03-07T07:31:19.368 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: load: jerasure load: lrc 2024-03-07T07:31:19.368 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: RocksDB version: 7.9.2 2024-03-07T07:31:19.368 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Git sha 0 2024-03-07T07:31:19.368 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Compile date 2024-03-06 21:30:51 2024-03-07T07:31:19.368 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: DB SUMMARY 2024-03-07T07:31:19.368 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: DB Session ID: EJ79XFV7VF3OAOSRA2R8 2024-03-07T07:31:19.368 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: CURRENT file: CURRENT 2024-03-07T07:31:19.368 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: IDENTITY file: IDENTITY 2024-03-07T07:31:19.368 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-07T07:31:19.369 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-07T07:31:19.369 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 100470 ; 2024-03-07T07:31:19.369 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.error_if_exists: 0 2024-03-07T07:31:19.369 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.create_if_missing: 0 2024-03-07T07:31:19.369 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.paranoid_checks: 1 2024-03-07T07:31:19.369 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-07T07:31:19.369 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-07T07:31:19.369 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-07T07:31:19.369 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.env: 0x55ba4eed9b20 2024-03-07T07:31:19.369 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.fs: PosixFileSystem 2024-03-07T07:31:19.369 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.info_log: 0x55ba4fcdd920 2024-03-07T07:31:19.369 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.max_file_opening_threads: 16 2024-03-07T07:31:19.369 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.statistics: (nil) 2024-03-07T07:31:19.370 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.use_fsync: 0 2024-03-07T07:31:19.370 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.max_log_file_size: 0 2024-03-07T07:31:19.370 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-07T07:31:19.370 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-07T07:31:19.370 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.keep_log_file_num: 1000 2024-03-07T07:31:19.370 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.recycle_log_file_num: 0 2024-03-07T07:31:19.370 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.allow_fallocate: 1 2024-03-07T07:31:19.370 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.allow_mmap_reads: 0 2024-03-07T07:31:19.370 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.allow_mmap_writes: 0 2024-03-07T07:31:19.371 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.use_direct_reads: 0 2024-03-07T07:31:19.371 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-07T07:31:19.371 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.create_missing_column_families: 0 2024-03-07T07:31:19.371 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.db_log_dir: 2024-03-07T07:31:19.371 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.wal_dir: 2024-03-07T07:31:19.371 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-07T07:31:19.371 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-07T07:31:19.371 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-07T07:31:19.371 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-07T07:31:19.371 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-07T07:31:19.371 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-07T07:31:19.371 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.advise_random_on_open: 1 2024-03-07T07:31:19.371 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.db_write_buffer_size: 0 2024-03-07T07:31:19.371 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.write_buffer_manager: 0x55ba4fcb1400 2024-03-07T07:31:19.372 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-07T07:31:19.372 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-07T07:31:19.372 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-07T07:31:19.372 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.rate_limiter: (nil) 2024-03-07T07:31:19.372 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-07T07:31:19.373 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.wal_recovery_mode: 2 2024-03-07T07:31:19.373 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.enable_thread_tracking: 0 2024-03-07T07:31:19.373 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.enable_pipelined_write: 0 2024-03-07T07:31:19.373 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.unordered_write: 0 2024-03-07T07:31:19.374 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-07T07:31:19.374 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-07T07:31:19.374 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-07T07:31:19.374 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-07T07:31:19.374 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.row_cache: None 2024-03-07T07:31:19.374 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.wal_filter: None 2024-03-07T07:31:19.374 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-07T07:31:19.374 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.allow_ingest_behind: 0 2024-03-07T07:31:19.374 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.two_write_queues: 0 2024-03-07T07:31:19.374 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.manual_wal_flush: 0 2024-03-07T07:31:19.374 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.wal_compression: 0 2024-03-07T07:31:19.374 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.atomic_flush: 0 2024-03-07T07:31:19.374 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-07T07:31:19.374 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-07T07:31:19.375 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-07T07:31:19.375 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.log_readahead_size: 0 2024-03-07T07:31:19.375 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-07T07:31:19.375 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.best_efforts_recovery: 0 2024-03-07T07:31:19.375 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-07T07:31:19.375 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-07T07:31:19.375 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.allow_data_in_errors: 0 2024-03-07T07:31:19.375 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.db_host_id: __hostname__ 2024-03-07T07:31:19.375 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-07T07:31:19.375 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.max_background_jobs: 2 2024-03-07T07:31:19.375 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.max_background_compactions: -1 2024-03-07T07:31:19.375 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.max_subcompactions: 1 2024-03-07T07:31:19.375 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-07T07:31:19.375 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-07T07:31:19.376 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-07T07:31:19.376 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.max_total_wal_size: 0 2024-03-07T07:31:19.376 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-07T07:31:19.376 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-07T07:31:19.376 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-07T07:31:19.376 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-07T07:31:19.376 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.max_open_files: -1 2024-03-07T07:31:19.376 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.bytes_per_sync: 0 2024-03-07T07:31:19.376 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-07T07:31:19.376 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-07T07:31:19.376 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.compaction_readahead_size: 0 2024-03-07T07:31:19.376 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.max_background_flushes: -1 2024-03-07T07:31:19.376 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Compression algorithms supported: 2024-03-07T07:31:19.377 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-07T07:31:19.377 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: kZSTD supported: 0 2024-03-07T07:31:19.377 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: kXpressCompression supported: 0 2024-03-07T07:31:19.377 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: kLZ4HCCompression supported: 1 2024-03-07T07:31:19.377 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: kZlibCompression supported: 1 2024-03-07T07:31:19.377 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: kSnappyCompression supported: 1 2024-03-07T07:31:19.377 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: kLZ4Compression supported: 1 2024-03-07T07:31:19.377 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: kBZip2Compression supported: 0 2024-03-07T07:31:19.377 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-07T07:31:19.377 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-07T07:31:19.377 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-07T07:31:19.377 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-07T07:31:19.377 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-07T07:31:19.377 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.merge_operator: 2024-03-07T07:31:19.378 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.compaction_filter: None 2024-03-07T07:31:19.378 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.compaction_filter_factory: None 2024-03-07T07:31:19.378 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.sst_partitioner_factory: None 2024-03-07T07:31:19.378 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-07T07:31:19.378 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-07T07:31:19.378 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ba4fcdda20) 2024-03-07T07:31:19.378 INFO:journalctl@ceph.mon.a.smithi160.stdout: cache_index_and_filter_blocks: 1 2024-03-07T07:31:19.378 INFO:journalctl@ceph.mon.a.smithi160.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-07T07:31:19.378 INFO:journalctl@ceph.mon.a.smithi160.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-07T07:31:19.378 INFO:journalctl@ceph.mon.a.smithi160.stdout: pin_top_level_index_and_filter: 1 2024-03-07T07:31:19.378 INFO:journalctl@ceph.mon.a.smithi160.stdout: index_type: 0 2024-03-07T07:31:19.378 INFO:journalctl@ceph.mon.a.smithi160.stdout: data_block_index_type: 0 2024-03-07T07:31:19.378 INFO:journalctl@ceph.mon.a.smithi160.stdout: index_shortening: 1 2024-03-07T07:31:19.378 INFO:journalctl@ceph.mon.a.smithi160.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-07T07:31:19.379 INFO:journalctl@ceph.mon.a.smithi160.stdout: checksum: 4 2024-03-07T07:31:19.379 INFO:journalctl@ceph.mon.a.smithi160.stdout: no_block_cache: 0 2024-03-07T07:31:19.379 INFO:journalctl@ceph.mon.a.smithi160.stdout: block_cache: 0x55ba4fce3090 2024-03-07T07:31:19.379 INFO:journalctl@ceph.mon.a.smithi160.stdout: block_cache_name: BinnedLRUCache 2024-03-07T07:31:19.379 INFO:journalctl@ceph.mon.a.smithi160.stdout: block_cache_options: 2024-03-07T07:31:19.379 INFO:journalctl@ceph.mon.a.smithi160.stdout: capacity : 536870912 2024-03-07T07:31:19.379 INFO:journalctl@ceph.mon.a.smithi160.stdout: num_shard_bits : 4 2024-03-07T07:31:19.379 INFO:journalctl@ceph.mon.a.smithi160.stdout: strict_capacity_limit : 0 2024-03-07T07:31:19.379 INFO:journalctl@ceph.mon.a.smithi160.stdout: high_pri_pool_ratio: 0.000 2024-03-07T07:31:19.379 INFO:journalctl@ceph.mon.a.smithi160.stdout: block_cache_compressed: (nil) 2024-03-07T07:31:19.379 INFO:journalctl@ceph.mon.a.smithi160.stdout: persistent_cache: (nil) 2024-03-07T07:31:19.379 INFO:journalctl@ceph.mon.a.smithi160.stdout: block_size: 4096 2024-03-07T07:31:19.379 INFO:journalctl@ceph.mon.a.smithi160.stdout: block_size_deviation: 10 2024-03-07T07:31:19.379 INFO:journalctl@ceph.mon.a.smithi160.stdout: block_restart_interval: 16 2024-03-07T07:31:19.380 INFO:journalctl@ceph.mon.a.smithi160.stdout: index_block_restart_interval: 1 2024-03-07T07:31:19.380 INFO:journalctl@ceph.mon.a.smithi160.stdout: metadata_block_size: 4096 2024-03-07T07:31:19.380 INFO:journalctl@ceph.mon.a.smithi160.stdout: partition_filters: 0 2024-03-07T07:31:19.380 INFO:journalctl@ceph.mon.a.smithi160.stdout: use_delta_encoding: 1 2024-03-07T07:31:19.380 INFO:journalctl@ceph.mon.a.smithi160.stdout: filter_policy: bloomfilter 2024-03-07T07:31:19.380 INFO:journalctl@ceph.mon.a.smithi160.stdout: whole_key_filtering: 1 2024-03-07T07:31:19.380 INFO:journalctl@ceph.mon.a.smithi160.stdout: verify_compression: 0 2024-03-07T07:31:19.380 INFO:journalctl@ceph.mon.a.smithi160.stdout: read_amp_bytes_per_bit: 0 2024-03-07T07:31:19.380 INFO:journalctl@ceph.mon.a.smithi160.stdout: format_version: 5 2024-03-07T07:31:19.380 INFO:journalctl@ceph.mon.a.smithi160.stdout: enable_index_compression: 1 2024-03-07T07:31:19.380 INFO:journalctl@ceph.mon.a.smithi160.stdout: block_align: 0 2024-03-07T07:31:19.380 INFO:journalctl@ceph.mon.a.smithi160.stdout: max_auto_readahead_size: 262144 2024-03-07T07:31:19.380 INFO:journalctl@ceph.mon.a.smithi160.stdout: prepopulate_block_cache: 0 2024-03-07T07:31:19.381 INFO:journalctl@ceph.mon.a.smithi160.stdout: initial_auto_readahead_size: 8192 2024-03-07T07:31:19.381 INFO:journalctl@ceph.mon.a.smithi160.stdout: num_file_reads_for_auto_readahead: 2 2024-03-07T07:31:19.381 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.write_buffer_size: 33554432 2024-03-07T07:31:19.381 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.max_write_buffer_number: 2 2024-03-07T07:31:19.381 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.compression: NoCompression 2024-03-07T07:31:19.381 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.bottommost_compression: Disabled 2024-03-07T07:31:19.381 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.prefix_extractor: nullptr 2024-03-07T07:31:19.381 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-07T07:31:19.381 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.num_levels: 7 2024-03-07T07:31:19.381 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-07T07:31:19.381 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-07T07:31:19.381 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-07T07:31:19.381 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-07T07:31:19.381 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-07T07:31:19.382 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-07T07:31:19.382 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-07T07:31:19.382 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-07T07:31:19.382 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-07T07:31:19.382 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-07T07:31:19.382 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-07T07:31:19.382 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-07T07:31:19.382 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-07T07:31:19.382 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.compression_opts.level: 32767 2024-03-07T07:31:19.382 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.compression_opts.strategy: 0 2024-03-07T07:31:19.382 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-07T07:31:19.382 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-07T07:31:19.382 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-07T07:31:19.382 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-07T07:31:19.383 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.compression_opts.enabled: false 2024-03-07T07:31:19.383 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-07T07:31:19.383 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-07T07:31:19.383 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-07T07:31:19.383 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-07T07:31:19.383 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.target_file_size_base: 67108864 2024-03-07T07:31:19.383 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-07T07:31:19.383 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-07T07:31:19.383 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-07T07:31:19.383 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-07T07:31:19.383 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-07T07:31:19.383 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-07T07:31:19.383 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-07T07:31:19.383 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-07T07:31:19.383 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-07T07:31:19.384 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-07T07:31:19.384 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-07T07:31:19.384 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-07T07:31:19.384 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-07T07:31:19.384 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-07T07:31:19.384 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.arena_block_size: 1048576 2024-03-07T07:31:19.384 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-07T07:31:19.384 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-07T07:31:19.384 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.disable_auto_compactions: 0 2024-03-07T07:31:19.384 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-07T07:31:19.384 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-07T07:31:19.384 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-07T07:31:19.384 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-07T07:31:19.385 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-07T07:31:19.385 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-07T07:31:19.385 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-07T07:31:19.385 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-07T07:31:19.385 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-07T07:31:19.385 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-07T07:31:19.385 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.table_properties_collectors: 2024-03-07T07:31:19.385 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.inplace_update_support: 0 2024-03-07T07:31:19.385 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-07T07:31:19.385 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-07T07:31:19.385 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-07T07:31:19.385 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-07T07:31:19.385 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.bloom_locality: 0 2024-03-07T07:31:19.385 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.max_successive_merges: 0 2024-03-07T07:31:19.386 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-07T07:31:19.386 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.paranoid_file_checks: 0 2024-03-07T07:31:19.386 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.force_consistency_checks: 1 2024-03-07T07:31:19.386 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.report_bg_io_stats: 0 2024-03-07T07:31:19.386 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.ttl: 2592000 2024-03-07T07:31:19.386 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-07T07:31:19.386 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-07T07:31:19.386 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-07T07:31:19.386 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.enable_blob_files: false 2024-03-07T07:31:19.386 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.min_blob_size: 0 2024-03-07T07:31:19.386 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.blob_file_size: 268435456 2024-03-07T07:31:19.386 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-07T07:31:19.386 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-07T07:31:19.386 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-07T07:31:19.386 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-07T07:31:19.387 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-07T07:31:19.387 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.blob_file_starting_level: 0 2024-03-07T07:31:19.387 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-07T07:31:19.387 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-07T07:31:19.387 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-07T07:31:19.387 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 9443d794-7994-4b51-9827-504c34883e4e 2024-03-07T07:31:19.387 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709796679325778, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-07T07:31:19.387 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-07T07:31:19.387 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709796679326808, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 95589, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 318, "table_properties": {"data_size": 93579, "index_size": 292, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13934, "raw_average_key_size": 50, "raw_value_size": 86007, "raw_average_value_size": 310, "num_data_blocks": 11, "num_entries": 277, "num_filter_entries": 277, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1709796679, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "9443d794-7994-4b51-9827-504c34883e4e", "db_session_id": "EJ79XFV7VF3OAOSRA2R8", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-07T07:31:19.387 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709796679326890, "job": 1, "event": "recovery_finished"} 2024-03-07T07:31:19.387 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-07T07:31:19.387 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-07T07:31:19.387 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55ba4fdd6000 2024-03-07T07:31:19.388 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: DB pointer 0x55ba4fdc0000 2024-03-07T07:31:19.388 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-07T07:31:19.388 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-07T07:31:19.388 INFO:journalctl@ceph.mon.a.smithi160.stdout: ** DB Stats ** 2024-03-07T07:31:19.388 INFO:journalctl@ceph.mon.a.smithi160.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-07T07:31:19.388 INFO:journalctl@ceph.mon.a.smithi160.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-07T07:31:19.388 INFO:journalctl@ceph.mon.a.smithi160.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-07T07:31:19.388 INFO:journalctl@ceph.mon.a.smithi160.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-07T07:31:19.388 INFO:journalctl@ceph.mon.a.smithi160.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-07T07:31:19.388 INFO:journalctl@ceph.mon.a.smithi160.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-07T07:31:19.388 INFO:journalctl@ceph.mon.a.smithi160.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-07T07:31:19.388 INFO:journalctl@ceph.mon.a.smithi160.stdout: 2024-03-07T07:31:19.388 INFO:journalctl@ceph.mon.a.smithi160.stdout: ** Compaction Stats [default] ** 2024-03-07T07:31:19.388 INFO:journalctl@ceph.mon.a.smithi160.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-07T07:31:19.389 INFO:journalctl@ceph.mon.a.smithi160.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-07T07:31:19.389 INFO:journalctl@ceph.mon.a.smithi160.stdout: L0 2/0 95.18 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 140.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-07T07:31:19.389 INFO:journalctl@ceph.mon.a.smithi160.stdout: Sum 2/0 95.18 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 140.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-07T07:31:19.389 INFO:journalctl@ceph.mon.a.smithi160.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 140.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-07T07:31:19.389 INFO:journalctl@ceph.mon.a.smithi160.stdout: 2024-03-07T07:31:19.389 INFO:journalctl@ceph.mon.a.smithi160.stdout: ** Compaction Stats [default] ** 2024-03-07T07:31:19.389 INFO:journalctl@ceph.mon.a.smithi160.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-07T07:31:19.389 INFO:journalctl@ceph.mon.a.smithi160.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-07T07:31:19.389 INFO:journalctl@ceph.mon.a.smithi160.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 140.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-07T07:31:19.389 INFO:journalctl@ceph.mon.a.smithi160.stdout: 2024-03-07T07:31:19.389 INFO:journalctl@ceph.mon.a.smithi160.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-07T07:31:19.389 INFO:journalctl@ceph.mon.a.smithi160.stdout: 2024-03-07T07:31:19.389 INFO:journalctl@ceph.mon.a.smithi160.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-07T07:31:19.389 INFO:journalctl@ceph.mon.a.smithi160.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-07T07:31:19.390 INFO:journalctl@ceph.mon.a.smithi160.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-07T07:31:19.390 INFO:journalctl@ceph.mon.a.smithi160.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-07T07:31:19.390 INFO:journalctl@ceph.mon.a.smithi160.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-07T07:31:19.390 INFO:journalctl@ceph.mon.a.smithi160.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-07T07:31:19.390 INFO:journalctl@ceph.mon.a.smithi160.stdout: Cumulative compaction: 0.00 GB write, 30.49 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-07T07:31:19.390 INFO:journalctl@ceph.mon.a.smithi160.stdout: Interval compaction: 0.00 GB write, 30.49 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-07T07:31:19.390 INFO:journalctl@ceph.mon.a.smithi160.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-07T07:31:19.390 INFO:journalctl@ceph.mon.a.smithi160.stdout: Block cache BinnedLRUCache@0x55ba4fce3090#2 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-03-07T07:31:19.390 INFO:journalctl@ceph.mon.a.smithi160.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-03-07T07:31:19.390 INFO:journalctl@ceph.mon.a.smithi160.stdout: 2024-03-07T07:31:19.390 INFO:journalctl@ceph.mon.a.smithi160.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-07T07:31:19.390 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: starting mon.a rank 0 at public addrs [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] at bind addrs [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 826f6582-dc54-11ee-95c6-87774f69a715 2024-03-07T07:31:19.390 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: mon.a@-1(???) e1 preinit fsid 826f6582-dc54-11ee-95c6-87774f69a715 2024-03-07T07:31:19.390 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: mon.a@-1(???).mds e1 new map 2024-03-07T07:31:19.391 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: mon.a@-1(???).mds e1 print_map 2024-03-07T07:31:19.391 INFO:journalctl@ceph.mon.a.smithi160.stdout: e1 2024-03-07T07:31:19.391 INFO:journalctl@ceph.mon.a.smithi160.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-07T07:31:19.391 INFO:journalctl@ceph.mon.a.smithi160.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-07T07:31:19.391 INFO:journalctl@ceph.mon.a.smithi160.stdout: legacy client fscid: -1 2024-03-07T07:31:19.391 INFO:journalctl@ceph.mon.a.smithi160.stdout: 2024-03-07T07:31:19.391 INFO:journalctl@ceph.mon.a.smithi160.stdout: No filesystems configured 2024-03-07T07:31:19.391 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-07T07:31:19.391 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-07T07:31:19.391 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-07T07:31:19.391 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-07T07:31:19.391 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-07T07:31:19.391 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 bash[31945]: 23e49f05d5ab628af000486507c44ce0bbea95cc185028910c25c1f64531a00e 2024-03-07T07:31:19.391 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 systemd[1]: Started Ceph mon.a for 826f6582-dc54-11ee-95c6-87774f69a715. 2024-03-07T07:31:19.392 INFO:teuthology.orchestra.run.smithi160.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-07T07:31:19.661 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-07T07:31:19.662 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: monmap e1: 1 mons at {a=[v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-07T07:31:19.662 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: fsmap 2024-03-07T07:31:19.662 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: osdmap e1: 0 total, 0 up, 0 in 2024-03-07T07:31:19.662 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:19 smithi160 ceph-mon[31963]: mgrmap e1: no daemons active 2024-03-07T07:31:20.442 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:19.763+0000 7fc727c8e700 1 Processor -- start 2024-03-07T07:31:20.447 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:19.763+0000 7fc727c8e700 1 -- start start 2024-03-07T07:31:20.447 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:19.764+0000 7fc727c8e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc720108580 0x7fc72010a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:20.448 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:19.764+0000 7fc727c8e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc720075860 con 0x7fc720108580 2024-03-07T07:31:20.448 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:19.764+0000 7fc725a2a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc720108580 0x7fc72010a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:20.448 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:19.764+0000 7fc725a2a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc720108580 0x7fc72010a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59912/0 (socket says 172.21.15.160:59912) 2024-03-07T07:31:20.448 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:19.764+0000 7fc725a2a700 1 -- 172.21.15.160:0/3402783550 learned_addr learned my addr 172.21.15.160:0/3402783550 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:31:20.448 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:19.764+0000 7fc725a2a700 1 -- 172.21.15.160:0/3402783550 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc72010aeb0 con 0x7fc720108580 2024-03-07T07:31:20.448 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:19.764+0000 7fc725a2a700 1 --2- 172.21.15.160:0/3402783550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc720108580 0x7fc72010a970 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc71c0098a0 tx=0x7fc71c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de5e13336b085bf8 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:20.448 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:19.765+0000 7fc724a28700 1 -- 172.21.15.160:0/3402783550 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc71c004160 con 0x7fc720108580 2024-03-07T07:31:20.448 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:19.765+0000 7fc724a28700 1 -- 172.21.15.160:0/3402783550 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc71c004300 con 0x7fc720108580 2024-03-07T07:31:20.448 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:19.765+0000 7fc724a28700 1 -- 172.21.15.160:0/3402783550 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc71c011b50 con 0x7fc720108580 2024-03-07T07:31:20.448 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:19.765+0000 7fc727c8e700 1 -- 172.21.15.160:0/3402783550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc720108580 msgr2=0x7fc72010a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:20.448 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:19.765+0000 7fc727c8e700 1 --2- 172.21.15.160:0/3402783550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc720108580 0x7fc72010a970 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc71c0098a0 tx=0x7fc71c009bb0 comp rx=0 tx=0).stop 2024-03-07T07:31:20.448 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:19.765+0000 7fc727c8e700 1 -- 172.21.15.160:0/3402783550 shutdown_connections 2024-03-07T07:31:20.448 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:19.765+0000 7fc727c8e700 1 --2- 172.21.15.160:0/3402783550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc720108580 0x7fc72010a970 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:20.449 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:19.765+0000 7fc727c8e700 1 -- 172.21.15.160:0/3402783550 >> 172.21.15.160:0/3402783550 conn(0x7fc720101f70 msgr2=0x7fc720104390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:31:20.449 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:19.765+0000 7fc727c8e700 1 -- 172.21.15.160:0/3402783550 shutdown_connections 2024-03-07T07:31:20.449 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:19.765+0000 7fc727c8e700 1 -- 172.21.15.160:0/3402783550 wait complete. 2024-03-07T07:31:20.449 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:19.766+0000 7fc727c8e700 1 Processor -- start 2024-03-07T07:31:20.449 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:19.766+0000 7fc727c8e700 1 -- start start 2024-03-07T07:31:20.449 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:19.766+0000 7fc727c8e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc720108580 0x7fc72019bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:20.449 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:19.766+0000 7fc727c8e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72010be80 con 0x7fc720108580 2024-03-07T07:31:20.449 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:19.766+0000 7fc725a2a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc720108580 0x7fc72019bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:20.449 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:19.766+0000 7fc725a2a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc720108580 0x7fc72019bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59916/0 (socket says 172.21.15.160:59916) 2024-03-07T07:31:20.449 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:19.766+0000 7fc725a2a700 1 -- 172.21.15.160:0/791286957 learned_addr learned my addr 172.21.15.160:0/791286957 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:31:20.449 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:19.766+0000 7fc725a2a700 1 -- 172.21.15.160:0/791286957 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc72019c040 con 0x7fc720108580 2024-03-07T07:31:20.449 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:19.767+0000 7fc725a2a700 1 --2- 172.21.15.160:0/791286957 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc720108580 0x7fc72019bb00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc71c011b60 tx=0x7fc71c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:20.449 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:19.767+0000 7fc716ffd700 1 -- 172.21.15.160:0/791286957 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc71c003970 con 0x7fc720108580 2024-03-07T07:31:20.449 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:19.767+0000 7fc716ffd700 1 -- 172.21.15.160:0/791286957 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc71c003b10 con 0x7fc720108580 2024-03-07T07:31:20.450 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:19.767+0000 7fc727c8e700 1 -- 172.21.15.160:0/791286957 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc72019c2d0 con 0x7fc720108580 2024-03-07T07:31:20.450 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:19.767+0000 7fc716ffd700 1 -- 172.21.15.160:0/791286957 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc71c024610 con 0x7fc720108580 2024-03-07T07:31:20.450 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:19.767+0000 7fc727c8e700 1 -- 172.21.15.160:0/791286957 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc72019efc0 con 0x7fc720108580 2024-03-07T07:31:20.450 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:19.767+0000 7fc716ffd700 1 -- 172.21.15.160:0/791286957 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fc71c024ca0 con 0x7fc720108580 2024-03-07T07:31:20.450 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:19.767+0000 7fc716ffd700 1 -- 172.21.15.160:0/791286957 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc71c011e00 con 0x7fc720108580 2024-03-07T07:31:20.450 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:19.768+0000 7fc727c8e700 1 -- 172.21.15.160:0/791286957 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc720063780 con 0x7fc720108580 2024-03-07T07:31:20.450 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:19.770+0000 7fc716ffd700 1 -- 172.21.15.160:0/791286957 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fc71c043dd0 con 0x7fc720108580 2024-03-07T07:31:20.450 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:19.836+0000 7fc727c8e700 1 -- 172.21.15.160:0/791286957 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7fc72019f190 con 0x7fc720108580 2024-03-07T07:31:20.450 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:19.841+0000 7fc716ffd700 1 -- 172.21.15.160:0/791286957 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7fc71c032030 con 0x7fc720108580 2024-03-07T07:31:20.450 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:19.841+0000 7fc727c8e700 1 -- 172.21.15.160:0/791286957 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc720108580 msgr2=0x7fc72019bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:20.450 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:19.841+0000 7fc727c8e700 1 --2- 172.21.15.160:0/791286957 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc720108580 0x7fc72019bb00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc71c011b60 tx=0x7fc71c011c40 comp rx=0 tx=0).stop 2024-03-07T07:31:20.450 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:19.842+0000 7fc727c8e700 1 -- 172.21.15.160:0/791286957 shutdown_connections 2024-03-07T07:31:20.450 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:19.842+0000 7fc727c8e700 1 --2- 172.21.15.160:0/791286957 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc720108580 0x7fc72019bb00 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:20.450 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:19.842+0000 7fc727c8e700 1 -- 172.21.15.160:0/791286957 >> 172.21.15.160:0/791286957 conn(0x7fc720101f70 msgr2=0x7fc720102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:31:20.451 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:19.842+0000 7fc727c8e700 1 -- 172.21.15.160:0/791286957 shutdown_connections 2024-03-07T07:31:20.451 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:19.842+0000 7fc727c8e700 1 -- 172.21.15.160:0/791286957 wait complete. 2024-03-07T07:31:20.451 INFO:teuthology.orchestra.run.smithi160.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-07T07:31:20.473 INFO:teuthology.orchestra.run.smithi160.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-07T07:31:20.473 INFO:teuthology.orchestra.run.smithi160.stdout:Creating mgr... 2024-03-07T07:31:20.473 INFO:teuthology.orchestra.run.smithi160.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-07T07:31:20.474 INFO:teuthology.orchestra.run.smithi160.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-07T07:31:20.724 INFO:teuthology.orchestra.run.smithi160.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-826f6582-dc54-11ee-95c6-87774f69a715@mgr.x 2024-03-07T07:31:20.725 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stderr Failed to reset failed state of unit ceph-826f6582-dc54-11ee-95c6-87774f69a715@mgr.x.service: Unit ceph-826f6582-dc54-11ee-95c6-87774f69a715@mgr.x.service not loaded. 2024-03-07T07:31:20.876 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-826f6582-dc54-11ee-95c6-87774f69a715.target.wants/ceph-826f6582-dc54-11ee-95c6-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-826f6582-dc54-11ee-95c6-87774f69a715@.service. 2024-03-07T07:31:21.108 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:20 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/791286957' entity='client.admin' 2024-03-07T07:31:21.109 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:20 smithi160 systemd[1]: Starting Ceph mgr.x for 826f6582-dc54-11ee-95c6-87774f69a715... 2024-03-07T07:31:21.378 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:21 smithi160 podman[32181]: 2024-03-07 07:31:21.109171143 +0000 UTC m=+0.108824575 container create c6ae33bdae0b8dd26272801a4ce3d3885e0323d0da99aafb66ad680f6825ff38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-07T07:31:21.378 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:21 smithi160 podman[32181]: 2024-03-07 07:31:21.018917319 +0000 UTC m=+0.018570764 image pull e362cd7cebfee51077fb90b7f01895811140c9a4272edd83ad9910b05e3a46f6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 2024-03-07T07:31:21.378 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:21 smithi160 podman[32181]: 2024-03-07 07:31:21.294827757 +0000 UTC m=+0.294481194 container init c6ae33bdae0b8dd26272801a4ce3d3885e0323d0da99aafb66ad680f6825ff38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, ceph=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-8b896a9, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-03-07T07:31:21.378 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:21 smithi160 podman[32181]: 2024-03-07 07:31:21.297323291 +0000 UTC m=+0.296976722 container start c6ae33bdae0b8dd26272801a4ce3d3885e0323d0da99aafb66ad680f6825ff38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True) 2024-03-07T07:31:21.378 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:21 smithi160 bash[32181]: c6ae33bdae0b8dd26272801a4ce3d3885e0323d0da99aafb66ad680f6825ff38 2024-03-07T07:31:21.391 INFO:teuthology.orchestra.run.smithi160.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-07T07:31:21.391 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stdout disabled 2024-03-07T07:31:21.399 INFO:teuthology.orchestra.run.smithi160.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-07T07:31:21.399 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stdout inactive 2024-03-07T07:31:21.399 INFO:teuthology.orchestra.run.smithi160.stdout:firewalld.service is not enabled 2024-03-07T07:31:21.399 INFO:teuthology.orchestra.run.smithi160.stdout:Not possible to enable service . firewalld.service is not available 2024-03-07T07:31:21.407 INFO:teuthology.orchestra.run.smithi160.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-07T07:31:21.408 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stdout disabled 2024-03-07T07:31:21.415 INFO:teuthology.orchestra.run.smithi160.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-07T07:31:21.415 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stdout inactive 2024-03-07T07:31:21.416 INFO:teuthology.orchestra.run.smithi160.stdout:firewalld.service is not enabled 2024-03-07T07:31:21.416 INFO:teuthology.orchestra.run.smithi160.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-07T07:31:21.416 INFO:teuthology.orchestra.run.smithi160.stdout:Waiting for mgr to start... 2024-03-07T07:31:21.416 INFO:teuthology.orchestra.run.smithi160.stdout:Waiting for mgr... 2024-03-07T07:31:21.677 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:21 smithi160 systemd[1]: Started Ceph mgr.x for 826f6582-dc54-11ee-95c6-87774f69a715. 2024-03-07T07:31:21.961 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:21 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:21.848+0000 7fa1462ae200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-07T07:31:21.962 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:21 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:21.945+0000 7fa1462ae200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-07T07:31:22.266 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:22 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/3555054173' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-07T07:31:22.554 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 2024-03-07T07:31:22.554 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout { 2024-03-07T07:31:22.554 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "fsid": "826f6582-dc54-11ee-95c6-87774f69a715", 2024-03-07T07:31:22.555 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "health": { 2024-03-07T07:31:22.555 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-07T07:31:22.555 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-07T07:31:22.555 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-07T07:31:22.555 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-03-07T07:31:22.555 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-07T07:31:22.555 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-07T07:31:22.555 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 0 2024-03-07T07:31:22.555 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout ], 2024-03-07T07:31:22.555 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-07T07:31:22.555 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "a" 2024-03-07T07:31:22.555 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout ], 2024-03-07T07:31:22.555 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-03-07T07:31:22.556 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-07T07:31:22.556 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-07T07:31:22.556 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-07T07:31:22.556 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-07T07:31:22.556 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-03-07T07:31:22.556 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-07T07:31:22.556 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-07T07:31:22.556 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-07T07:31:22.556 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-07T07:31:22.556 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-07T07:31:22.556 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-07T07:31:22.556 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-07T07:31:22.556 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-07T07:31:22.556 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-03-07T07:31:22.557 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-07T07:31:22.557 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-07T07:31:22.557 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-07T07:31:22.557 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-07T07:31:22.557 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-07T07:31:22.557 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-07T07:31:22.557 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-07T07:31:22.557 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-07T07:31:22.557 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-07T07:31:22.557 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-03-07T07:31:22.557 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-07T07:31:22.557 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-07T07:31:22.557 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-07T07:31:22.557 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-07T07:31:22.558 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-03-07T07:31:22.558 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-07T07:31:22.558 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-07T07:31:22.558 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-07T07:31:22.558 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-07T07:31:22.558 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-07T07:31:22.558 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-07T07:31:22.558 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "restful" 2024-03-07T07:31:22.558 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout ], 2024-03-07T07:31:22.558 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-07T07:31:22.558 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-03-07T07:31:22.558 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-07T07:31:22.558 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-07T07:31:22.558 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "modified": "2024-03-07T07:31:14.230017+0000", 2024-03-07T07:31:22.559 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-07T07:31:22.559 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-03-07T07:31:22.559 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-07T07:31:22.559 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout } 2024-03-07T07:31:22.559 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:21.881+0000 7f62f34d2700 1 Processor -- start 2024-03-07T07:31:22.559 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:21.881+0000 7f62f34d2700 1 -- start start 2024-03-07T07:31:22.559 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:21.881+0000 7f62f34d2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62ec07de80 0x7f62ec07c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:22.559 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:21.882+0000 7f62f34d2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec07c8b0 con 0x7f62ec07de80 2024-03-07T07:31:22.559 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:21.882+0000 7f62f126e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62ec07de80 0x7f62ec07c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:22.559 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:21.882+0000 7f62f126e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62ec07de80 0x7f62ec07c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59956/0 (socket says 172.21.15.160:59956) 2024-03-07T07:31:22.559 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:21.882+0000 7f62f126e700 1 -- 172.21.15.160:0/241955256 learned_addr learned my addr 172.21.15.160:0/241955256 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:31:22.559 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:21.882+0000 7f62f126e700 1 -- 172.21.15.160:0/241955256 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ec07ca30 con 0x7f62ec07de80 2024-03-07T07:31:22.559 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:21.882+0000 7f62f126e700 1 --2- 172.21.15.160:0/241955256 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62ec07de80 0x7f62ec07c2e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f62e80098a0 tx=0x7f62e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c416cfca5b05d5d server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:22.560 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:21.883+0000 7f62e3fff700 1 -- 172.21.15.160:0/241955256 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f62e8004160 con 0x7f62ec07de80 2024-03-07T07:31:22.560 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:21.883+0000 7f62e3fff700 1 -- 172.21.15.160:0/241955256 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f62e8004300 con 0x7f62ec07de80 2024-03-07T07:31:22.560 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:21.883+0000 7f62e3fff700 1 -- 172.21.15.160:0/241955256 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f62e8011b50 con 0x7f62ec07de80 2024-03-07T07:31:22.560 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:21.883+0000 7f62f34d2700 1 -- 172.21.15.160:0/241955256 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62ec07de80 msgr2=0x7f62ec07c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:22.560 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:21.883+0000 7f62f34d2700 1 --2- 172.21.15.160:0/241955256 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62ec07de80 0x7f62ec07c2e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f62e80098a0 tx=0x7f62e8009bb0 comp rx=0 tx=0).stop 2024-03-07T07:31:22.560 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:21.883+0000 7f62f34d2700 1 -- 172.21.15.160:0/241955256 shutdown_connections 2024-03-07T07:31:22.560 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:21.883+0000 7f62f34d2700 1 --2- 172.21.15.160:0/241955256 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62ec07de80 0x7f62ec07c2e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:22.560 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:21.883+0000 7f62f34d2700 1 -- 172.21.15.160:0/241955256 >> 172.21.15.160:0/241955256 conn(0x7f62ec103060 msgr2=0x7f62ec1054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:31:22.560 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:21.883+0000 7f62f34d2700 1 -- 172.21.15.160:0/241955256 shutdown_connections 2024-03-07T07:31:22.560 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:21.883+0000 7f62f34d2700 1 -- 172.21.15.160:0/241955256 wait complete. 2024-03-07T07:31:22.560 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:21.884+0000 7f62f34d2700 1 Processor -- start 2024-03-07T07:31:22.560 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:21.884+0000 7f62f34d2700 1 -- start start 2024-03-07T07:31:22.560 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:21.884+0000 7f62f34d2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62ec07de80 0x7f62ec19dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:22.561 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:21.884+0000 7f62f34d2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec10e0f0 con 0x7f62ec07de80 2024-03-07T07:31:22.561 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:21.884+0000 7f62f126e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62ec07de80 0x7f62ec19dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:22.561 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:21.884+0000 7f62f126e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62ec07de80 0x7f62ec19dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59970/0 (socket says 172.21.15.160:59970) 2024-03-07T07:31:22.561 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:21.884+0000 7f62f126e700 1 -- 172.21.15.160:0/3555054173 learned_addr learned my addr 172.21.15.160:0/3555054173 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:31:22.561 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:21.885+0000 7f62f126e700 1 -- 172.21.15.160:0/3555054173 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ec19e2e0 con 0x7f62ec07de80 2024-03-07T07:31:22.561 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:21.885+0000 7f62f126e700 1 --2- 172.21.15.160:0/3555054173 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62ec07de80 0x7f62ec19dda0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f62e8009fb0 tx=0x7f62e8011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:22.561 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:21.885+0000 7f62e27fc700 1 -- 172.21.15.160:0/3555054173 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f62e80038b0 con 0x7f62ec07de80 2024-03-07T07:31:22.561 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:21.885+0000 7f62e27fc700 1 -- 172.21.15.160:0/3555054173 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f62e8011e50 con 0x7f62ec07de80 2024-03-07T07:31:22.561 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:21.885+0000 7f62f34d2700 1 -- 172.21.15.160:0/3555054173 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62ec19e570 con 0x7f62ec07de80 2024-03-07T07:31:22.561 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:21.885+0000 7f62e27fc700 1 -- 172.21.15.160:0/3555054173 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f62e8019550 con 0x7f62ec07de80 2024-03-07T07:31:22.561 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:21.885+0000 7f62f34d2700 1 -- 172.21.15.160:0/3555054173 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62ec1a1220 con 0x7f62ec07de80 2024-03-07T07:31:22.561 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:21.885+0000 7f62e27fc700 1 -- 172.21.15.160:0/3555054173 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f62e8019b90 con 0x7f62ec07de80 2024-03-07T07:31:22.561 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:21.885+0000 7f62e27fc700 1 -- 172.21.15.160:0/3555054173 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f62e8024730 con 0x7f62ec07de80 2024-03-07T07:31:22.561 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:21.886+0000 7f62f34d2700 1 -- 172.21.15.160:0/3555054173 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62ec063780 con 0x7f62ec07de80 2024-03-07T07:31:22.562 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:21.888+0000 7f62e27fc700 1 -- 172.21.15.160:0/3555054173 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f62e802dda0 con 0x7f62ec07de80 2024-03-07T07:31:22.562 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:21.961+0000 7f62f34d2700 1 -- 172.21.15.160:0/3555054173 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f62ec1a1490 con 0x7f62ec07de80 2024-03-07T07:31:22.562 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:21.961+0000 7f62e27fc700 1 -- 172.21.15.160:0/3555054173 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f62e8032030 con 0x7f62ec07de80 2024-03-07T07:31:22.562 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:21.963+0000 7f62f34d2700 1 -- 172.21.15.160:0/3555054173 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62ec07de80 msgr2=0x7f62ec19dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:22.562 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:21.963+0000 7f62f34d2700 1 --2- 172.21.15.160:0/3555054173 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62ec07de80 0x7f62ec19dda0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f62e8009fb0 tx=0x7f62e8011bf0 comp rx=0 tx=0).stop 2024-03-07T07:31:22.562 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:21.963+0000 7f62f34d2700 1 -- 172.21.15.160:0/3555054173 shutdown_connections 2024-03-07T07:31:22.562 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:21.963+0000 7f62f34d2700 1 --2- 172.21.15.160:0/3555054173 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62ec07de80 0x7f62ec19dda0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:22.562 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:21.963+0000 7f62f34d2700 1 -- 172.21.15.160:0/3555054173 >> 172.21.15.160:0/3555054173 conn(0x7f62ec103060 msgr2=0x7f62ec103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:31:22.562 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:21.963+0000 7f62f34d2700 1 -- 172.21.15.160:0/3555054173 shutdown_connections 2024-03-07T07:31:22.562 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:21.963+0000 7f62f34d2700 1 -- 172.21.15.160:0/3555054173 wait complete. 2024-03-07T07:31:22.562 INFO:teuthology.orchestra.run.smithi160.stdout:mgr not available, waiting (1/15)... 2024-03-07T07:31:22.834 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:22 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:22.602+0000 7fa1462ae200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-07T07:31:22.834 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:22 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:22.694+0000 7fa1462ae200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-07T07:31:22.835 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:22 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:22.834+0000 7fa1462ae200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-07T07:31:23.177 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:22 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:22.928+0000 7fa1462ae200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-07T07:31:23.178 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:23 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:23.014+0000 7fa1462ae200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-07T07:31:23.471 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:23 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:23.275+0000 7fa1462ae200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-07T07:31:23.471 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:23 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:23.385+0000 7fa1462ae200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-07T07:31:23.471 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:23 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:23.471+0000 7fa1462ae200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-07T07:31:23.753 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:23 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:23.568+0000 7fa1462ae200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-07T07:31:23.753 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:23 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:23.752+0000 7fa1462ae200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-07T07:31:24.928 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:24 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:24.775+0000 7fa1462ae200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-07T07:31:24.928 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:24 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:24.914+0000 7fa1462ae200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-07T07:31:25.260 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:25 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/1777496865' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-07T07:31:25.548 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:25 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:25.260+0000 7fa1462ae200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-07T07:31:25.548 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:25 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:25.352+0000 7fa1462ae200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-07T07:31:25.548 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:25 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:25.548+0000 7fa1462ae200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-07T07:31:25.660 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 2024-03-07T07:31:25.662 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout { 2024-03-07T07:31:25.662 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "fsid": "826f6582-dc54-11ee-95c6-87774f69a715", 2024-03-07T07:31:25.663 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "health": { 2024-03-07T07:31:25.663 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-07T07:31:25.663 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-07T07:31:25.663 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-07T07:31:25.663 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-03-07T07:31:25.663 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-07T07:31:25.663 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-07T07:31:25.663 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 0 2024-03-07T07:31:25.663 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout ], 2024-03-07T07:31:25.663 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-07T07:31:25.663 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "a" 2024-03-07T07:31:25.663 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout ], 2024-03-07T07:31:25.663 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-03-07T07:31:25.663 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-07T07:31:25.664 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-07T07:31:25.664 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-07T07:31:25.664 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-07T07:31:25.664 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-03-07T07:31:25.664 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-07T07:31:25.664 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-07T07:31:25.664 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-07T07:31:25.664 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-07T07:31:25.672 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-07T07:31:25.672 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-07T07:31:25.672 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-07T07:31:25.672 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-07T07:31:25.672 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-03-07T07:31:25.672 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-07T07:31:25.672 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-07T07:31:25.672 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-07T07:31:25.672 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-07T07:31:25.672 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-07T07:31:25.672 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-07T07:31:25.673 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-07T07:31:25.673 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-07T07:31:25.673 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-07T07:31:25.673 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-03-07T07:31:25.673 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-07T07:31:25.673 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-07T07:31:25.673 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-07T07:31:25.673 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-07T07:31:25.673 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-03-07T07:31:25.673 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-07T07:31:25.673 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-07T07:31:25.673 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-07T07:31:25.673 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-07T07:31:25.673 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-07T07:31:25.674 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-07T07:31:25.674 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "restful" 2024-03-07T07:31:25.674 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout ], 2024-03-07T07:31:25.674 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-07T07:31:25.674 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-03-07T07:31:25.674 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-07T07:31:25.674 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-07T07:31:25.674 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "modified": "2024-03-07T07:31:14.230017+0000", 2024-03-07T07:31:25.674 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-07T07:31:25.674 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-03-07T07:31:25.674 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-07T07:31:25.674 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout } 2024-03-07T07:31:25.674 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:24.978+0000 7fdea40c3700 1 Processor -- start 2024-03-07T07:31:25.674 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:24.979+0000 7fdea40c3700 1 -- start start 2024-03-07T07:31:25.674 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:24.979+0000 7fdea40c3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde9c1063b0 0x7fde9c1067d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:25.675 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:24.979+0000 7fdea40c3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c07be10 con 0x7fde9c1063b0 2024-03-07T07:31:25.675 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:24.979+0000 7fdea1e5f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde9c1063b0 0x7fde9c1067d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:25.675 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:24.979+0000 7fdea1e5f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde9c1063b0 0x7fde9c1067d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59978/0 (socket says 172.21.15.160:59978) 2024-03-07T07:31:25.675 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:24.979+0000 7fdea1e5f700 1 -- 172.21.15.160:0/489404719 learned_addr learned my addr 172.21.15.160:0/489404719 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:31:25.675 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:24.980+0000 7fdea1e5f700 1 -- 172.21.15.160:0/489404719 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde9c106d10 con 0x7fde9c1063b0 2024-03-07T07:31:25.675 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:24.980+0000 7fdea1e5f700 1 --2- 172.21.15.160:0/489404719 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde9c1063b0 0x7fde9c1067d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fde980098a0 tx=0x7fde98009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee9d25cc63c85982 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:25.675 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:24.980+0000 7fdea0e5d700 1 -- 172.21.15.160:0/489404719 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fde98004160 con 0x7fde9c1063b0 2024-03-07T07:31:25.675 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:24.980+0000 7fdea0e5d700 1 -- 172.21.15.160:0/489404719 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fde98004300 con 0x7fde9c1063b0 2024-03-07T07:31:25.675 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:24.980+0000 7fdea0e5d700 1 -- 172.21.15.160:0/489404719 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fde98011b50 con 0x7fde9c1063b0 2024-03-07T07:31:25.675 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:24.981+0000 7fdea40c3700 1 -- 172.21.15.160:0/489404719 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde9c1063b0 msgr2=0x7fde9c1067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:25.675 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:24.981+0000 7fdea40c3700 1 --2- 172.21.15.160:0/489404719 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde9c1063b0 0x7fde9c1067d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fde980098a0 tx=0x7fde98009bb0 comp rx=0 tx=0).stop 2024-03-07T07:31:25.675 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:24.981+0000 7fdea40c3700 1 -- 172.21.15.160:0/489404719 shutdown_connections 2024-03-07T07:31:25.675 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:24.981+0000 7fdea40c3700 1 --2- 172.21.15.160:0/489404719 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde9c1063b0 0x7fde9c1067d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:25.675 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:24.981+0000 7fdea40c3700 1 -- 172.21.15.160:0/489404719 >> 172.21.15.160:0/489404719 conn(0x7fde9c101f90 msgr2=0x7fde9c1043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:31:25.676 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:24.981+0000 7fdea40c3700 1 -- 172.21.15.160:0/489404719 shutdown_connections 2024-03-07T07:31:25.676 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:24.981+0000 7fdea40c3700 1 -- 172.21.15.160:0/489404719 wait complete. 2024-03-07T07:31:25.676 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:24.982+0000 7fdea40c3700 1 Processor -- start 2024-03-07T07:31:25.676 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:24.982+0000 7fdea40c3700 1 -- start start 2024-03-07T07:31:25.676 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:24.982+0000 7fdea40c3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde9c1063b0 0x7fde9c19dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:25.676 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:24.982+0000 7fdea40c3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c107da0 con 0x7fde9c1063b0 2024-03-07T07:31:25.676 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:24.982+0000 7fdea1e5f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde9c1063b0 0x7fde9c19dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:25.676 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:24.982+0000 7fdea1e5f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde9c1063b0 0x7fde9c19dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59994/0 (socket says 172.21.15.160:59994) 2024-03-07T07:31:25.676 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:24.982+0000 7fdea1e5f700 1 -- 172.21.15.160:0/1777496865 learned_addr learned my addr 172.21.15.160:0/1777496865 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:31:25.676 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:24.982+0000 7fdea1e5f700 1 -- 172.21.15.160:0/1777496865 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde9c19e1e0 con 0x7fde9c1063b0 2024-03-07T07:31:25.676 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:24.982+0000 7fdea1e5f700 1 --2- 172.21.15.160:0/1777496865 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde9c1063b0 0x7fde9c19dca0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fde98011670 tx=0x7fde98011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:25.676 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:24.983+0000 7fde92ffd700 1 -- 172.21.15.160:0/1777496865 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fde980039c0 con 0x7fde9c1063b0 2024-03-07T07:31:25.676 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:24.983+0000 7fde92ffd700 1 -- 172.21.15.160:0/1777496865 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fde98011e50 con 0x7fde9c1063b0 2024-03-07T07:31:25.677 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:24.983+0000 7fde92ffd700 1 -- 172.21.15.160:0/1777496865 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fde9801a5c0 con 0x7fde9c1063b0 2024-03-07T07:31:25.677 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:24.983+0000 7fdea40c3700 1 -- 172.21.15.160:0/1777496865 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde9c19e470 con 0x7fde9c1063b0 2024-03-07T07:31:25.677 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:24.983+0000 7fdea40c3700 1 -- 172.21.15.160:0/1777496865 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde9c1a1160 con 0x7fde9c1063b0 2024-03-07T07:31:25.677 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:24.983+0000 7fde92ffd700 1 -- 172.21.15.160:0/1777496865 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fde9801ac50 con 0x7fde9c1063b0 2024-03-07T07:31:25.677 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:24.983+0000 7fde92ffd700 1 -- 172.21.15.160:0/1777496865 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fde98019700 con 0x7fde9c1063b0 2024-03-07T07:31:25.677 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:24.984+0000 7fdea40c3700 1 -- 172.21.15.160:0/1777496865 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde9c063780 con 0x7fde9c1063b0 2024-03-07T07:31:25.677 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:24.986+0000 7fde92ffd700 1 -- 172.21.15.160:0/1777496865 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fde98003b60 con 0x7fde9c1063b0 2024-03-07T07:31:25.677 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:25.058+0000 7fdea40c3700 1 -- 172.21.15.160:0/1777496865 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fde9c02dc40 con 0x7fde9c1063b0 2024-03-07T07:31:25.677 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:25.058+0000 7fde92ffd700 1 -- 172.21.15.160:0/1777496865 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fde98019900 con 0x7fde9c1063b0 2024-03-07T07:31:25.677 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:25.059+0000 7fdea40c3700 1 -- 172.21.15.160:0/1777496865 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde9c1063b0 msgr2=0x7fde9c19dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:25.677 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:25.059+0000 7fdea40c3700 1 --2- 172.21.15.160:0/1777496865 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde9c1063b0 0x7fde9c19dca0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fde98011670 tx=0x7fde98011b60 comp rx=0 tx=0).stop 2024-03-07T07:31:25.677 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:25.059+0000 7fdea40c3700 1 -- 172.21.15.160:0/1777496865 shutdown_connections 2024-03-07T07:31:25.677 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:25.059+0000 7fdea40c3700 1 --2- 172.21.15.160:0/1777496865 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde9c1063b0 0x7fde9c19dca0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:25.677 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:25.059+0000 7fdea40c3700 1 -- 172.21.15.160:0/1777496865 >> 172.21.15.160:0/1777496865 conn(0x7fde9c101f90 msgr2=0x7fde9c194910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:31:25.678 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:25.059+0000 7fdea40c3700 1 -- 172.21.15.160:0/1777496865 shutdown_connections 2024-03-07T07:31:25.678 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:25.059+0000 7fdea40c3700 1 -- 172.21.15.160:0/1777496865 wait complete. 2024-03-07T07:31:25.678 INFO:teuthology.orchestra.run.smithi160.stdout:mgr not available, waiting (2/15)... 2024-03-07T07:31:25.927 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:25 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:25.735+0000 7fa1462ae200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-07T07:31:26.281 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:26 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:26.019+0000 7fa1462ae200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-07T07:31:26.677 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:26 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:26.280+0000 7fa1462ae200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-07T07:31:26.678 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:26 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:26.516+0000 7fa1462ae200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-07T07:31:27.678 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:27 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:27.312+0000 7fa1462ae200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-07T07:31:28.075 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:27 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:27.894+0000 7fa1462ae200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-07T07:31:28.076 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:27 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:27.995+0000 7fa1462ae200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-07T07:31:28.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:28 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/2190376624' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-07T07:31:28.428 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:28 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:28.225+0000 7fa1462ae200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-07T07:31:28.951 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 2024-03-07T07:31:28.951 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout { 2024-03-07T07:31:28.951 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "fsid": "826f6582-dc54-11ee-95c6-87774f69a715", 2024-03-07T07:31:28.951 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "health": { 2024-03-07T07:31:28.951 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-07T07:31:28.952 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-07T07:31:28.952 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-07T07:31:28.952 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-03-07T07:31:28.952 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-07T07:31:28.952 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-07T07:31:28.952 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 0 2024-03-07T07:31:28.952 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout ], 2024-03-07T07:31:28.952 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-07T07:31:28.952 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "a" 2024-03-07T07:31:28.952 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout ], 2024-03-07T07:31:28.952 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-03-07T07:31:28.952 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-07T07:31:28.952 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-07T07:31:28.953 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-07T07:31:28.953 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-07T07:31:28.953 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-03-07T07:31:28.953 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-07T07:31:28.953 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-07T07:31:28.953 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-07T07:31:28.953 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-07T07:31:28.953 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-07T07:31:28.961 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-07T07:31:28.961 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-07T07:31:28.961 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-07T07:31:28.961 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-03-07T07:31:28.961 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-07T07:31:28.961 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-07T07:31:28.961 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-07T07:31:28.962 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-07T07:31:28.962 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-07T07:31:28.962 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-07T07:31:28.962 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-07T07:31:28.962 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-07T07:31:28.962 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-07T07:31:28.962 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-03-07T07:31:28.962 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-07T07:31:28.962 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-07T07:31:28.962 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-07T07:31:28.962 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-07T07:31:28.962 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-03-07T07:31:28.962 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-07T07:31:28.962 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-07T07:31:28.963 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-07T07:31:28.963 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-07T07:31:28.963 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-07T07:31:28.963 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-07T07:31:28.963 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "restful" 2024-03-07T07:31:28.963 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout ], 2024-03-07T07:31:28.963 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-07T07:31:28.963 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-03-07T07:31:28.963 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-07T07:31:28.963 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-07T07:31:28.963 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "modified": "2024-03-07T07:31:14.230017+0000", 2024-03-07T07:31:28.963 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-07T07:31:28.963 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-03-07T07:31:28.963 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-07T07:31:28.964 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout } 2024-03-07T07:31:28.964 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:28.075+0000 7f05070d2700 1 Processor -- start 2024-03-07T07:31:28.964 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:28.075+0000 7f05070d2700 1 -- start start 2024-03-07T07:31:28.964 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:28.075+0000 7f05070d2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0500106a00 0x7f0500106e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:28.964 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:28.075+0000 7f05070d2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05001073f0 con 0x7f0500106a00 2024-03-07T07:31:28.964 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:28.075+0000 7f0504e6e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0500106a00 0x7f0500106e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:28.964 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:28.075+0000 7f0504e6e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0500106a00 0x7f0500106e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59996/0 (socket says 172.21.15.160:59996) 2024-03-07T07:31:28.964 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:28.075+0000 7f0504e6e700 1 -- 172.21.15.160:0/2660737289 learned_addr learned my addr 172.21.15.160:0/2660737289 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:31:28.964 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:28.076+0000 7f0504e6e700 1 -- 172.21.15.160:0/2660737289 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0500107c00 con 0x7f0500106a00 2024-03-07T07:31:28.964 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:28.076+0000 7f0504e6e700 1 --2- 172.21.15.160:0/2660737289 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0500106a00 0x7f0500106e20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f04f40098a0 tx=0x7f04f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=899a33e5af23c987 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:28.964 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:28.076+0000 7f04ff7fe700 1 -- 172.21.15.160:0/2660737289 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f04f4004160 con 0x7f0500106a00 2024-03-07T07:31:28.964 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:28.076+0000 7f04ff7fe700 1 -- 172.21.15.160:0/2660737289 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f04f4004300 con 0x7f0500106a00 2024-03-07T07:31:28.965 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:28.076+0000 7f04ff7fe700 1 -- 172.21.15.160:0/2660737289 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f04f4011b50 con 0x7f0500106a00 2024-03-07T07:31:28.965 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:28.077+0000 7f05070d2700 1 -- 172.21.15.160:0/2660737289 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0500106a00 msgr2=0x7f0500106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:28.965 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:28.077+0000 7f05070d2700 1 --2- 172.21.15.160:0/2660737289 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0500106a00 0x7f0500106e20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f04f40098a0 tx=0x7f04f4009bb0 comp rx=0 tx=0).stop 2024-03-07T07:31:28.965 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:28.077+0000 7f05070d2700 1 -- 172.21.15.160:0/2660737289 shutdown_connections 2024-03-07T07:31:28.965 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:28.077+0000 7f05070d2700 1 --2- 172.21.15.160:0/2660737289 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0500106a00 0x7f0500106e20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:28.965 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:28.077+0000 7f05070d2700 1 -- 172.21.15.160:0/2660737289 >> 172.21.15.160:0/2660737289 conn(0x7f0500101fc0 msgr2=0x7f05001043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:31:28.965 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:28.077+0000 7f05070d2700 1 -- 172.21.15.160:0/2660737289 shutdown_connections 2024-03-07T07:31:28.965 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:28.077+0000 7f05070d2700 1 -- 172.21.15.160:0/2660737289 wait complete. 2024-03-07T07:31:28.965 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:28.077+0000 7f05070d2700 1 Processor -- start 2024-03-07T07:31:28.965 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:28.078+0000 7f05070d2700 1 -- start start 2024-03-07T07:31:28.965 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:28.078+0000 7f05070d2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0500106a00 0x7f050019bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:28.965 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:28.078+0000 7f05070d2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050007afb0 con 0x7f0500106a00 2024-03-07T07:31:28.965 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:28.078+0000 7f0504e6e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0500106a00 0x7f050019bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:28.966 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:28.078+0000 7f0504e6e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0500106a00 0x7f050019bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:60008/0 (socket says 172.21.15.160:60008) 2024-03-07T07:31:28.966 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:28.078+0000 7f0504e6e700 1 -- 172.21.15.160:0/2190376624 learned_addr learned my addr 172.21.15.160:0/2190376624 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:31:28.966 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:28.078+0000 7f0504e6e700 1 -- 172.21.15.160:0/2190376624 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f050019c0b0 con 0x7f0500106a00 2024-03-07T07:31:28.966 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:28.078+0000 7f0504e6e700 1 --2- 172.21.15.160:0/2190376624 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0500106a00 0x7f050019bb70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f04f4009fb0 tx=0x7f04f4011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:28.966 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:28.078+0000 7f04fdffb700 1 -- 172.21.15.160:0/2190376624 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f04f40038b0 con 0x7f0500106a00 2024-03-07T07:31:28.966 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:28.078+0000 7f04fdffb700 1 -- 172.21.15.160:0/2190376624 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f04f4011e50 con 0x7f0500106a00 2024-03-07T07:31:28.966 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:28.078+0000 7f04fdffb700 1 -- 172.21.15.160:0/2190376624 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f04f4019550 con 0x7f0500106a00 2024-03-07T07:31:28.966 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:28.078+0000 7f05070d2700 1 -- 172.21.15.160:0/2190376624 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f050019c340 con 0x7f0500106a00 2024-03-07T07:31:28.966 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:28.079+0000 7f05070d2700 1 -- 172.21.15.160:0/2190376624 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f050019f030 con 0x7f0500106a00 2024-03-07T07:31:28.966 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:28.079+0000 7f04fdffb700 1 -- 172.21.15.160:0/2190376624 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f04f4019be0 con 0x7f0500106a00 2024-03-07T07:31:28.966 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:28.079+0000 7f04fdffb700 1 -- 172.21.15.160:0/2190376624 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f04f4024730 con 0x7f0500106a00 2024-03-07T07:31:28.966 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:28.079+0000 7f05070d2700 1 -- 172.21.15.160:0/2190376624 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0500063780 con 0x7f0500106a00 2024-03-07T07:31:28.966 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:28.082+0000 7f04fdffb700 1 -- 172.21.15.160:0/2190376624 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f04f4003a50 con 0x7f0500106a00 2024-03-07T07:31:28.966 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:28.155+0000 7f05070d2700 1 -- 172.21.15.160:0/2190376624 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f05000008d0 con 0x7f0500106a00 2024-03-07T07:31:28.967 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:28.155+0000 7f04fdffb700 1 -- 172.21.15.160:0/2190376624 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f04f40249d0 con 0x7f0500106a00 2024-03-07T07:31:28.967 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:28.156+0000 7f05070d2700 1 -- 172.21.15.160:0/2190376624 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0500106a00 msgr2=0x7f050019bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:28.967 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:28.156+0000 7f05070d2700 1 --2- 172.21.15.160:0/2190376624 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0500106a00 0x7f050019bb70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f04f4009fb0 tx=0x7f04f4011bf0 comp rx=0 tx=0).stop 2024-03-07T07:31:28.967 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:28.156+0000 7f05070d2700 1 -- 172.21.15.160:0/2190376624 shutdown_connections 2024-03-07T07:31:28.967 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:28.156+0000 7f05070d2700 1 --2- 172.21.15.160:0/2190376624 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0500106a00 0x7f050019bb70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:28.967 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:28.156+0000 7f05070d2700 1 -- 172.21.15.160:0/2190376624 >> 172.21.15.160:0/2190376624 conn(0x7f0500101fc0 msgr2=0x7f0500102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:31:28.967 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:28.156+0000 7f05070d2700 1 -- 172.21.15.160:0/2190376624 shutdown_connections 2024-03-07T07:31:28.967 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:28.156+0000 7f05070d2700 1 -- 172.21.15.160:0/2190376624 wait complete. 2024-03-07T07:31:28.967 INFO:teuthology.orchestra.run.smithi160.stdout:mgr not available, waiting (3/15)... 2024-03-07T07:31:29.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:29 smithi160 ceph-mon[31963]: Activating manager daemon x 2024-03-07T07:31:29.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:29 smithi160 ceph-mon[31963]: mgrmap e2: x(active, starting, since 0.00324137s) 2024-03-07T07:31:29.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:29 smithi160 ceph-mon[31963]: from='mgr.14100 172.21.15.160:0/3392081505' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-07T07:31:29.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:29 smithi160 ceph-mon[31963]: from='mgr.14100 172.21.15.160:0/3392081505' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-07T07:31:29.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:29 smithi160 ceph-mon[31963]: from='mgr.14100 172.21.15.160:0/3392081505' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-07T07:31:29.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:29 smithi160 ceph-mon[31963]: from='mgr.14100 172.21.15.160:0/3392081505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-07T07:31:29.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:29 smithi160 ceph-mon[31963]: from='mgr.14100 172.21.15.160:0/3392081505' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-07T07:31:29.679 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:29 smithi160 ceph-mon[31963]: Manager daemon x is now available 2024-03-07T07:31:29.679 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:29 smithi160 ceph-mon[31963]: from='mgr.14100 172.21.15.160:0/3392081505' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-07T07:31:29.679 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:29 smithi160 ceph-mon[31963]: from='mgr.14100 172.21.15.160:0/3392081505' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-07T07:31:29.679 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:29 smithi160 ceph-mon[31963]: from='mgr.14100 172.21.15.160:0/3392081505' entity='mgr.x' 2024-03-07T07:31:29.679 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:29 smithi160 ceph-mon[31963]: from='mgr.14100 172.21.15.160:0/3392081505' entity='mgr.x' 2024-03-07T07:31:29.679 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:29 smithi160 ceph-mon[31963]: from='mgr.14100 172.21.15.160:0/3392081505' entity='mgr.x' 2024-03-07T07:31:30.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:30 smithi160 ceph-mon[31963]: mgrmap e3: x(active, since 1.00936s) 2024-03-07T07:31:32.157 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 2024-03-07T07:31:32.157 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout { 2024-03-07T07:31:32.157 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "fsid": "826f6582-dc54-11ee-95c6-87774f69a715", 2024-03-07T07:31:32.157 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "health": { 2024-03-07T07:31:32.157 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-07T07:31:32.157 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-07T07:31:32.157 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-07T07:31:32.157 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-03-07T07:31:32.157 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-07T07:31:32.158 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-07T07:31:32.158 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 0 2024-03-07T07:31:32.158 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout ], 2024-03-07T07:31:32.158 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-07T07:31:32.158 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "a" 2024-03-07T07:31:32.158 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout ], 2024-03-07T07:31:32.158 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-03-07T07:31:32.158 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-07T07:31:32.158 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-07T07:31:32.158 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-07T07:31:32.158 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-07T07:31:32.158 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-03-07T07:31:32.159 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-07T07:31:32.159 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-07T07:31:32.159 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-07T07:31:32.159 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-07T07:31:32.159 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-07T07:31:32.169 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-07T07:31:32.169 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-07T07:31:32.170 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-07T07:31:32.170 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-03-07T07:31:32.170 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-07T07:31:32.170 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-07T07:31:32.170 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-07T07:31:32.170 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-07T07:31:32.170 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-07T07:31:32.170 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-07T07:31:32.170 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-07T07:31:32.170 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-07T07:31:32.170 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-07T07:31:32.171 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-03-07T07:31:32.171 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-07T07:31:32.171 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-07T07:31:32.171 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-07T07:31:32.171 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-07T07:31:32.171 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-03-07T07:31:32.171 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-07T07:31:32.171 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-07T07:31:32.171 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-07T07:31:32.171 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-07T07:31:32.171 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-07T07:31:32.171 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-07T07:31:32.172 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "restful" 2024-03-07T07:31:32.172 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout ], 2024-03-07T07:31:32.172 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-07T07:31:32.172 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-03-07T07:31:32.172 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-07T07:31:32.172 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-07T07:31:32.172 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "modified": "2024-03-07T07:31:14.230017+0000", 2024-03-07T07:31:32.172 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-07T07:31:32.172 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout }, 2024-03-07T07:31:32.172 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-07T07:31:32.172 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout } 2024-03-07T07:31:32.172 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.368+0000 7fb47c0fb700 1 Processor -- start 2024-03-07T07:31:32.172 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.368+0000 7fb47c0fb700 1 -- start start 2024-03-07T07:31:32.173 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.368+0000 7fb47c0fb700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4741085a0 0x7fb47410a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:32.173 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.368+0000 7fb47c0fb700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb47407be70 con 0x7fb4741085a0 2024-03-07T07:31:32.173 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.368+0000 7fb479e97700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4741085a0 0x7fb47410a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:32.173 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.368+0000 7fb479e97700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4741085a0 0x7fb47410a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41228/0 (socket says 172.21.15.160:41228) 2024-03-07T07:31:32.173 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.369+0000 7fb479e97700 1 -- 172.21.15.160:0/4162459541 learned_addr learned my addr 172.21.15.160:0/4162459541 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:31:32.173 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.369+0000 7fb479e97700 1 -- 172.21.15.160:0/4162459541 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb47410aed0 con 0x7fb4741085a0 2024-03-07T07:31:32.173 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.369+0000 7fb479e97700 1 --2- 172.21.15.160:0/4162459541 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4741085a0 0x7fb47410a990 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb4640098a0 tx=0x7fb464009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74bcdbb5e3076499 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:32.173 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.369+0000 7fb478e95700 1 -- 172.21.15.160:0/4162459541 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb464004160 con 0x7fb4741085a0 2024-03-07T07:31:32.173 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.369+0000 7fb478e95700 1 -- 172.21.15.160:0/4162459541 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb464004300 con 0x7fb4741085a0 2024-03-07T07:31:32.173 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.369+0000 7fb478e95700 1 -- 172.21.15.160:0/4162459541 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb464011b50 con 0x7fb4741085a0 2024-03-07T07:31:32.173 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.370+0000 7fb47c0fb700 1 -- 172.21.15.160:0/4162459541 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4741085a0 msgr2=0x7fb47410a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:32.173 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.370+0000 7fb47c0fb700 1 --2- 172.21.15.160:0/4162459541 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4741085a0 0x7fb47410a990 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb4640098a0 tx=0x7fb464009bb0 comp rx=0 tx=0).stop 2024-03-07T07:31:32.173 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.370+0000 7fb47c0fb700 1 -- 172.21.15.160:0/4162459541 shutdown_connections 2024-03-07T07:31:32.173 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.370+0000 7fb47c0fb700 1 --2- 172.21.15.160:0/4162459541 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4741085a0 0x7fb47410a990 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:32.174 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.370+0000 7fb47c0fb700 1 -- 172.21.15.160:0/4162459541 >> 172.21.15.160:0/4162459541 conn(0x7fb474101ff0 msgr2=0x7fb474104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:31:32.174 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.370+0000 7fb47c0fb700 1 -- 172.21.15.160:0/4162459541 shutdown_connections 2024-03-07T07:31:32.174 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.370+0000 7fb47c0fb700 1 -- 172.21.15.160:0/4162459541 wait complete. 2024-03-07T07:31:32.174 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.370+0000 7fb47c0fb700 1 Processor -- start 2024-03-07T07:31:32.174 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.370+0000 7fb47c0fb700 1 -- start start 2024-03-07T07:31:32.174 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.371+0000 7fb47c0fb700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4741085a0 0x7fb47419bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:32.174 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.371+0000 7fb47c0fb700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb47410be80 con 0x7fb4741085a0 2024-03-07T07:31:32.174 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.371+0000 7fb479e97700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4741085a0 0x7fb47419bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:32.174 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.371+0000 7fb479e97700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4741085a0 0x7fb47419bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41244/0 (socket says 172.21.15.160:41244) 2024-03-07T07:31:32.174 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.371+0000 7fb479e97700 1 -- 172.21.15.160:0/717859015 learned_addr learned my addr 172.21.15.160:0/717859015 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:31:32.174 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.371+0000 7fb479e97700 1 -- 172.21.15.160:0/717859015 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb47419c010 con 0x7fb4741085a0 2024-03-07T07:31:32.174 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.371+0000 7fb479e97700 1 --2- 172.21.15.160:0/717859015 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4741085a0 0x7fb47419bad0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb464011b60 tx=0x7fb464011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:32.174 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.371+0000 7fb46affd700 1 -- 172.21.15.160:0/717859015 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb464003970 con 0x7fb4741085a0 2024-03-07T07:31:32.175 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.371+0000 7fb46affd700 1 -- 172.21.15.160:0/717859015 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb464003b10 con 0x7fb4741085a0 2024-03-07T07:31:32.175 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.371+0000 7fb47c0fb700 1 -- 172.21.15.160:0/717859015 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb47419c300 con 0x7fb4741085a0 2024-03-07T07:31:32.175 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.371+0000 7fb46affd700 1 -- 172.21.15.160:0/717859015 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb464024610 con 0x7fb4741085a0 2024-03-07T07:31:32.175 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.371+0000 7fb47c0fb700 1 -- 172.21.15.160:0/717859015 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb47419ef90 con 0x7fb4741085a0 2024-03-07T07:31:32.175 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.372+0000 7fb46affd700 1 -- 172.21.15.160:0/717859015 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 4) v1 ==== 45916+0+0 (secure 0 0 0) 0x7fb4640248c0 con 0x7fb4741085a0 2024-03-07T07:31:32.175 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.372+0000 7fb47c0fb700 1 -- 172.21.15.160:0/717859015 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb474063780 con 0x7fb4741085a0 2024-03-07T07:31:32.175 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.372+0000 7fb46affd700 1 --2- 172.21.15.160:0/717859015 >> [v2:172.21.15.160:6800/1130675225,v1:172.21.15.160:6801/1130675225] conn(0x7fb460038ba0 0x7fb46003b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:32.175 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.372+0000 7fb46affd700 1 -- 172.21.15.160:0/717859015 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb464011e00 con 0x7fb4741085a0 2024-03-07T07:31:32.175 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.372+0000 7fb479696700 1 --2- 172.21.15.160:0/717859015 >> [v2:172.21.15.160:6800/1130675225,v1:172.21.15.160:6801/1130675225] conn(0x7fb460038ba0 0x7fb46003b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:32.175 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.373+0000 7fb479696700 1 --2- 172.21.15.160:0/717859015 >> [v2:172.21.15.160:6800/1130675225,v1:172.21.15.160:6801/1130675225] conn(0x7fb460038ba0 0x7fb46003b060 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb470006fd0 tx=0x7fb470006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:32.175 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.377+0000 7fb46affd700 1 -- 172.21.15.160:0/717859015 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb46404c5c0 con 0x7fb4741085a0 2024-03-07T07:31:32.175 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.648+0000 7fb47c0fb700 1 -- 172.21.15.160:0/717859015 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fb47419f650 con 0x7fb4741085a0 2024-03-07T07:31:32.175 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.648+0000 7fb46affd700 1 -- 172.21.15.160:0/717859015 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fb46402d910 con 0x7fb4741085a0 2024-03-07T07:31:32.175 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.650+0000 7fb47c0fb700 1 -- 172.21.15.160:0/717859015 >> [v2:172.21.15.160:6800/1130675225,v1:172.21.15.160:6801/1130675225] conn(0x7fb460038ba0 msgr2=0x7fb46003b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:32.176 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.650+0000 7fb47c0fb700 1 --2- 172.21.15.160:0/717859015 >> [v2:172.21.15.160:6800/1130675225,v1:172.21.15.160:6801/1130675225] conn(0x7fb460038ba0 0x7fb46003b060 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb470006fd0 tx=0x7fb470006e40 comp rx=0 tx=0).stop 2024-03-07T07:31:32.176 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.650+0000 7fb47c0fb700 1 -- 172.21.15.160:0/717859015 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4741085a0 msgr2=0x7fb47419bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:32.176 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.650+0000 7fb47c0fb700 1 --2- 172.21.15.160:0/717859015 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4741085a0 0x7fb47419bad0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb464011b60 tx=0x7fb464011c40 comp rx=0 tx=0).stop 2024-03-07T07:31:32.176 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.650+0000 7fb47c0fb700 1 -- 172.21.15.160:0/717859015 shutdown_connections 2024-03-07T07:31:32.176 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.650+0000 7fb47c0fb700 1 --2- 172.21.15.160:0/717859015 >> [v2:172.21.15.160:6800/1130675225,v1:172.21.15.160:6801/1130675225] conn(0x7fb460038ba0 0x7fb46003b060 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:32.176 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.650+0000 7fb47c0fb700 1 --2- 172.21.15.160:0/717859015 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4741085a0 0x7fb47419bad0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:32.176 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.650+0000 7fb47c0fb700 1 -- 172.21.15.160:0/717859015 >> 172.21.15.160:0/717859015 conn(0x7fb474101ff0 msgr2=0x7fb474102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:31:32.176 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.650+0000 7fb47c0fb700 1 -- 172.21.15.160:0/717859015 shutdown_connections 2024-03-07T07:31:32.176 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:31.650+0000 7fb47c0fb700 1 -- 172.21.15.160:0/717859015 wait complete. 2024-03-07T07:31:32.176 INFO:teuthology.orchestra.run.smithi160.stdout:mgr is available 2024-03-07T07:31:32.387 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:32 smithi160 ceph-mon[31963]: mgrmap e4: x(active, since 3s) 2024-03-07T07:31:32.388 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:32 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/717859015' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-07T07:31:33.350 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:33 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/464719591' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-07T07:31:33.359 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 2024-03-07T07:31:33.359 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout [global] 2024-03-07T07:31:33.359 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout fsid = 826f6582-dc54-11ee-95c6-87774f69a715 2024-03-07T07:31:33.359 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-07T07:31:33.359 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-07T07:31:33.359 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-07T07:31:33.359 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-07T07:31:33.359 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 2024-03-07T07:31:33.360 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-07T07:31:33.360 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-07T07:31:33.360 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 2024-03-07T07:31:33.360 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout [osd] 2024-03-07T07:31:33.360 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-07T07:31:33.360 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-07T07:31:33.360 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-07T07:31:33.360 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-07T07:31:33.360 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-07T07:31:33.360 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.578+0000 7f81fef2b700 1 Processor -- start 2024-03-07T07:31:33.360 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.578+0000 7f81fef2b700 1 -- start start 2024-03-07T07:31:33.360 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.578+0000 7f81fef2b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81f8107b10 0x7f81f8107f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:33.360 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.578+0000 7f81fef2b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f8108500 con 0x7f81f8107b10 2024-03-07T07:31:33.361 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.578+0000 7f81fccc7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81f8107b10 0x7f81f8107f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:33.361 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.578+0000 7f81fccc7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81f8107b10 0x7f81f8107f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41246/0 (socket says 172.21.15.160:41246) 2024-03-07T07:31:33.361 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.578+0000 7f81fccc7700 1 -- 172.21.15.160:0/3624175127 learned_addr learned my addr 172.21.15.160:0/3624175127 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:31:33.361 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.578+0000 7f81fccc7700 1 -- 172.21.15.160:0/3624175127 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f8108d10 con 0x7f81f8107b10 2024-03-07T07:31:33.361 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.579+0000 7f81fccc7700 1 --2- 172.21.15.160:0/3624175127 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81f8107b10 0x7f81f8107f30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f81ec0098a0 tx=0x7f81ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d6420d49bc09780 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:33.361 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.579+0000 7f81f77fe700 1 -- 172.21.15.160:0/3624175127 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f81ec004160 con 0x7f81f8107b10 2024-03-07T07:31:33.361 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.579+0000 7f81f77fe700 1 -- 172.21.15.160:0/3624175127 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f81ec004300 con 0x7f81f8107b10 2024-03-07T07:31:33.361 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.579+0000 7f81f77fe700 1 -- 172.21.15.160:0/3624175127 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f81ec011b50 con 0x7f81f8107b10 2024-03-07T07:31:33.361 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.579+0000 7f81fef2b700 1 -- 172.21.15.160:0/3624175127 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81f8107b10 msgr2=0x7f81f8107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:33.361 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.579+0000 7f81fef2b700 1 --2- 172.21.15.160:0/3624175127 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81f8107b10 0x7f81f8107f30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f81ec0098a0 tx=0x7f81ec009bb0 comp rx=0 tx=0).stop 2024-03-07T07:31:33.361 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.579+0000 7f81fef2b700 1 -- 172.21.15.160:0/3624175127 shutdown_connections 2024-03-07T07:31:33.361 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.579+0000 7f81fef2b700 1 --2- 172.21.15.160:0/3624175127 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81f8107b10 0x7f81f8107f30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:33.361 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.579+0000 7f81fef2b700 1 -- 172.21.15.160:0/3624175127 >> 172.21.15.160:0/3624175127 conn(0x7f81f8103110 msgr2=0x7f81f8105530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:31:33.361 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.579+0000 7f81fef2b700 1 -- 172.21.15.160:0/3624175127 shutdown_connections 2024-03-07T07:31:33.362 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.579+0000 7f81fef2b700 1 -- 172.21.15.160:0/3624175127 wait complete. 2024-03-07T07:31:33.362 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.580+0000 7f81fef2b700 1 Processor -- start 2024-03-07T07:31:33.362 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.580+0000 7f81fef2b700 1 -- start start 2024-03-07T07:31:33.362 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.580+0000 7f81fef2b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81f8107b10 0x7f81f807db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:33.362 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.580+0000 7f81fef2b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f810c0c0 con 0x7f81f8107b10 2024-03-07T07:31:33.362 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.580+0000 7f81fccc7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81f8107b10 0x7f81f807db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:33.362 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.580+0000 7f81fccc7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81f8107b10 0x7f81f807db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41262/0 (socket says 172.21.15.160:41262) 2024-03-07T07:31:33.362 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.580+0000 7f81fccc7700 1 -- 172.21.15.160:0/464719591 learned_addr learned my addr 172.21.15.160:0/464719591 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:31:33.363 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.581+0000 7f81fccc7700 1 -- 172.21.15.160:0/464719591 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f807e090 con 0x7f81f8107b10 2024-03-07T07:31:33.363 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.581+0000 7f81fccc7700 1 --2- 172.21.15.160:0/464719591 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81f8107b10 0x7f81f807db50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f81ec009fb0 tx=0x7f81ec011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:33.363 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.581+0000 7f81f5ffb700 1 -- 172.21.15.160:0/464719591 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f81ec0038b0 con 0x7f81f8107b10 2024-03-07T07:31:33.363 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.581+0000 7f81f5ffb700 1 -- 172.21.15.160:0/464719591 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f81ec011e50 con 0x7f81f8107b10 2024-03-07T07:31:33.363 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.581+0000 7f81fef2b700 1 -- 172.21.15.160:0/464719591 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81f807c280 con 0x7f81f8107b10 2024-03-07T07:31:33.363 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.581+0000 7f81f5ffb700 1 -- 172.21.15.160:0/464719591 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f81ec019550 con 0x7f81f8107b10 2024-03-07T07:31:33.363 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.581+0000 7f81fef2b700 1 -- 172.21.15.160:0/464719591 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81f807c7c0 con 0x7f81f8107b10 2024-03-07T07:31:33.363 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.582+0000 7f81f5ffb700 1 -- 172.21.15.160:0/464719591 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 4) v1 ==== 45916+0+0 (secure 0 0 0) 0x7f81ec019850 con 0x7f81f8107b10 2024-03-07T07:31:33.363 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.582+0000 7f81f5ffb700 1 --2- 172.21.15.160:0/464719591 >> [v2:172.21.15.160:6800/1130675225,v1:172.21.15.160:6801/1130675225] conn(0x7f81e8038b00 0x7f81e803afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:33.363 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.582+0000 7f81fef2b700 1 -- 172.21.15.160:0/464719591 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81f8063780 con 0x7f81f8107b10 2024-03-07T07:31:33.363 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.582+0000 7f81f5ffb700 1 -- 172.21.15.160:0/464719591 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f81ec04cc60 con 0x7f81f8107b10 2024-03-07T07:31:33.363 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.582+0000 7f81f7fff700 1 --2- 172.21.15.160:0/464719591 >> [v2:172.21.15.160:6800/1130675225,v1:172.21.15.160:6801/1130675225] conn(0x7f81e8038b00 0x7f81e803afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:33.363 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.582+0000 7f81f7fff700 1 --2- 172.21.15.160:0/464719591 >> [v2:172.21.15.160:6800/1130675225,v1:172.21.15.160:6801/1130675225] conn(0x7f81e8038b00 0x7f81e803afc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f81e4006fd0 tx=0x7f81e4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:33.364 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.587+0000 7f81f5ffb700 1 -- 172.21.15.160:0/464719591 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f81ec04c680 con 0x7f81f8107b10 2024-03-07T07:31:33.364 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.780+0000 7f81fef2b700 1 -- 172.21.15.160:0/464719591 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f81f807d0c0 con 0x7f81f8107b10 2024-03-07T07:31:33.364 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.781+0000 7f81f5ffb700 1 -- 172.21.15.160:0/464719591 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f81ec02d960 con 0x7f81f8107b10 2024-03-07T07:31:33.364 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.783+0000 7f81fef2b700 1 -- 172.21.15.160:0/464719591 >> [v2:172.21.15.160:6800/1130675225,v1:172.21.15.160:6801/1130675225] conn(0x7f81e8038b00 msgr2=0x7f81e803afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:33.364 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.783+0000 7f81fef2b700 1 --2- 172.21.15.160:0/464719591 >> [v2:172.21.15.160:6800/1130675225,v1:172.21.15.160:6801/1130675225] conn(0x7f81e8038b00 0x7f81e803afc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f81e4006fd0 tx=0x7f81e4006e40 comp rx=0 tx=0).stop 2024-03-07T07:31:33.364 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.783+0000 7f81fef2b700 1 -- 172.21.15.160:0/464719591 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81f8107b10 msgr2=0x7f81f807db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:33.364 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.783+0000 7f81fef2b700 1 --2- 172.21.15.160:0/464719591 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81f8107b10 0x7f81f807db50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f81ec009fb0 tx=0x7f81ec011bf0 comp rx=0 tx=0).stop 2024-03-07T07:31:33.364 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.783+0000 7f81fef2b700 1 -- 172.21.15.160:0/464719591 shutdown_connections 2024-03-07T07:31:33.364 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.783+0000 7f81fef2b700 1 --2- 172.21.15.160:0/464719591 >> [v2:172.21.15.160:6800/1130675225,v1:172.21.15.160:6801/1130675225] conn(0x7f81e8038b00 0x7f81e803afc0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:33.364 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.783+0000 7f81fef2b700 1 --2- 172.21.15.160:0/464719591 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81f8107b10 0x7f81f807db50 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:33.364 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.783+0000 7f81fef2b700 1 -- 172.21.15.160:0/464719591 >> 172.21.15.160:0/464719591 conn(0x7f81f8103110 msgr2=0x7f81f8103ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:31:33.364 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.784+0000 7f81fef2b700 1 -- 172.21.15.160:0/464719591 shutdown_connections 2024-03-07T07:31:33.364 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:32.784+0000 7f81fef2b700 1 -- 172.21.15.160:0/464719591 wait complete. 2024-03-07T07:31:33.365 INFO:teuthology.orchestra.run.smithi160.stdout:Enabling cephadm module... 2024-03-07T07:31:34.303 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:34 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/2255022429' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-07T07:31:34.564 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:34 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: ignoring --setuser ceph since I am not root 2024-03-07T07:31:34.564 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:34 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: ignoring --setgroup ceph since I am not root 2024-03-07T07:31:34.872 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:33.768+0000 7f95e1fd7700 1 Processor -- start 2024-03-07T07:31:34.872 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:33.768+0000 7f95e1fd7700 1 -- start start 2024-03-07T07:31:34.872 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:33.768+0000 7f95e1fd7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95dc107fc0 0x7f95dc1083e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:34.873 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:33.768+0000 7f95e1fd7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95dc1089b0 con 0x7f95dc107fc0 2024-03-07T07:31:34.873 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:33.768+0000 7f95db7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95dc107fc0 0x7f95dc1083e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:34.873 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:33.769+0000 7f95db7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95dc107fc0 0x7f95dc1083e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41276/0 (socket says 172.21.15.160:41276) 2024-03-07T07:31:34.873 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:33.769+0000 7f95db7fe700 1 -- 172.21.15.160:0/2849763231 learned_addr learned my addr 172.21.15.160:0/2849763231 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:31:34.873 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:33.769+0000 7f95db7fe700 1 -- 172.21.15.160:0/2849763231 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95dc106c20 con 0x7f95dc107fc0 2024-03-07T07:31:34.873 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:33.769+0000 7f95db7fe700 1 --2- 172.21.15.160:0/2849763231 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95dc107fc0 0x7f95dc1083e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f95cc0098a0 tx=0x7f95cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=577d669ff5c4ea97 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:34.873 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:33.769+0000 7f95daffd700 1 -- 172.21.15.160:0/2849763231 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f95cc004160 con 0x7f95dc107fc0 2024-03-07T07:31:34.873 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:33.769+0000 7f95daffd700 1 -- 172.21.15.160:0/2849763231 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f95cc004300 con 0x7f95dc107fc0 2024-03-07T07:31:34.873 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:33.769+0000 7f95daffd700 1 -- 172.21.15.160:0/2849763231 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f95cc011b50 con 0x7f95dc107fc0 2024-03-07T07:31:34.873 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:33.770+0000 7f95e1fd7700 1 -- 172.21.15.160:0/2849763231 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95dc107fc0 msgr2=0x7f95dc1083e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:34.873 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:33.770+0000 7f95e1fd7700 1 --2- 172.21.15.160:0/2849763231 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95dc107fc0 0x7f95dc1083e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f95cc0098a0 tx=0x7f95cc009bb0 comp rx=0 tx=0).stop 2024-03-07T07:31:34.873 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:33.770+0000 7f95e1fd7700 1 -- 172.21.15.160:0/2849763231 shutdown_connections 2024-03-07T07:31:34.873 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:33.770+0000 7f95e1fd7700 1 --2- 172.21.15.160:0/2849763231 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95dc107fc0 0x7f95dc1083e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:34.874 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:33.770+0000 7f95e1fd7700 1 -- 172.21.15.160:0/2849763231 >> 172.21.15.160:0/2849763231 conn(0x7f95dc101790 msgr2=0x7f95dc103bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:31:34.874 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:33.770+0000 7f95e1fd7700 1 -- 172.21.15.160:0/2849763231 shutdown_connections 2024-03-07T07:31:34.874 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:33.770+0000 7f95e1fd7700 1 -- 172.21.15.160:0/2849763231 wait complete. 2024-03-07T07:31:34.874 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:33.770+0000 7f95e1fd7700 1 Processor -- start 2024-03-07T07:31:34.874 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:33.770+0000 7f95e1fd7700 1 -- start start 2024-03-07T07:31:34.874 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:33.771+0000 7f95e1fd7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95dc107fc0 0x7f95dc19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:34.874 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:33.771+0000 7f95e1fd7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95dc10afe0 con 0x7f95dc107fc0 2024-03-07T07:31:34.874 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:33.771+0000 7f95db7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95dc107fc0 0x7f95dc19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:34.874 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:33.771+0000 7f95db7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95dc107fc0 0x7f95dc19aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41292/0 (socket says 172.21.15.160:41292) 2024-03-07T07:31:34.874 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:33.771+0000 7f95db7fe700 1 -- 172.21.15.160:0/2255022429 learned_addr learned my addr 172.21.15.160:0/2255022429 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:31:34.874 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:33.771+0000 7f95db7fe700 1 -- 172.21.15.160:0/2255022429 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95dc19af70 con 0x7f95dc107fc0 2024-03-07T07:31:34.874 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:33.771+0000 7f95db7fe700 1 --2- 172.21.15.160:0/2255022429 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95dc107fc0 0x7f95dc19aa30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f95cc0046e0 tx=0x7f95cc004260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:34.874 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:33.771+0000 7f95d97fa700 1 -- 172.21.15.160:0/2255022429 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f95cc003a90 con 0x7f95dc107fc0 2024-03-07T07:31:34.875 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:33.771+0000 7f95d97fa700 1 -- 172.21.15.160:0/2255022429 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f95cc003c30 con 0x7f95dc107fc0 2024-03-07T07:31:34.875 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:33.771+0000 7f95d97fa700 1 -- 172.21.15.160:0/2255022429 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f95cc01c8a0 con 0x7f95dc107fc0 2024-03-07T07:31:34.875 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:33.771+0000 7f95e1fd7700 1 -- 172.21.15.160:0/2255022429 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95dc19b200 con 0x7f95dc107fc0 2024-03-07T07:31:34.875 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:33.771+0000 7f95e1fd7700 1 -- 172.21.15.160:0/2255022429 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95dc105b10 con 0x7f95dc107fc0 2024-03-07T07:31:34.875 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:33.772+0000 7f95d97fa700 1 -- 172.21.15.160:0/2255022429 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 4) v1 ==== 45916+0+0 (secure 0 0 0) 0x7f95cc01b450 con 0x7f95dc107fc0 2024-03-07T07:31:34.875 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:33.772+0000 7f95e1fd7700 1 -- 172.21.15.160:0/2255022429 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95dc063780 con 0x7f95dc107fc0 2024-03-07T07:31:34.875 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:33.772+0000 7f95d97fa700 1 --2- 172.21.15.160:0/2255022429 >> [v2:172.21.15.160:6800/1130675225,v1:172.21.15.160:6801/1130675225] conn(0x7f95c8038ba0 0x7f95c803b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:34.875 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:33.772+0000 7f95d97fa700 1 -- 172.21.15.160:0/2255022429 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f95cc011e00 con 0x7f95dc107fc0 2024-03-07T07:31:34.875 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:33.772+0000 7f95d3fff700 1 --2- 172.21.15.160:0/2255022429 >> [v2:172.21.15.160:6800/1130675225,v1:172.21.15.160:6801/1130675225] conn(0x7f95c8038ba0 0x7f95c803b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:34.875 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:33.773+0000 7f95d3fff700 1 --2- 172.21.15.160:0/2255022429 >> [v2:172.21.15.160:6800/1130675225,v1:172.21.15.160:6801/1130675225] conn(0x7f95c8038ba0 0x7f95c803b060 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f95c4006fd0 tx=0x7f95c4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:34.875 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:33.777+0000 7f95d97fa700 1 -- 172.21.15.160:0/2255022429 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f95cc01a3d0 con 0x7f95dc107fc0 2024-03-07T07:31:34.875 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:34.030+0000 7f95e1fd7700 1 -- 172.21.15.160:0/2255022429 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f95dc106220 con 0x7f95dc107fc0 2024-03-07T07:31:34.875 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:34.250+0000 7f95d97fa700 1 -- 172.21.15.160:0/2255022429 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f95cc00f2e0 con 0x7f95dc107fc0 2024-03-07T07:31:34.876 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:34.251+0000 7f95d97fa700 1 -- 172.21.15.160:0/2255022429 <== mon.0 v2:172.21.15.160:3300/0 8 ==== mgrmap(e 5) v1 ==== 45927+0+0 (secure 0 0 0) 0x7f95cc0243d0 con 0x7f95dc107fc0 2024-03-07T07:31:34.876 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:34.253+0000 7f95e1fd7700 1 -- 172.21.15.160:0/2255022429 >> [v2:172.21.15.160:6800/1130675225,v1:172.21.15.160:6801/1130675225] conn(0x7f95c8038ba0 msgr2=0x7f95c803b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:34.876 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:34.253+0000 7f95e1fd7700 1 --2- 172.21.15.160:0/2255022429 >> [v2:172.21.15.160:6800/1130675225,v1:172.21.15.160:6801/1130675225] conn(0x7f95c8038ba0 0x7f95c803b060 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f95c4006fd0 tx=0x7f95c4006e40 comp rx=0 tx=0).stop 2024-03-07T07:31:34.876 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:34.253+0000 7f95e1fd7700 1 -- 172.21.15.160:0/2255022429 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95dc107fc0 msgr2=0x7f95dc19aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:34.876 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:34.253+0000 7f95e1fd7700 1 --2- 172.21.15.160:0/2255022429 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95dc107fc0 0x7f95dc19aa30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f95cc0046e0 tx=0x7f95cc004260 comp rx=0 tx=0).stop 2024-03-07T07:31:34.876 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:34.253+0000 7f95e1fd7700 1 -- 172.21.15.160:0/2255022429 shutdown_connections 2024-03-07T07:31:34.876 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:34.253+0000 7f95e1fd7700 1 --2- 172.21.15.160:0/2255022429 >> [v2:172.21.15.160:6800/1130675225,v1:172.21.15.160:6801/1130675225] conn(0x7f95c8038ba0 0x7f95c803b060 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:34.876 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:34.253+0000 7f95e1fd7700 1 --2- 172.21.15.160:0/2255022429 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95dc107fc0 0x7f95dc19aa30 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:34.876 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:34.253+0000 7f95e1fd7700 1 -- 172.21.15.160:0/2255022429 >> 172.21.15.160:0/2255022429 conn(0x7f95dc101790 msgr2=0x7f95dc1022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:31:34.876 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:34.253+0000 7f95e1fd7700 1 -- 172.21.15.160:0/2255022429 shutdown_connections 2024-03-07T07:31:34.876 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:34.253+0000 7f95e1fd7700 1 -- 172.21.15.160:0/2255022429 wait complete. 2024-03-07T07:31:35.137 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:34 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:34.825+0000 7f4e41a9b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-07T07:31:35.137 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:34 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:34.926+0000 7f4e41a9b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-07T07:31:35.427 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:35 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/2255022429' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-07T07:31:35.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:35 smithi160 ceph-mon[31963]: mgrmap e5: x(active, since 6s) 2024-03-07T07:31:35.837 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:35 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:35.588+0000 7f4e41a9b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-07T07:31:35.838 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:35 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:35.694+0000 7f4e41a9b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-07T07:31:35.838 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:35 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:35.837+0000 7f4e41a9b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-07T07:31:36.178 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:35 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:35.935+0000 7f4e41a9b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-07T07:31:36.178 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:36 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:36.020+0000 7f4e41a9b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-07T07:31:36.288 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout { 2024-03-07T07:31:36.288 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-07T07:31:36.288 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-07T07:31:36.288 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-07T07:31:36.288 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-07T07:31:36.288 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout } 2024-03-07T07:31:36.288 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.343+0000 7faa88669700 1 Processor -- start 2024-03-07T07:31:36.288 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.343+0000 7faa88669700 1 -- start start 2024-03-07T07:31:36.289 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.343+0000 7faa88669700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa801069b0 0x7faa80106dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:36.289 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.343+0000 7faa88669700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa801073a0 con 0x7faa801069b0 2024-03-07T07:31:36.289 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.343+0000 7faa86405700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa801069b0 0x7faa80106dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:36.289 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.344+0000 7faa86405700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa801069b0 0x7faa80106dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41314/0 (socket says 172.21.15.160:41314) 2024-03-07T07:31:36.289 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.344+0000 7faa86405700 1 -- 172.21.15.160:0/3110992340 learned_addr learned my addr 172.21.15.160:0/3110992340 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:31:36.289 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.344+0000 7faa86405700 1 -- 172.21.15.160:0/3110992340 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa80107bb0 con 0x7faa801069b0 2024-03-07T07:31:36.289 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.344+0000 7faa86405700 1 --2- 172.21.15.160:0/3110992340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa801069b0 0x7faa80106dd0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7faa7000bb40 tx=0x7faa7000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa81f5e8ca69cacb server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:36.289 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.344+0000 7faa85403700 1 -- 172.21.15.160:0/3110992340 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faa7000dbe0 con 0x7faa801069b0 2024-03-07T07:31:36.289 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.344+0000 7faa85403700 1 -- 172.21.15.160:0/3110992340 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa700044f0 con 0x7faa801069b0 2024-03-07T07:31:36.289 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.344+0000 7faa85403700 1 -- 172.21.15.160:0/3110992340 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faa70010470 con 0x7faa801069b0 2024-03-07T07:31:36.289 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.345+0000 7faa88669700 1 -- 172.21.15.160:0/3110992340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa801069b0 msgr2=0x7faa80106dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:36.289 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.345+0000 7faa88669700 1 --2- 172.21.15.160:0/3110992340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa801069b0 0x7faa80106dd0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7faa7000bb40 tx=0x7faa7000be50 comp rx=0 tx=0).stop 2024-03-07T07:31:36.289 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.345+0000 7faa88669700 1 -- 172.21.15.160:0/3110992340 shutdown_connections 2024-03-07T07:31:36.290 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.345+0000 7faa88669700 1 --2- 172.21.15.160:0/3110992340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa801069b0 0x7faa80106dd0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:36.290 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.345+0000 7faa88669700 1 -- 172.21.15.160:0/3110992340 >> 172.21.15.160:0/3110992340 conn(0x7faa80101f70 msgr2=0x7faa80104390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:31:36.290 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.345+0000 7faa88669700 1 -- 172.21.15.160:0/3110992340 shutdown_connections 2024-03-07T07:31:36.290 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.345+0000 7faa88669700 1 -- 172.21.15.160:0/3110992340 wait complete. 2024-03-07T07:31:36.290 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.345+0000 7faa88669700 1 Processor -- start 2024-03-07T07:31:36.290 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.346+0000 7faa88669700 1 -- start start 2024-03-07T07:31:36.290 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.346+0000 7faa88669700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa801069b0 0x7faa8019dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:36.290 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.346+0000 7faa88669700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa800749e0 con 0x7faa801069b0 2024-03-07T07:31:36.290 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.346+0000 7faa86405700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa801069b0 0x7faa8019dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:36.290 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.346+0000 7faa86405700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa801069b0 0x7faa8019dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41316/0 (socket says 172.21.15.160:41316) 2024-03-07T07:31:36.290 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.346+0000 7faa86405700 1 -- 172.21.15.160:0/2459221801 learned_addr learned my addr 172.21.15.160:0/2459221801 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:31:36.290 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.346+0000 7faa86405700 1 -- 172.21.15.160:0/2459221801 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa8019e280 con 0x7faa801069b0 2024-03-07T07:31:36.290 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.346+0000 7faa86405700 1 --2- 172.21.15.160:0/2459221801 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa801069b0 0x7faa8019dd40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7faa7000b340 tx=0x7faa700040b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:36.291 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.346+0000 7faa777fe700 1 -- 172.21.15.160:0/2459221801 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faa70010720 con 0x7faa801069b0 2024-03-07T07:31:36.291 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.346+0000 7faa88669700 1 -- 172.21.15.160:0/2459221801 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa8019e510 con 0x7faa801069b0 2024-03-07T07:31:36.291 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.346+0000 7faa777fe700 1 -- 172.21.15.160:0/2459221801 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa7001d070 con 0x7faa801069b0 2024-03-07T07:31:36.291 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.346+0000 7faa777fe700 1 -- 172.21.15.160:0/2459221801 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faa70021b80 con 0x7faa801069b0 2024-03-07T07:31:36.291 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.346+0000 7faa88669700 1 -- 172.21.15.160:0/2459221801 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa801a1200 con 0x7faa801069b0 2024-03-07T07:31:36.291 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.347+0000 7faa777fe700 1 -- 172.21.15.160:0/2459221801 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) v1 ==== 45927+0+0 (secure 0 0 0) 0x7faa7001f030 con 0x7faa801069b0 2024-03-07T07:31:36.291 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.347+0000 7faa88669700 1 -- 172.21.15.160:0/2459221801 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa80063780 con 0x7faa801069b0 2024-03-07T07:31:36.291 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.348+0000 7faa777fe700 1 --2- 172.21.15.160:0/2459221801 >> [v2:172.21.15.160:6800/1130675225,v1:172.21.15.160:6801/1130675225] conn(0x7faa6c038b50 0x7faa6c03b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:36.291 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.348+0000 7faa777fe700 1 -- 172.21.15.160:0/2459221801 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7faa7004c990 con 0x7faa801069b0 2024-03-07T07:31:36.291 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.348+0000 7faa85c04700 1 -- 172.21.15.160:0/2459221801 >> [v2:172.21.15.160:6800/1130675225,v1:172.21.15.160:6801/1130675225] conn(0x7faa6c038b50 msgr2=0x7faa6c03b010 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6800/1130675225 2024-03-07T07:31:36.291 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.348+0000 7faa85c04700 1 --2- 172.21.15.160:0/2459221801 >> [v2:172.21.15.160:6800/1130675225,v1:172.21.15.160:6801/1130675225] conn(0x7faa6c038b50 0x7faa6c03b010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-07T07:31:36.291 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.355+0000 7faa777fe700 1 -- 172.21.15.160:0/2459221801 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7faa70015070 con 0x7faa801069b0 2024-03-07T07:31:36.292 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.549+0000 7faa85c04700 1 -- 172.21.15.160:0/2459221801 >> [v2:172.21.15.160:6800/1130675225,v1:172.21.15.160:6801/1130675225] conn(0x7faa6c038b50 msgr2=0x7faa6c03b010 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6800/1130675225 2024-03-07T07:31:36.292 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.549+0000 7faa85c04700 1 --2- 172.21.15.160:0/2459221801 >> [v2:172.21.15.160:6800/1130675225,v1:172.21.15.160:6801/1130675225] conn(0x7faa6c038b50 0x7faa6c03b010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-07T07:31:36.292 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.613+0000 7faa88669700 1 -- 172.21.15.160:0/2459221801 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7faa8002ddc0 con 0x7faa801069b0 2024-03-07T07:31:36.292 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.614+0000 7faa777fe700 1 -- 172.21.15.160:0/2459221801 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7faa70026210 con 0x7faa801069b0 2024-03-07T07:31:36.292 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.616+0000 7faa88669700 1 -- 172.21.15.160:0/2459221801 >> [v2:172.21.15.160:6800/1130675225,v1:172.21.15.160:6801/1130675225] conn(0x7faa6c038b50 msgr2=0x7faa6c03b010 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-07T07:31:36.292 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.616+0000 7faa88669700 1 --2- 172.21.15.160:0/2459221801 >> [v2:172.21.15.160:6800/1130675225,v1:172.21.15.160:6801/1130675225] conn(0x7faa6c038b50 0x7faa6c03b010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:36.292 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.616+0000 7faa88669700 1 -- 172.21.15.160:0/2459221801 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa801069b0 msgr2=0x7faa8019dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:36.292 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.616+0000 7faa88669700 1 --2- 172.21.15.160:0/2459221801 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa801069b0 0x7faa8019dd40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7faa7000b340 tx=0x7faa700040b0 comp rx=0 tx=0).stop 2024-03-07T07:31:36.292 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.616+0000 7faa88669700 1 -- 172.21.15.160:0/2459221801 shutdown_connections 2024-03-07T07:31:36.292 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.616+0000 7faa88669700 1 --2- 172.21.15.160:0/2459221801 >> [v2:172.21.15.160:6800/1130675225,v1:172.21.15.160:6801/1130675225] conn(0x7faa6c038b50 0x7faa6c03b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:36.292 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.616+0000 7faa88669700 1 --2- 172.21.15.160:0/2459221801 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa801069b0 0x7faa8019dd40 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:36.292 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.616+0000 7faa88669700 1 -- 172.21.15.160:0/2459221801 >> 172.21.15.160:0/2459221801 conn(0x7faa80101f70 msgr2=0x7faa80190380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:31:36.292 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.616+0000 7faa88669700 1 -- 172.21.15.160:0/2459221801 shutdown_connections 2024-03-07T07:31:36.293 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:35.616+0000 7faa88669700 1 -- 172.21.15.160:0/2459221801 wait complete. 2024-03-07T07:31:36.293 INFO:teuthology.orchestra.run.smithi160.stdout:Waiting for the mgr to restart... 2024-03-07T07:31:36.293 INFO:teuthology.orchestra.run.smithi160.stdout:Waiting for mgr epoch 5... 2024-03-07T07:31:36.516 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:36 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/2459221801' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-07T07:31:36.516 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:36 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:36.287+0000 7f4e41a9b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-07T07:31:36.516 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:36 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:36.401+0000 7f4e41a9b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-07T07:31:36.517 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:36 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:36.489+0000 7f4e41a9b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-07T07:31:36.772 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:36 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:36.588+0000 7f4e41a9b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-07T07:31:36.773 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:36 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:36.772+0000 7f4e41a9b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-07T07:31:38.127 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:37 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:37.732+0000 7f4e41a9b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-07T07:31:38.127 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:37 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:37.861+0000 7f4e41a9b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-07T07:31:38.428 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:38 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:38.205+0000 7f4e41a9b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-07T07:31:38.428 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:38 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:38.295+0000 7f4e41a9b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-07T07:31:38.927 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:38 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:38.492+0000 7f4e41a9b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-07T07:31:38.928 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:38 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:38.679+0000 7f4e41a9b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-07T07:31:39.227 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:38 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:38.966+0000 7f4e41a9b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-07T07:31:39.227 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:39 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:39.226+0000 7f4e41a9b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-07T07:31:39.677 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:39 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:39.463+0000 7f4e41a9b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-07T07:31:40.677 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:40 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:40.268+0000 7f4e41a9b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-07T07:31:41.154 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:40 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:40.838+0000 7f4e41a9b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-07T07:31:41.154 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:40 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:40.936+0000 7f4e41a9b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-07T07:31:41.154 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:41 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:31:41.153+0000 7f4e41a9b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-07T07:31:41.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:41 smithi160 ceph-mon[31963]: Active manager daemon x restarted 2024-03-07T07:31:41.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:41 smithi160 ceph-mon[31963]: Activating manager daemon x 2024-03-07T07:31:41.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:41 smithi160 ceph-mon[31963]: osdmap e2: 0 total, 0 up, 0 in 2024-03-07T07:31:41.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:41 smithi160 ceph-mon[31963]: mgrmap e6: x(active, starting, since 0.00337302s) 2024-03-07T07:31:41.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:41 smithi160 ceph-mon[31963]: from='mgr.14120 172.21.15.160:0/1092429691' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-07T07:31:41.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:41 smithi160 ceph-mon[31963]: from='mgr.14120 172.21.15.160:0/1092429691' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-07T07:31:41.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:41 smithi160 ceph-mon[31963]: from='mgr.14120 172.21.15.160:0/1092429691' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-07T07:31:41.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:41 smithi160 ceph-mon[31963]: from='mgr.14120 172.21.15.160:0/1092429691' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-07T07:31:41.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:41 smithi160 ceph-mon[31963]: from='mgr.14120 172.21.15.160:0/1092429691' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-07T07:31:41.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:41 smithi160 ceph-mon[31963]: Manager daemon x is now available 2024-03-07T07:31:41.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:41 smithi160 ceph-mon[31963]: from='mgr.14120 172.21.15.160:0/1092429691' entity='mgr.x' 2024-03-07T07:31:41.429 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:41 smithi160 ceph-mon[31963]: from='mgr.14120 172.21.15.160:0/1092429691' entity='mgr.x' 2024-03-07T07:31:41.429 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:41 smithi160 ceph-mon[31963]: from='mgr.14120 172.21.15.160:0/1092429691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:31:41.429 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:41 smithi160 ceph-mon[31963]: from='mgr.14120 172.21.15.160:0/1092429691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:31:42.427 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:42 smithi160 ceph-mon[31963]: Found migration_current of "None". Setting to last migration. 2024-03-07T07:31:42.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:42 smithi160 ceph-mon[31963]: from='mgr.14120 172.21.15.160:0/1092429691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:31:42.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:42 smithi160 ceph-mon[31963]: from='mgr.14120 172.21.15.160:0/1092429691' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-07T07:31:42.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:42 smithi160 ceph-mon[31963]: from='mgr.14120 172.21.15.160:0/1092429691' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-07T07:31:42.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:42 smithi160 ceph-mon[31963]: from='mgr.14120 172.21.15.160:0/1092429691' entity='mgr.x' 2024-03-07T07:31:42.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:42 smithi160 ceph-mon[31963]: from='mgr.14120 172.21.15.160:0/1092429691' entity='mgr.x' 2024-03-07T07:31:42.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:42 smithi160 ceph-mon[31963]: mgrmap e7: x(active, since 1.00521s) 2024-03-07T07:31:42.778 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout { 2024-03-07T07:31:42.778 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-07T07:31:42.779 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-07T07:31:42.779 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout } 2024-03-07T07:31:42.779 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:36.719+0000 7f293efab700 1 Processor -- start 2024-03-07T07:31:42.779 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:36.719+0000 7f293efab700 1 -- start start 2024-03-07T07:31:42.779 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:36.720+0000 7f293efab700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2938108590 0x7f293810a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:42.779 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:36.720+0000 7f293efab700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f293807be10 con 0x7f2938108590 2024-03-07T07:31:42.779 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:36.720+0000 7f293cd47700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2938108590 0x7f293810a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:42.779 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:36.720+0000 7f293cd47700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2938108590 0x7f293810a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41332/0 (socket says 172.21.15.160:41332) 2024-03-07T07:31:42.779 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:36.720+0000 7f293cd47700 1 -- 172.21.15.160:0/3787371670 learned_addr learned my addr 172.21.15.160:0/3787371670 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:31:42.779 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:36.720+0000 7f293cd47700 1 -- 172.21.15.160:0/3787371670 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f293810aec0 con 0x7f2938108590 2024-03-07T07:31:42.779 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:36.720+0000 7f293cd47700 1 --2- 172.21.15.160:0/3787371670 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2938108590 0x7f293810a980 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f292c0098a0 tx=0x7f292c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e28efb44a80a2a server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:42.780 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:36.720+0000 7f29377fe700 1 -- 172.21.15.160:0/3787371670 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f292c004160 con 0x7f2938108590 2024-03-07T07:31:42.780 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:36.720+0000 7f29377fe700 1 -- 172.21.15.160:0/3787371670 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f292c004300 con 0x7f2938108590 2024-03-07T07:31:42.780 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:36.720+0000 7f29377fe700 1 -- 172.21.15.160:0/3787371670 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f292c011b50 con 0x7f2938108590 2024-03-07T07:31:42.780 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:36.721+0000 7f293efab700 1 -- 172.21.15.160:0/3787371670 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2938108590 msgr2=0x7f293810a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:42.780 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:36.721+0000 7f293efab700 1 --2- 172.21.15.160:0/3787371670 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2938108590 0x7f293810a980 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f292c0098a0 tx=0x7f292c009bb0 comp rx=0 tx=0).stop 2024-03-07T07:31:42.780 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:36.721+0000 7f293efab700 1 -- 172.21.15.160:0/3787371670 shutdown_connections 2024-03-07T07:31:42.780 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:36.721+0000 7f293efab700 1 --2- 172.21.15.160:0/3787371670 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2938108590 0x7f293810a980 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:42.780 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:36.721+0000 7f293efab700 1 -- 172.21.15.160:0/3787371670 >> 172.21.15.160:0/3787371670 conn(0x7f2938101f40 msgr2=0x7f2938104380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:31:42.780 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:36.721+0000 7f293efab700 1 -- 172.21.15.160:0/3787371670 shutdown_connections 2024-03-07T07:31:42.780 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:36.721+0000 7f293efab700 1 -- 172.21.15.160:0/3787371670 wait complete. 2024-03-07T07:31:42.780 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:36.722+0000 7f293efab700 1 Processor -- start 2024-03-07T07:31:42.780 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:36.722+0000 7f293efab700 1 -- start start 2024-03-07T07:31:42.780 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:36.722+0000 7f293efab700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2938108590 0x7f293819ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:42.780 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:36.722+0000 7f293efab700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f293810be80 con 0x7f2938108590 2024-03-07T07:31:42.781 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:36.722+0000 7f293cd47700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2938108590 0x7f293819ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:42.781 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:36.722+0000 7f293cd47700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2938108590 0x7f293819ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41344/0 (socket says 172.21.15.160:41344) 2024-03-07T07:31:42.781 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:36.722+0000 7f293cd47700 1 -- 172.21.15.160:0/1176702649 learned_addr learned my addr 172.21.15.160:0/1176702649 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:31:42.781 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:36.722+0000 7f293cd47700 1 -- 172.21.15.160:0/1176702649 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f293819bfa0 con 0x7f2938108590 2024-03-07T07:31:42.781 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:36.722+0000 7f293cd47700 1 --2- 172.21.15.160:0/1176702649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2938108590 0x7f293819ba60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f292c011670 tx=0x7f292c011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:42.781 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:36.723+0000 7f2935ffb700 1 -- 172.21.15.160:0/1176702649 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f292c0039c0 con 0x7f2938108590 2024-03-07T07:31:42.781 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:36.723+0000 7f293efab700 1 -- 172.21.15.160:0/1176702649 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f293819c230 con 0x7f2938108590 2024-03-07T07:31:42.781 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:36.723+0000 7f2935ffb700 1 -- 172.21.15.160:0/1176702649 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f292c011e50 con 0x7f2938108590 2024-03-07T07:31:42.781 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:36.723+0000 7f2935ffb700 1 -- 172.21.15.160:0/1176702649 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f292c01a5c0 con 0x7f2938108590 2024-03-07T07:31:42.781 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:36.723+0000 7f293efab700 1 -- 172.21.15.160:0/1176702649 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f293819ef20 con 0x7f2938108590 2024-03-07T07:31:42.781 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:36.724+0000 7f2935ffb700 1 -- 172.21.15.160:0/1176702649 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) v1 ==== 45927+0+0 (secure 0 0 0) 0x7f292c01a870 con 0x7f2938108590 2024-03-07T07:31:42.781 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:36.724+0000 7f2935ffb700 1 --2- 172.21.15.160:0/1176702649 >> [v2:172.21.15.160:6800/1130675225,v1:172.21.15.160:6801/1130675225] conn(0x7f2928038ba0 0x7f292803b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:42.781 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:36.724+0000 7f2935ffb700 1 -- 172.21.15.160:0/1176702649 --> [v2:172.21.15.160:6800/1130675225,v1:172.21.15.160:6801/1130675225] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f292803b770 con 0x7f2928038ba0 2024-03-07T07:31:42.781 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:36.724+0000 7f2935ffb700 1 -- 172.21.15.160:0/1176702649 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f292c04cdc0 con 0x7f2938108590 2024-03-07T07:31:42.782 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:36.724+0000 7f2937fff700 1 -- 172.21.15.160:0/1176702649 >> [v2:172.21.15.160:6800/1130675225,v1:172.21.15.160:6801/1130675225] conn(0x7f2928038ba0 msgr2=0x7f292803b060 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6800/1130675225 2024-03-07T07:31:42.782 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:36.724+0000 7f2937fff700 1 --2- 172.21.15.160:0/1176702649 >> [v2:172.21.15.160:6800/1130675225,v1:172.21.15.160:6801/1130675225] conn(0x7f2928038ba0 0x7f292803b060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-07T07:31:42.782 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:36.925+0000 7f2937fff700 1 -- 172.21.15.160:0/1176702649 >> [v2:172.21.15.160:6800/1130675225,v1:172.21.15.160:6801/1130675225] conn(0x7f2928038ba0 msgr2=0x7f292803b060 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6800/1130675225 2024-03-07T07:31:42.782 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:36.925+0000 7f2937fff700 1 --2- 172.21.15.160:0/1176702649 >> [v2:172.21.15.160:6800/1130675225,v1:172.21.15.160:6801/1130675225] conn(0x7f2928038ba0 0x7f292803b060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-07T07:31:42.782 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:37.326+0000 7f2937fff700 1 -- 172.21.15.160:0/1176702649 >> [v2:172.21.15.160:6800/1130675225,v1:172.21.15.160:6801/1130675225] conn(0x7f2928038ba0 msgr2=0x7f292803b060 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6800/1130675225 2024-03-07T07:31:42.782 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:37.326+0000 7f2937fff700 1 --2- 172.21.15.160:0/1176702649 >> [v2:172.21.15.160:6800/1130675225,v1:172.21.15.160:6801/1130675225] conn(0x7f2928038ba0 0x7f292803b060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-07T07:31:42.782 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:38.127+0000 7f2937fff700 1 -- 172.21.15.160:0/1176702649 >> [v2:172.21.15.160:6800/1130675225,v1:172.21.15.160:6801/1130675225] conn(0x7f2928038ba0 msgr2=0x7f292803b060 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6800/1130675225 2024-03-07T07:31:42.782 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:38.127+0000 7f2937fff700 1 --2- 172.21.15.160:0/1176702649 >> [v2:172.21.15.160:6800/1130675225,v1:172.21.15.160:6801/1130675225] conn(0x7f2928038ba0 0x7f292803b060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-07T07:31:42.782 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:39.728+0000 7f2937fff700 1 -- 172.21.15.160:0/1176702649 >> [v2:172.21.15.160:6800/1130675225,v1:172.21.15.160:6801/1130675225] conn(0x7f2928038ba0 msgr2=0x7f292803b060 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6800/1130675225 2024-03-07T07:31:42.782 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:39.728+0000 7f2937fff700 1 --2- 172.21.15.160:0/1176702649 >> [v2:172.21.15.160:6800/1130675225,v1:172.21.15.160:6801/1130675225] conn(0x7f2928038ba0 0x7f292803b060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-07T07:31:42.782 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:41.168+0000 7f2935ffb700 1 -- 172.21.15.160:0/1176702649 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mgrmap(e 6) v1 ==== 45694+0+0 (secure 0 0 0) 0x7f292c01abb0 con 0x7f2938108590 2024-03-07T07:31:42.782 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:41.168+0000 7f2935ffb700 1 -- 172.21.15.160:0/1176702649 >> [v2:172.21.15.160:6800/1130675225,v1:172.21.15.160:6801/1130675225] conn(0x7f2928038ba0 msgr2=0x7f292803b060 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-07T07:31:42.782 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:41.168+0000 7f2935ffb700 1 --2- 172.21.15.160:0/1176702649 >> [v2:172.21.15.160:6800/1130675225,v1:172.21.15.160:6801/1130675225] conn(0x7f2928038ba0 0x7f292803b060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:42.782 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:42.168+0000 7f2935ffb700 1 -- 172.21.15.160:0/1176702649 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mgrmap(e 7) v1 ==== 45821+0+0 (secure 0 0 0) 0x7f292c04e580 con 0x7f2938108590 2024-03-07T07:31:42.783 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:42.169+0000 7f2935ffb700 1 --2- 172.21.15.160:0/1176702649 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7f292803c5f0 0x7f292803e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:42.783 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:42.169+0000 7f2935ffb700 1 -- 172.21.15.160:0/1176702649 --> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f292c008ea0 con 0x7f292803c5f0 2024-03-07T07:31:42.783 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:42.169+0000 7f2937fff700 1 --2- 172.21.15.160:0/1176702649 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7f292803c5f0 0x7f292803e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:42.783 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:42.170+0000 7f2937fff700 1 --2- 172.21.15.160:0/1176702649 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7f292803c5f0 0x7f292803e9e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2924003a10 tx=0x7f29240092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:42.783 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:42.171+0000 7f2935ffb700 1 -- 172.21.15.160:0/1176702649 <== mgr.14120 v2:172.21.15.160:6800/3859174315 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f292c008ea0 con 0x7f292803c5f0 2024-03-07T07:31:42.783 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:42.177+0000 7f293efab700 1 -- 172.21.15.160:0/1176702649 --> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f2938063780 con 0x7f292803c5f0 2024-03-07T07:31:42.783 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:42.178+0000 7f2935ffb700 1 -- 172.21.15.160:0/1176702649 <== mgr.14120 v2:172.21.15.160:6800/3859174315 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f2938063780 con 0x7f292803c5f0 2024-03-07T07:31:42.783 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:42.178+0000 7f293efab700 1 -- 172.21.15.160:0/1176702649 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7f292803c5f0 msgr2=0x7f292803e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:42.783 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:42.178+0000 7f293efab700 1 --2- 172.21.15.160:0/1176702649 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7f292803c5f0 0x7f292803e9e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2924003a10 tx=0x7f29240092b0 comp rx=0 tx=0).stop 2024-03-07T07:31:42.783 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:42.178+0000 7f293efab700 1 -- 172.21.15.160:0/1176702649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2938108590 msgr2=0x7f293819ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:42.783 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:42.178+0000 7f293efab700 1 --2- 172.21.15.160:0/1176702649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2938108590 0x7f293819ba60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f292c011670 tx=0x7f292c011b60 comp rx=0 tx=0).stop 2024-03-07T07:31:42.783 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:42.178+0000 7f293efab700 1 -- 172.21.15.160:0/1176702649 shutdown_connections 2024-03-07T07:31:42.783 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:42.178+0000 7f293efab700 1 --2- 172.21.15.160:0/1176702649 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7f292803c5f0 0x7f292803e9e0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:42.784 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:42.178+0000 7f293efab700 1 --2- 172.21.15.160:0/1176702649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2938108590 0x7f293819ba60 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:42.784 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:42.178+0000 7f293efab700 1 --2- 172.21.15.160:0/1176702649 >> [v2:172.21.15.160:6800/1130675225,v1:172.21.15.160:6801/1130675225] conn(0x7f2928038ba0 0x7f292803b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:42.784 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:42.178+0000 7f293efab700 1 -- 172.21.15.160:0/1176702649 >> 172.21.15.160:0/1176702649 conn(0x7f2938101f40 msgr2=0x7f2938102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:31:42.784 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:42.178+0000 7f293efab700 1 -- 172.21.15.160:0/1176702649 shutdown_connections 2024-03-07T07:31:42.784 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:42.178+0000 7f293efab700 1 -- 172.21.15.160:0/1176702649 wait complete. 2024-03-07T07:31:42.785 INFO:teuthology.orchestra.run.smithi160.stdout:mgr epoch 5 is available 2024-03-07T07:31:42.785 INFO:teuthology.orchestra.run.smithi160.stdout:Setting orchestrator backend to cephadm... 2024-03-07T07:31:43.419 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:43 smithi160 ceph-mon[31963]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-07T07:31:43.420 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:43 smithi160 ceph-mon[31963]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-07T07:31:43.420 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:43 smithi160 ceph-mon[31963]: from='mgr.14120 172.21.15.160:0/1092429691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:31:43.950 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.196+0000 7fbb5fab2700 1 Processor -- start 2024-03-07T07:31:43.950 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.196+0000 7fbb5fab2700 1 -- start start 2024-03-07T07:31:43.950 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.197+0000 7fbb5fab2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb58105e00 0x7fbb58106220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:43.950 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.197+0000 7fbb5fab2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb581067f0 con 0x7fbb58105e00 2024-03-07T07:31:43.950 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.197+0000 7fbb5d84e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb58105e00 0x7fbb58106220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:43.950 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.197+0000 7fbb5d84e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb58105e00 0x7fbb58106220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59606/0 (socket says 172.21.15.160:59606) 2024-03-07T07:31:43.950 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.197+0000 7fbb5d84e700 1 -- 172.21.15.160:0/685835955 learned_addr learned my addr 172.21.15.160:0/685835955 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:31:43.950 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.197+0000 7fbb5d84e700 1 -- 172.21.15.160:0/685835955 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb58108d90 con 0x7fbb58105e00 2024-03-07T07:31:43.950 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.197+0000 7fbb5d84e700 1 --2- 172.21.15.160:0/685835955 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb58105e00 0x7fbb58106220 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbb540098a0 tx=0x7fbb54009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af5eac37a380ef9e server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:43.950 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.197+0000 7fbb5c84c700 1 -- 172.21.15.160:0/685835955 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbb54004160 con 0x7fbb58105e00 2024-03-07T07:31:43.950 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.197+0000 7fbb5c84c700 1 -- 172.21.15.160:0/685835955 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbb54004300 con 0x7fbb58105e00 2024-03-07T07:31:43.951 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.197+0000 7fbb5c84c700 1 -- 172.21.15.160:0/685835955 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbb54011b50 con 0x7fbb58105e00 2024-03-07T07:31:43.951 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.198+0000 7fbb5fab2700 1 -- 172.21.15.160:0/685835955 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb58105e00 msgr2=0x7fbb58106220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:43.951 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.198+0000 7fbb5fab2700 1 --2- 172.21.15.160:0/685835955 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb58105e00 0x7fbb58106220 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbb540098a0 tx=0x7fbb54009bb0 comp rx=0 tx=0).stop 2024-03-07T07:31:43.951 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.198+0000 7fbb5fab2700 1 -- 172.21.15.160:0/685835955 shutdown_connections 2024-03-07T07:31:43.951 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.198+0000 7fbb5fab2700 1 --2- 172.21.15.160:0/685835955 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb58105e00 0x7fbb58106220 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:43.951 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.198+0000 7fbb5fab2700 1 -- 172.21.15.160:0/685835955 >> 172.21.15.160:0/685835955 conn(0x7fbb58101790 msgr2=0x7fbb58103bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:31:43.951 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.198+0000 7fbb5fab2700 1 -- 172.21.15.160:0/685835955 shutdown_connections 2024-03-07T07:31:43.951 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.198+0000 7fbb5fab2700 1 -- 172.21.15.160:0/685835955 wait complete. 2024-03-07T07:31:43.951 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.199+0000 7fbb5fab2700 1 Processor -- start 2024-03-07T07:31:43.951 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.199+0000 7fbb5fab2700 1 -- start start 2024-03-07T07:31:43.951 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.199+0000 7fbb5fab2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb58105e00 0x7fbb5819bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:43.951 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.199+0000 7fbb5fab2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5810c140 con 0x7fbb58105e00 2024-03-07T07:31:43.951 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.199+0000 7fbb5d84e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb58105e00 0x7fbb5819bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:43.951 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.199+0000 7fbb5d84e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb58105e00 0x7fbb5819bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59612/0 (socket says 172.21.15.160:59612) 2024-03-07T07:31:43.952 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.199+0000 7fbb5d84e700 1 -- 172.21.15.160:0/4236312438 learned_addr learned my addr 172.21.15.160:0/4236312438 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:31:43.952 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.199+0000 7fbb5d84e700 1 -- 172.21.15.160:0/4236312438 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb5819c0b0 con 0x7fbb58105e00 2024-03-07T07:31:43.952 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.199+0000 7fbb5d84e700 1 --2- 172.21.15.160:0/4236312438 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb58105e00 0x7fbb5819bb70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbb54009fb0 tx=0x7fbb54011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:43.952 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.199+0000 7fbb4effd700 1 -- 172.21.15.160:0/4236312438 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbb540038b0 con 0x7fbb58105e00 2024-03-07T07:31:43.952 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.199+0000 7fbb4effd700 1 -- 172.21.15.160:0/4236312438 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbb54011e50 con 0x7fbb58105e00 2024-03-07T07:31:43.952 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.199+0000 7fbb5fab2700 1 -- 172.21.15.160:0/4236312438 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb5819c340 con 0x7fbb58105e00 2024-03-07T07:31:43.952 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.199+0000 7fbb4effd700 1 -- 172.21.15.160:0/4236312438 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbb54019550 con 0x7fbb58105e00 2024-03-07T07:31:43.952 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.199+0000 7fbb5fab2700 1 -- 172.21.15.160:0/4236312438 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb5819f030 con 0x7fbb58105e00 2024-03-07T07:31:43.952 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.200+0000 7fbb4effd700 1 -- 172.21.15.160:0/4236312438 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 7) v1 ==== 45821+0+0 (secure 0 0 0) 0x7fbb54019850 con 0x7fbb58105e00 2024-03-07T07:31:43.952 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.200+0000 7fbb5fab2700 1 -- 172.21.15.160:0/4236312438 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb58063780 con 0x7fbb58105e00 2024-03-07T07:31:43.952 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.200+0000 7fbb4effd700 1 --2- 172.21.15.160:0/4236312438 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7fbb44038a40 0x7fbb4403af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:43.952 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.200+0000 7fbb4effd700 1 -- 172.21.15.160:0/4236312438 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fbb5404cc80 con 0x7fbb58105e00 2024-03-07T07:31:43.952 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.201+0000 7fbb5d04d700 1 --2- 172.21.15.160:0/4236312438 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7fbb44038a40 0x7fbb4403af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:43.952 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.201+0000 7fbb5d04d700 1 --2- 172.21.15.160:0/4236312438 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7fbb44038a40 0x7fbb4403af00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbb48006fd0 tx=0x7fbb48006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:43.953 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.206+0000 7fbb4effd700 1 -- 172.21.15.160:0/4236312438 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fbb54020da0 con 0x7fbb58105e00 2024-03-07T07:31:43.953 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.223+0000 7fbb4effd700 1 -- 172.21.15.160:0/4236312438 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mgrmap(e 8) v1 ==== 45927+0+0 (secure 0 0 0) 0x7fbb5402d960 con 0x7fbb58105e00 2024-03-07T07:31:43.953 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.418+0000 7fbb5fab2700 1 -- 172.21.15.160:0/4236312438 --> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb581029e0 con 0x7fbb44038a40 2024-03-07T07:31:43.953 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.425+0000 7fbb4effd700 1 -- 172.21.15.160:0/4236312438 <== mgr.14120 v2:172.21.15.160:6800/3859174315 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fbb581029e0 con 0x7fbb44038a40 2024-03-07T07:31:43.953 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.427+0000 7fbb5fab2700 1 -- 172.21.15.160:0/4236312438 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7fbb44038a40 msgr2=0x7fbb4403af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:43.953 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.427+0000 7fbb5fab2700 1 --2- 172.21.15.160:0/4236312438 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7fbb44038a40 0x7fbb4403af00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbb48006fd0 tx=0x7fbb48006e40 comp rx=0 tx=0).stop 2024-03-07T07:31:43.953 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.427+0000 7fbb5fab2700 1 -- 172.21.15.160:0/4236312438 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb58105e00 msgr2=0x7fbb5819bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:43.953 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.427+0000 7fbb5fab2700 1 --2- 172.21.15.160:0/4236312438 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb58105e00 0x7fbb5819bb70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbb54009fb0 tx=0x7fbb54011bf0 comp rx=0 tx=0).stop 2024-03-07T07:31:43.953 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.427+0000 7fbb5fab2700 1 -- 172.21.15.160:0/4236312438 shutdown_connections 2024-03-07T07:31:43.953 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.427+0000 7fbb5fab2700 1 --2- 172.21.15.160:0/4236312438 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7fbb44038a40 0x7fbb4403af00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:43.953 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.427+0000 7fbb5fab2700 1 --2- 172.21.15.160:0/4236312438 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb58105e00 0x7fbb5819bb70 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:43.953 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.427+0000 7fbb5fab2700 1 -- 172.21.15.160:0/4236312438 >> 172.21.15.160:0/4236312438 conn(0x7fbb58101790 msgr2=0x7fbb581022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:31:43.953 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.428+0000 7fbb5fab2700 1 -- 172.21.15.160:0/4236312438 shutdown_connections 2024-03-07T07:31:43.953 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:43.428+0000 7fbb5fab2700 1 -- 172.21.15.160:0/4236312438 wait complete. 2024-03-07T07:31:44.320 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:44 smithi160 ceph-mon[31963]: [07/Mar/2024:07:31:42] ENGINE Bus STARTING 2024-03-07T07:31:44.320 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:44 smithi160 ceph-mon[31963]: [07/Mar/2024:07:31:42] ENGINE Serving on http://172.21.15.160:8765 2024-03-07T07:31:44.320 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:44 smithi160 ceph-mon[31963]: [07/Mar/2024:07:31:42] ENGINE Serving on https://172.21.15.160:7150 2024-03-07T07:31:44.320 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:44 smithi160 ceph-mon[31963]: [07/Mar/2024:07:31:42] ENGINE Bus STARTED 2024-03-07T07:31:44.320 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:44 smithi160 ceph-mon[31963]: mgrmap e8: x(active, since 2s) 2024-03-07T07:31:44.320 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:44 smithi160 ceph-mon[31963]: from='mgr.14120 172.21.15.160:0/1092429691' entity='mgr.x' 2024-03-07T07:31:44.320 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:44 smithi160 ceph-mon[31963]: from='mgr.14120 172.21.15.160:0/1092429691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:31:45.172 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-07T07:31:45.172 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.352+0000 7f1e9a0c4700 1 Processor -- start 2024-03-07T07:31:45.172 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.352+0000 7f1e9a0c4700 1 -- start start 2024-03-07T07:31:45.172 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.352+0000 7f1e9a0c4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e94108520 0x7f1e9410a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:45.172 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.352+0000 7f1e9a0c4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e940758c0 con 0x7f1e94108520 2024-03-07T07:31:45.172 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.352+0000 7f1e937fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e94108520 0x7f1e9410a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:45.172 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.352+0000 7f1e937fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e94108520 0x7f1e9410a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59618/0 (socket says 172.21.15.160:59618) 2024-03-07T07:31:45.172 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.352+0000 7f1e937fe700 1 -- 172.21.15.160:0/3463399871 learned_addr learned my addr 172.21.15.160:0/3463399871 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:31:45.173 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.352+0000 7f1e937fe700 1 -- 172.21.15.160:0/3463399871 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e9410ae50 con 0x7f1e94108520 2024-03-07T07:31:45.173 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.352+0000 7f1e937fe700 1 --2- 172.21.15.160:0/3463399871 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e94108520 0x7f1e9410a910 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1e840098a0 tx=0x7f1e84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3fd631bd0f911f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:45.173 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.353+0000 7f1e927fc700 1 -- 172.21.15.160:0/3463399871 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1e84004160 con 0x7f1e94108520 2024-03-07T07:31:45.173 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.353+0000 7f1e927fc700 1 -- 172.21.15.160:0/3463399871 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1e84004300 con 0x7f1e94108520 2024-03-07T07:31:45.173 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.353+0000 7f1e927fc700 1 -- 172.21.15.160:0/3463399871 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1e84011b50 con 0x7f1e94108520 2024-03-07T07:31:45.173 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.353+0000 7f1e9a0c4700 1 -- 172.21.15.160:0/3463399871 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e94108520 msgr2=0x7f1e9410a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:45.173 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.353+0000 7f1e9a0c4700 1 --2- 172.21.15.160:0/3463399871 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e94108520 0x7f1e9410a910 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1e840098a0 tx=0x7f1e84009bb0 comp rx=0 tx=0).stop 2024-03-07T07:31:45.173 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.353+0000 7f1e9a0c4700 1 -- 172.21.15.160:0/3463399871 shutdown_connections 2024-03-07T07:31:45.173 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.353+0000 7f1e9a0c4700 1 --2- 172.21.15.160:0/3463399871 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e94108520 0x7f1e9410a910 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:45.173 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.353+0000 7f1e9a0c4700 1 -- 172.21.15.160:0/3463399871 >> 172.21.15.160:0/3463399871 conn(0x7f1e94101f70 msgr2=0x7f1e94104390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:31:45.173 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.353+0000 7f1e9a0c4700 1 -- 172.21.15.160:0/3463399871 shutdown_connections 2024-03-07T07:31:45.173 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.353+0000 7f1e9a0c4700 1 -- 172.21.15.160:0/3463399871 wait complete. 2024-03-07T07:31:45.173 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.354+0000 7f1e9a0c4700 1 Processor -- start 2024-03-07T07:31:45.173 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.354+0000 7f1e9a0c4700 1 -- start start 2024-03-07T07:31:45.174 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.354+0000 7f1e9a0c4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e94108520 0x7f1e94107da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:45.174 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.354+0000 7f1e9a0c4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9410be80 con 0x7f1e94108520 2024-03-07T07:31:45.174 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.354+0000 7f1e937fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e94108520 0x7f1e94107da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:45.174 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.354+0000 7f1e937fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e94108520 0x7f1e94107da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59630/0 (socket says 172.21.15.160:59630) 2024-03-07T07:31:45.174 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.354+0000 7f1e937fe700 1 -- 172.21.15.160:0/2995246264 learned_addr learned my addr 172.21.15.160:0/2995246264 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:31:45.174 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.354+0000 7f1e937fe700 1 -- 172.21.15.160:0/2995246264 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e941082e0 con 0x7f1e94108520 2024-03-07T07:31:45.174 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.355+0000 7f1e937fe700 1 --2- 172.21.15.160:0/2995246264 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e94108520 0x7f1e94107da0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1e84009fb0 tx=0x7f1e84011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:45.174 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.355+0000 7f1e90ff9700 1 -- 172.21.15.160:0/2995246264 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1e840038b0 con 0x7f1e94108520 2024-03-07T07:31:45.174 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.355+0000 7f1e90ff9700 1 -- 172.21.15.160:0/2995246264 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1e84011e50 con 0x7f1e94108520 2024-03-07T07:31:45.174 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.355+0000 7f1e9a0c4700 1 -- 172.21.15.160:0/2995246264 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e941064d0 con 0x7f1e94108520 2024-03-07T07:31:45.174 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.355+0000 7f1e90ff9700 1 -- 172.21.15.160:0/2995246264 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1e84019420 con 0x7f1e94108520 2024-03-07T07:31:45.174 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.355+0000 7f1e9a0c4700 1 -- 172.21.15.160:0/2995246264 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e94106a10 con 0x7f1e94108520 2024-03-07T07:31:45.174 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.356+0000 7f1e90ff9700 1 -- 172.21.15.160:0/2995246264 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 8) v1 ==== 45927+0+0 (secure 0 0 0) 0x7f1e840196f0 con 0x7f1e94108520 2024-03-07T07:31:45.174 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.356+0000 7f1e9a0c4700 1 -- 172.21.15.160:0/2995246264 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e94063780 con 0x7f1e94108520 2024-03-07T07:31:45.175 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.356+0000 7f1e90ff9700 1 --2- 172.21.15.160:0/2995246264 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7f1e7c038ba0 0x7f1e7c03b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:45.175 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.356+0000 7f1e90ff9700 1 -- 172.21.15.160:0/2995246264 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f1e8404cee0 con 0x7f1e94108520 2024-03-07T07:31:45.175 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.356+0000 7f1e92ffd700 1 --2- 172.21.15.160:0/2995246264 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7f1e7c038ba0 0x7f1e7c03b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:45.175 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.356+0000 7f1e92ffd700 1 --2- 172.21.15.160:0/2995246264 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7f1e7c038ba0 0x7f1e7c03b060 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1e88006fd0 tx=0x7f1e88006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:45.175 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.361+0000 7f1e90ff9700 1 -- 172.21.15.160:0/2995246264 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1e8400ba30 con 0x7f1e94108520 2024-03-07T07:31:45.175 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.558+0000 7f1e9a0c4700 1 -- 172.21.15.160:0/2995246264 --> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e94107270 con 0x7f1e7c038ba0 2024-03-07T07:31:45.175 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.558+0000 7f1e90ff9700 1 -- 172.21.15.160:0/2995246264 <== mgr.14120 v2:172.21.15.160:6800/3859174315 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f1e94107270 con 0x7f1e7c038ba0 2024-03-07T07:31:45.175 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.561+0000 7f1e9a0c4700 1 -- 172.21.15.160:0/2995246264 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7f1e7c038ba0 msgr2=0x7f1e7c03b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:45.175 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.561+0000 7f1e9a0c4700 1 --2- 172.21.15.160:0/2995246264 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7f1e7c038ba0 0x7f1e7c03b060 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1e88006fd0 tx=0x7f1e88006e40 comp rx=0 tx=0).stop 2024-03-07T07:31:45.175 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.561+0000 7f1e9a0c4700 1 -- 172.21.15.160:0/2995246264 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e94108520 msgr2=0x7f1e94107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:45.175 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.561+0000 7f1e9a0c4700 1 --2- 172.21.15.160:0/2995246264 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e94108520 0x7f1e94107da0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1e84009fb0 tx=0x7f1e84011bf0 comp rx=0 tx=0).stop 2024-03-07T07:31:45.175 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.561+0000 7f1e9a0c4700 1 -- 172.21.15.160:0/2995246264 shutdown_connections 2024-03-07T07:31:45.175 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.561+0000 7f1e9a0c4700 1 --2- 172.21.15.160:0/2995246264 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7f1e7c038ba0 0x7f1e7c03b060 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:45.175 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.561+0000 7f1e9a0c4700 1 --2- 172.21.15.160:0/2995246264 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e94108520 0x7f1e94107da0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:45.176 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.561+0000 7f1e9a0c4700 1 -- 172.21.15.160:0/2995246264 >> 172.21.15.160:0/2995246264 conn(0x7f1e94101f70 msgr2=0x7f1e94102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:31:45.176 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.561+0000 7f1e9a0c4700 1 -- 172.21.15.160:0/2995246264 shutdown_connections 2024-03-07T07:31:45.176 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:44.561+0000 7f1e9a0c4700 1 -- 172.21.15.160:0/2995246264 wait complete. 2024-03-07T07:31:45.176 INFO:teuthology.orchestra.run.smithi160.stdout:Generating ssh key... 2024-03-07T07:31:45.427 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:45 smithi160 ceph-mon[31963]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:31:46.404 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:46 smithi160 ceph-mon[31963]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:31:46.404 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:46 smithi160 ceph-mon[31963]: from='mgr.14120 172.21.15.160:0/1092429691' entity='mgr.x' 2024-03-07T07:31:46.404 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:46 smithi160 ceph-mon[31963]: from='mgr.14120 172.21.15.160:0/1092429691' entity='mgr.x' 2024-03-07T07:31:46.404 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:46 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: Generating public/private rsa key pair. 2024-03-07T07:31:46.404 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:46 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: Your identification has been saved in /tmp/tmpnpjamzew/key. 2024-03-07T07:31:46.404 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:46 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: Your public key has been saved in /tmp/tmpnpjamzew/key.pub. 2024-03-07T07:31:46.404 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:46 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: The key fingerprint is: 2024-03-07T07:31:46.404 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:46 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: SHA256:GrySsJYtm0NB5wg9JhYj0RfyKUIyLi6pw5k12UbM1xA ceph-826f6582-dc54-11ee-95c6-87774f69a715 2024-03-07T07:31:46.405 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:46 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: The key's randomart image is: 2024-03-07T07:31:46.405 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:46 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: +---[RSA 3072]----+ 2024-03-07T07:31:46.405 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:46 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: |=Bo .. E. | 2024-03-07T07:31:46.405 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:46 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: |BoBo=. o | 2024-03-07T07:31:46.405 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:46 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: |+*.*o+ . . | 2024-03-07T07:31:46.405 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:46 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: |ooo.=.. | 2024-03-07T07:31:46.405 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:46 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: |o..= oo S | 2024-03-07T07:31:46.405 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:46 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: |+ ==o. + | 2024-03-07T07:31:46.405 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:46 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: |o== + o | 2024-03-07T07:31:46.405 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:46 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: | o.+ . | 2024-03-07T07:31:46.405 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:46 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: | o. | 2024-03-07T07:31:46.405 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:31:46 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: +----[SHA256]-----+ 2024-03-07T07:31:46.687 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:45.608+0000 7fbea2f89700 1 Processor -- start 2024-03-07T07:31:46.687 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:45.608+0000 7fbea2f89700 1 -- start start 2024-03-07T07:31:46.687 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:45.608+0000 7fbea2f89700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe9c07cd70 0x7fbe9c07b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:46.687 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:45.609+0000 7fbea2f89700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9c07b7a0 con 0x7fbe9c07cd70 2024-03-07T07:31:46.687 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:45.609+0000 7fbea0d25700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe9c07cd70 0x7fbe9c07b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:46.687 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:45.609+0000 7fbea0d25700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe9c07cd70 0x7fbe9c07b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59634/0 (socket says 172.21.15.160:59634) 2024-03-07T07:31:46.687 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:45.609+0000 7fbea0d25700 1 -- 172.21.15.160:0/752336109 learned_addr learned my addr 172.21.15.160:0/752336109 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:31:46.687 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:45.609+0000 7fbea0d25700 1 -- 172.21.15.160:0/752336109 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe9c07b920 con 0x7fbe9c07cd70 2024-03-07T07:31:46.687 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:45.609+0000 7fbea0d25700 1 --2- 172.21.15.160:0/752336109 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe9c07cd70 0x7fbe9c07b1d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbe900098a0 tx=0x7fbe90009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e22ea0c93ddb5eb server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:46.687 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:45.609+0000 7fbe9b7fe700 1 -- 172.21.15.160:0/752336109 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe90004160 con 0x7fbe9c07cd70 2024-03-07T07:31:46.687 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:45.609+0000 7fbe9b7fe700 1 -- 172.21.15.160:0/752336109 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe90004300 con 0x7fbe9c07cd70 2024-03-07T07:31:46.688 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:45.609+0000 7fbe9b7fe700 1 -- 172.21.15.160:0/752336109 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe90011b50 con 0x7fbe9c07cd70 2024-03-07T07:31:46.688 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:45.610+0000 7fbea2f89700 1 -- 172.21.15.160:0/752336109 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe9c07cd70 msgr2=0x7fbe9c07b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:46.688 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:45.610+0000 7fbea2f89700 1 --2- 172.21.15.160:0/752336109 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe9c07cd70 0x7fbe9c07b1d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbe900098a0 tx=0x7fbe90009bb0 comp rx=0 tx=0).stop 2024-03-07T07:31:46.688 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:45.610+0000 7fbea2f89700 1 -- 172.21.15.160:0/752336109 shutdown_connections 2024-03-07T07:31:46.688 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:45.610+0000 7fbea2f89700 1 --2- 172.21.15.160:0/752336109 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe9c07cd70 0x7fbe9c07b1d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:46.688 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:45.610+0000 7fbea2f89700 1 -- 172.21.15.160:0/752336109 >> 172.21.15.160:0/752336109 conn(0x7fbe9c103080 msgr2=0x7fbe9c1054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:31:46.688 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:45.610+0000 7fbea2f89700 1 -- 172.21.15.160:0/752336109 shutdown_connections 2024-03-07T07:31:46.688 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:45.610+0000 7fbea2f89700 1 -- 172.21.15.160:0/752336109 wait complete. 2024-03-07T07:31:46.688 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:45.610+0000 7fbea2f89700 1 Processor -- start 2024-03-07T07:31:46.688 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:45.611+0000 7fbea2f89700 1 -- start start 2024-03-07T07:31:46.688 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:45.611+0000 7fbea2f89700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe9c07cd70 0x7fbe9c19dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:46.688 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:45.611+0000 7fbea2f89700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9c10e0f0 con 0x7fbe9c07cd70 2024-03-07T07:31:46.689 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:45.611+0000 7fbea0d25700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe9c07cd70 0x7fbe9c19dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:46.689 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:45.611+0000 7fbea0d25700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe9c07cd70 0x7fbe9c19dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59648/0 (socket says 172.21.15.160:59648) 2024-03-07T07:31:46.689 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:45.611+0000 7fbea0d25700 1 -- 172.21.15.160:0/1563892296 learned_addr learned my addr 172.21.15.160:0/1563892296 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:31:46.689 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:45.611+0000 7fbea0d25700 1 -- 172.21.15.160:0/1563892296 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe9c19e270 con 0x7fbe9c07cd70 2024-03-07T07:31:46.689 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:45.611+0000 7fbea0d25700 1 --2- 172.21.15.160:0/1563892296 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe9c07cd70 0x7fbe9c19dd30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbe90011670 tx=0x7fbe90011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:46.689 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:45.611+0000 7fbe99ffb700 1 -- 172.21.15.160:0/1563892296 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe900039c0 con 0x7fbe9c07cd70 2024-03-07T07:31:46.689 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:45.611+0000 7fbe99ffb700 1 -- 172.21.15.160:0/1563892296 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe90011e50 con 0x7fbe9c07cd70 2024-03-07T07:31:46.689 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:45.611+0000 7fbea2f89700 1 -- 172.21.15.160:0/1563892296 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe9c19e500 con 0x7fbe9c07cd70 2024-03-07T07:31:46.689 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:45.611+0000 7fbe99ffb700 1 -- 172.21.15.160:0/1563892296 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe9001a5c0 con 0x7fbe9c07cd70 2024-03-07T07:31:46.689 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:45.611+0000 7fbea2f89700 1 -- 172.21.15.160:0/1563892296 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe9c1a11f0 con 0x7fbe9c07cd70 2024-03-07T07:31:46.689 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:45.612+0000 7fbe99ffb700 1 -- 172.21.15.160:0/1563892296 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 8) v1 ==== 45927+0+0 (secure 0 0 0) 0x7fbe9001a870 con 0x7fbe9c07cd70 2024-03-07T07:31:46.689 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:45.612+0000 7fbe99ffb700 1 --2- 172.21.15.160:0/1563892296 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7fbe8c038ba0 0x7fbe8c03b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:46.689 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:45.612+0000 7fbe99ffb700 1 -- 172.21.15.160:0/1563892296 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fbe9004cfc0 con 0x7fbe9c07cd70 2024-03-07T07:31:46.689 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:45.612+0000 7fbea2f89700 1 -- 172.21.15.160:0/1563892296 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe9c063780 con 0x7fbe9c07cd70 2024-03-07T07:31:46.690 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:45.613+0000 7fbe9bfff700 1 --2- 172.21.15.160:0/1563892296 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7fbe8c038ba0 0x7fbe8c03b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:46.690 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:45.613+0000 7fbe9bfff700 1 --2- 172.21.15.160:0/1563892296 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7fbe8c038ba0 0x7fbe8c03b060 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbe88006fd0 tx=0x7fbe88006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:46.690 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:45.617+0000 7fbe99ffb700 1 -- 172.21.15.160:0/1563892296 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fbe90019c70 con 0x7fbe9c07cd70 2024-03-07T07:31:46.690 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:45.811+0000 7fbea2f89700 1 -- 172.21.15.160:0/1563892296 --> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe9c1042d0 con 0x7fbe8c038ba0 2024-03-07T07:31:46.690 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:46.116+0000 7fbe99ffb700 1 -- 172.21.15.160:0/1563892296 <== mgr.14120 v2:172.21.15.160:6800/3859174315 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fbe9c1042d0 con 0x7fbe8c038ba0 2024-03-07T07:31:46.690 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:46.118+0000 7fbea2f89700 1 -- 172.21.15.160:0/1563892296 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7fbe8c038ba0 msgr2=0x7fbe8c03b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:46.690 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:46.118+0000 7fbea2f89700 1 --2- 172.21.15.160:0/1563892296 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7fbe8c038ba0 0x7fbe8c03b060 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbe88006fd0 tx=0x7fbe88006e40 comp rx=0 tx=0).stop 2024-03-07T07:31:46.690 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:46.118+0000 7fbea2f89700 1 -- 172.21.15.160:0/1563892296 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe9c07cd70 msgr2=0x7fbe9c19dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:46.690 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:46.118+0000 7fbea2f89700 1 --2- 172.21.15.160:0/1563892296 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe9c07cd70 0x7fbe9c19dd30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbe90011670 tx=0x7fbe90011b60 comp rx=0 tx=0).stop 2024-03-07T07:31:46.690 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:46.119+0000 7fbea2f89700 1 -- 172.21.15.160:0/1563892296 shutdown_connections 2024-03-07T07:31:46.690 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:46.119+0000 7fbea2f89700 1 --2- 172.21.15.160:0/1563892296 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7fbe8c038ba0 0x7fbe8c03b060 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:46.690 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:46.119+0000 7fbea2f89700 1 --2- 172.21.15.160:0/1563892296 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe9c07cd70 0x7fbe9c19dd30 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:46.690 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:46.119+0000 7fbea2f89700 1 -- 172.21.15.160:0/1563892296 >> 172.21.15.160:0/1563892296 conn(0x7fbe9c103080 msgr2=0x7fbe9c103bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:31:46.690 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:46.119+0000 7fbea2f89700 1 -- 172.21.15.160:0/1563892296 shutdown_connections 2024-03-07T07:31:46.691 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:46.119+0000 7fbea2f89700 1 -- 172.21.15.160:0/1563892296 wait complete. 2024-03-07T07:31:47.310 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:47 smithi160 ceph-mon[31963]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:31:47.310 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:47 smithi160 ceph-mon[31963]: Generating ssh key... 2024-03-07T07:31:47.877 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDlldiONkKqY+THenf2nA7MYac60F5ox53Go88wJkC1j9uHHzwc7zD8GnFOIaj1Ztv/DgFLrUGhw0v8c0K6lzv57txMtfCMajOTJGmTiIjXejHm+kTP+mAfduB9Jy2m+wTN2NWhkaI3QLjKt0XYv+NZmeZLrwhQEaHiVwwzle5OQ+PAyaNeFllDJYgze+g09yJfOi6l0GBPdb40pZkdQIi1Ch1FGscB57fUxovgUypkwlzt1h8XrVJHVBYCzsArhxovd63RYV/DFprigczv3vJoSjNLnivyZfOeNLFj6beZq6svm3KbksdUS6vJVKE4OT7L9ppvu1mjHcWP1dCrNlvYjpwSI0rdRD3UVdXJ9KfNEVuL4IaWMKS2etuEImvQdiaf3t+KJuJ+INS7BuJjnRA3fSUmn/ImkRl3ccAiNW3mN0rV6d7bfKQfdal4oulh/HUwsvpOT+tkO8WTQdzoCJGDgStwUheNZvhIHcNtSzdyAT4SHsKMjpy2e90squcuYwM= ceph-826f6582-dc54-11ee-95c6-87774f69a715 2024-03-07T07:31:47.877 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.103+0000 7fc7965e3700 1 Processor -- start 2024-03-07T07:31:47.877 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.103+0000 7fc7965e3700 1 -- start start 2024-03-07T07:31:47.877 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.104+0000 7fc7965e3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7901085b0 0x7fc79010a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:47.877 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.104+0000 7fc7965e3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc79007be50 con 0x7fc7901085b0 2024-03-07T07:31:47.877 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.104+0000 7fc78ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7901085b0 0x7fc79010a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:47.878 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.104+0000 7fc78ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7901085b0 0x7fc79010a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59654/0 (socket says 172.21.15.160:59654) 2024-03-07T07:31:47.878 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.104+0000 7fc78ffff700 1 -- 172.21.15.160:0/46463022 learned_addr learned my addr 172.21.15.160:0/46463022 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:31:47.878 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.104+0000 7fc78ffff700 1 -- 172.21.15.160:0/46463022 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc79010aee0 con 0x7fc7901085b0 2024-03-07T07:31:47.878 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.104+0000 7fc78ffff700 1 --2- 172.21.15.160:0/46463022 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7901085b0 0x7fc79010a9a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc7780098a0 tx=0x7fc778009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=476035d15d5ad139 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:47.878 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.104+0000 7fc78effd700 1 -- 172.21.15.160:0/46463022 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc778004160 con 0x7fc7901085b0 2024-03-07T07:31:47.878 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.104+0000 7fc78effd700 1 -- 172.21.15.160:0/46463022 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc778004300 con 0x7fc7901085b0 2024-03-07T07:31:47.878 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.104+0000 7fc78effd700 1 -- 172.21.15.160:0/46463022 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc778011b50 con 0x7fc7901085b0 2024-03-07T07:31:47.878 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.105+0000 7fc7965e3700 1 -- 172.21.15.160:0/46463022 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7901085b0 msgr2=0x7fc79010a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:47.878 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.105+0000 7fc7965e3700 1 --2- 172.21.15.160:0/46463022 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7901085b0 0x7fc79010a9a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc7780098a0 tx=0x7fc778009bb0 comp rx=0 tx=0).stop 2024-03-07T07:31:47.878 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.105+0000 7fc7965e3700 1 -- 172.21.15.160:0/46463022 shutdown_connections 2024-03-07T07:31:47.878 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.105+0000 7fc7965e3700 1 --2- 172.21.15.160:0/46463022 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7901085b0 0x7fc79010a9a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:47.878 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.105+0000 7fc7965e3700 1 -- 172.21.15.160:0/46463022 >> 172.21.15.160:0/46463022 conn(0x7fc790101fc0 msgr2=0x7fc7901043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:31:47.878 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.105+0000 7fc7965e3700 1 -- 172.21.15.160:0/46463022 shutdown_connections 2024-03-07T07:31:47.879 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.105+0000 7fc7965e3700 1 -- 172.21.15.160:0/46463022 wait complete. 2024-03-07T07:31:47.879 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.105+0000 7fc7965e3700 1 Processor -- start 2024-03-07T07:31:47.879 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.106+0000 7fc7965e3700 1 -- start start 2024-03-07T07:31:47.879 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.106+0000 7fc7965e3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7901085b0 0x7fc790107e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:47.879 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.106+0000 7fc7965e3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc79010bf00 con 0x7fc7901085b0 2024-03-07T07:31:47.879 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.106+0000 7fc78ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7901085b0 0x7fc790107e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:47.879 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.106+0000 7fc78ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7901085b0 0x7fc790107e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59668/0 (socket says 172.21.15.160:59668) 2024-03-07T07:31:47.879 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.106+0000 7fc78ffff700 1 -- 172.21.15.160:0/279152286 learned_addr learned my addr 172.21.15.160:0/279152286 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:31:47.879 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.106+0000 7fc78ffff700 1 -- 172.21.15.160:0/279152286 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc790108370 con 0x7fc7901085b0 2024-03-07T07:31:47.879 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.106+0000 7fc78ffff700 1 --2- 172.21.15.160:0/279152286 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7901085b0 0x7fc790107e30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc778009fb0 tx=0x7fc778011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:47.879 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.107+0000 7fc78d7fa700 1 -- 172.21.15.160:0/279152286 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc7780038b0 con 0x7fc7901085b0 2024-03-07T07:31:47.879 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.107+0000 7fc78d7fa700 1 -- 172.21.15.160:0/279152286 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc778011e50 con 0x7fc7901085b0 2024-03-07T07:31:47.879 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.107+0000 7fc7965e3700 1 -- 172.21.15.160:0/279152286 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc790106560 con 0x7fc7901085b0 2024-03-07T07:31:47.880 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.107+0000 7fc78d7fa700 1 -- 172.21.15.160:0/279152286 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc778019420 con 0x7fc7901085b0 2024-03-07T07:31:47.880 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.107+0000 7fc7965e3700 1 -- 172.21.15.160:0/279152286 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc790106aa0 con 0x7fc7901085b0 2024-03-07T07:31:47.880 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.107+0000 7fc78d7fa700 1 -- 172.21.15.160:0/279152286 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 8) v1 ==== 45927+0+0 (secure 0 0 0) 0x7fc778019740 con 0x7fc7901085b0 2024-03-07T07:31:47.880 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.108+0000 7fc78d7fa700 1 --2- 172.21.15.160:0/279152286 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7fc77c038b00 0x7fc77c03afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:47.880 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.108+0000 7fc78d7fa700 1 -- 172.21.15.160:0/279152286 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc77801a8f0 con 0x7fc7901085b0 2024-03-07T07:31:47.880 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.108+0000 7fc78f7fe700 1 --2- 172.21.15.160:0/279152286 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7fc77c038b00 0x7fc77c03afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:47.880 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.108+0000 7fc7965e3700 1 -- 172.21.15.160:0/279152286 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc790063780 con 0x7fc7901085b0 2024-03-07T07:31:47.880 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.108+0000 7fc78f7fe700 1 --2- 172.21.15.160:0/279152286 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7fc77c038b00 0x7fc77c03afc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc780006fd0 tx=0x7fc780006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:47.880 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.113+0000 7fc78d7fa700 1 -- 172.21.15.160:0/279152286 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc778003b20 con 0x7fc7901085b0 2024-03-07T07:31:47.880 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.310+0000 7fc7965e3700 1 -- 172.21.15.160:0/279152286 --> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fc7901071b0 con 0x7fc77c038b00 2024-03-07T07:31:47.880 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.310+0000 7fc78d7fa700 1 -- 172.21.15.160:0/279152286 <== mgr.14120 v2:172.21.15.160:6800/3859174315 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fc7901071b0 con 0x7fc77c038b00 2024-03-07T07:31:47.881 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.312+0000 7fc7965e3700 1 -- 172.21.15.160:0/279152286 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7fc77c038b00 msgr2=0x7fc77c03afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:47.881 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.312+0000 7fc7965e3700 1 --2- 172.21.15.160:0/279152286 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7fc77c038b00 0x7fc77c03afc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc780006fd0 tx=0x7fc780006e40 comp rx=0 tx=0).stop 2024-03-07T07:31:47.881 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.312+0000 7fc7965e3700 1 -- 172.21.15.160:0/279152286 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7901085b0 msgr2=0x7fc790107e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:47.881 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.312+0000 7fc7965e3700 1 --2- 172.21.15.160:0/279152286 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7901085b0 0x7fc790107e30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc778009fb0 tx=0x7fc778011bf0 comp rx=0 tx=0).stop 2024-03-07T07:31:47.881 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.312+0000 7fc7965e3700 1 -- 172.21.15.160:0/279152286 shutdown_connections 2024-03-07T07:31:47.881 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.312+0000 7fc7965e3700 1 --2- 172.21.15.160:0/279152286 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7fc77c038b00 0x7fc77c03afc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:47.881 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.312+0000 7fc7965e3700 1 --2- 172.21.15.160:0/279152286 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7901085b0 0x7fc790107e30 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:47.881 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.312+0000 7fc7965e3700 1 -- 172.21.15.160:0/279152286 >> 172.21.15.160:0/279152286 conn(0x7fc790101fc0 msgr2=0x7fc790102bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:31:47.881 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.313+0000 7fc7965e3700 1 -- 172.21.15.160:0/279152286 shutdown_connections 2024-03-07T07:31:47.881 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:47.313+0000 7fc7965e3700 1 -- 172.21.15.160:0/279152286 wait complete. 2024-03-07T07:31:47.881 INFO:teuthology.orchestra.run.smithi160.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-07T07:31:47.881 INFO:teuthology.orchestra.run.smithi160.stdout:Adding key to root@localhost authorized_keys... 2024-03-07T07:31:47.882 INFO:teuthology.orchestra.run.smithi160.stdout:Adding host smithi160... 2024-03-07T07:31:48.610 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:48 smithi160 ceph-mon[31963]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:31:49.427 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:49 smithi160 ceph-mon[31963]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi160", "addr": "172.21.15.160", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:31:49.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:49 smithi160 ceph-mon[31963]: Deploying cephadm binary to smithi160 2024-03-07T07:31:53.276 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout Added host 'smithi160' with addr '172.21.15.160' 2024-03-07T07:31:53.276 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:48.286+0000 7efd0f4ae700 1 Processor -- start 2024-03-07T07:31:53.276 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:48.286+0000 7efd0f4ae700 1 -- start start 2024-03-07T07:31:53.277 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:48.287+0000 7efd0f4ae700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd08108520 0x7efd0810a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:53.277 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:48.287+0000 7efd0f4ae700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd080758c0 con 0x7efd08108520 2024-03-07T07:31:53.277 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:48.287+0000 7efd0d24a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd08108520 0x7efd0810a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:53.277 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:48.287+0000 7efd0d24a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd08108520 0x7efd0810a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59670/0 (socket says 172.21.15.160:59670) 2024-03-07T07:31:53.277 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:48.287+0000 7efd0d24a700 1 -- 172.21.15.160:0/133339178 learned_addr learned my addr 172.21.15.160:0/133339178 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:31:53.277 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:48.287+0000 7efd0d24a700 1 -- 172.21.15.160:0/133339178 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd0810ae50 con 0x7efd08108520 2024-03-07T07:31:53.277 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:48.287+0000 7efd0d24a700 1 --2- 172.21.15.160:0/133339178 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd08108520 0x7efd0810a910 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7efd040098a0 tx=0x7efd04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e77494e40fe77575 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:53.277 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:48.287+0000 7efcfffff700 1 -- 172.21.15.160:0/133339178 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd04004160 con 0x7efd08108520 2024-03-07T07:31:53.277 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:48.287+0000 7efcfffff700 1 -- 172.21.15.160:0/133339178 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efd04004300 con 0x7efd08108520 2024-03-07T07:31:53.277 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:48.287+0000 7efcfffff700 1 -- 172.21.15.160:0/133339178 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd04011b50 con 0x7efd08108520 2024-03-07T07:31:53.277 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:48.288+0000 7efd0f4ae700 1 -- 172.21.15.160:0/133339178 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd08108520 msgr2=0x7efd0810a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:53.277 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:48.288+0000 7efd0f4ae700 1 --2- 172.21.15.160:0/133339178 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd08108520 0x7efd0810a910 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7efd040098a0 tx=0x7efd04009bb0 comp rx=0 tx=0).stop 2024-03-07T07:31:53.277 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:48.288+0000 7efd0f4ae700 1 -- 172.21.15.160:0/133339178 shutdown_connections 2024-03-07T07:31:53.278 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:48.288+0000 7efd0f4ae700 1 --2- 172.21.15.160:0/133339178 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd08108520 0x7efd0810a910 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:53.278 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:48.288+0000 7efd0f4ae700 1 -- 172.21.15.160:0/133339178 >> 172.21.15.160:0/133339178 conn(0x7efd08101f70 msgr2=0x7efd08104390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:31:53.278 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:48.288+0000 7efd0f4ae700 1 -- 172.21.15.160:0/133339178 shutdown_connections 2024-03-07T07:31:53.278 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:48.288+0000 7efd0f4ae700 1 -- 172.21.15.160:0/133339178 wait complete. 2024-03-07T07:31:53.278 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:48.289+0000 7efd0f4ae700 1 Processor -- start 2024-03-07T07:31:53.278 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:48.289+0000 7efd0f4ae700 1 -- start start 2024-03-07T07:31:53.278 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:48.289+0000 7efd0f4ae700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd08108520 0x7efd0819bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:53.278 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:48.289+0000 7efd0f4ae700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd0810be80 con 0x7efd08108520 2024-03-07T07:31:53.278 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:48.289+0000 7efd0d24a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd08108520 0x7efd0819bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:53.278 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:48.289+0000 7efd0d24a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd08108520 0x7efd0819bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59684/0 (socket says 172.21.15.160:59684) 2024-03-07T07:31:53.278 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:48.289+0000 7efd0d24a700 1 -- 172.21.15.160:0/3765533100 learned_addr learned my addr 172.21.15.160:0/3765533100 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:31:53.278 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:48.289+0000 7efd0d24a700 1 -- 172.21.15.160:0/3765533100 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd0819bff0 con 0x7efd08108520 2024-03-07T07:31:53.278 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:48.289+0000 7efd0d24a700 1 --2- 172.21.15.160:0/3765533100 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd08108520 0x7efd0819bab0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7efd04011b60 tx=0x7efd04011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:53.278 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:48.290+0000 7efcfe7fc700 1 -- 172.21.15.160:0/3765533100 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd04003970 con 0x7efd08108520 2024-03-07T07:31:53.278 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:48.290+0000 7efcfe7fc700 1 -- 172.21.15.160:0/3765533100 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efd04003b10 con 0x7efd08108520 2024-03-07T07:31:53.279 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:48.290+0000 7efd0f4ae700 1 -- 172.21.15.160:0/3765533100 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd0819c280 con 0x7efd08108520 2024-03-07T07:31:53.279 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:48.290+0000 7efcfe7fc700 1 -- 172.21.15.160:0/3765533100 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd04024610 con 0x7efd08108520 2024-03-07T07:31:53.279 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:48.290+0000 7efd0f4ae700 1 -- 172.21.15.160:0/3765533100 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd0819ef70 con 0x7efd08108520 2024-03-07T07:31:53.279 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:48.290+0000 7efcfe7fc700 1 -- 172.21.15.160:0/3765533100 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 8) v1 ==== 45927+0+0 (secure 0 0 0) 0x7efd040248c0 con 0x7efd08108520 2024-03-07T07:31:53.279 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:48.291+0000 7efd0f4ae700 1 -- 172.21.15.160:0/3765533100 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd08063780 con 0x7efd08108520 2024-03-07T07:31:53.279 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:48.291+0000 7efcfe7fc700 1 --2- 172.21.15.160:0/3765533100 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7efcf4038bf0 0x7efcf403b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:53.279 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:48.291+0000 7efcfe7fc700 1 -- 172.21.15.160:0/3765533100 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7efd04011e00 con 0x7efd08108520 2024-03-07T07:31:53.279 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:48.291+0000 7efd0ca49700 1 --2- 172.21.15.160:0/3765533100 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7efcf4038bf0 0x7efcf403b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:53.279 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:48.291+0000 7efd0ca49700 1 --2- 172.21.15.160:0/3765533100 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7efcf4038bf0 0x7efcf403b0b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7efcf8006fd0 tx=0x7efcf8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:53.279 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:48.296+0000 7efcfe7fc700 1 -- 172.21.15.160:0/3765533100 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7efd0401a770 con 0x7efd08108520 2024-03-07T07:31:53.279 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:48.500+0000 7efd0f4ae700 1 -- 172.21.15.160:0/3765533100 --> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi160", "addr": "172.21.15.160", "target": ["mon-mgr", ""]}) v1 -- 0x7efd081031b0 con 0x7efcf4038bf0 2024-03-07T07:31:53.279 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:52.658+0000 7efcfe7fc700 1 -- 172.21.15.160:0/3765533100 <== mgr.14120 v2:172.21.15.160:6800/3859174315 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7efd081031b0 con 0x7efcf4038bf0 2024-03-07T07:31:53.279 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:52.661+0000 7efd0f4ae700 1 -- 172.21.15.160:0/3765533100 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7efcf4038bf0 msgr2=0x7efcf403b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:53.279 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:52.661+0000 7efd0f4ae700 1 --2- 172.21.15.160:0/3765533100 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7efcf4038bf0 0x7efcf403b0b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7efcf8006fd0 tx=0x7efcf8006e40 comp rx=0 tx=0).stop 2024-03-07T07:31:53.280 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:52.661+0000 7efd0f4ae700 1 -- 172.21.15.160:0/3765533100 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd08108520 msgr2=0x7efd0819bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:53.280 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:52.662+0000 7efd0f4ae700 1 --2- 172.21.15.160:0/3765533100 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd08108520 0x7efd0819bab0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7efd04011b60 tx=0x7efd04011c40 comp rx=0 tx=0).stop 2024-03-07T07:31:53.280 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:52.662+0000 7efd0f4ae700 1 -- 172.21.15.160:0/3765533100 shutdown_connections 2024-03-07T07:31:53.280 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:52.662+0000 7efd0f4ae700 1 --2- 172.21.15.160:0/3765533100 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7efcf4038bf0 0x7efcf403b0b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:53.280 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:52.662+0000 7efd0f4ae700 1 --2- 172.21.15.160:0/3765533100 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd08108520 0x7efd0819bab0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:53.280 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:52.662+0000 7efd0f4ae700 1 -- 172.21.15.160:0/3765533100 >> 172.21.15.160:0/3765533100 conn(0x7efd08101f70 msgr2=0x7efd08102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:31:53.280 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:52.662+0000 7efd0f4ae700 1 -- 172.21.15.160:0/3765533100 shutdown_connections 2024-03-07T07:31:53.280 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:52.662+0000 7efd0f4ae700 1 -- 172.21.15.160:0/3765533100 wait complete. 2024-03-07T07:31:53.280 INFO:teuthology.orchestra.run.smithi160.stdout:Deploying unmanaged mon service... 2024-03-07T07:31:53.927 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:53 smithi160 ceph-mon[31963]: from='mgr.14120 172.21.15.160:0/1092429691' entity='mgr.x' 2024-03-07T07:31:53.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:53 smithi160 ceph-mon[31963]: Added host smithi160 2024-03-07T07:31:53.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:53 smithi160 ceph-mon[31963]: from='mgr.14120 172.21.15.160:0/1092429691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:31:54.693 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-07T07:31:54.694 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.732+0000 7fbed0969700 1 Processor -- start 2024-03-07T07:31:54.694 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.732+0000 7fbed0969700 1 -- start start 2024-03-07T07:31:54.694 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.732+0000 7fbed0969700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbecc106ea0 0x7fbecc1072c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:54.694 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.732+0000 7fbed0969700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc107890 con 0x7fbecc106ea0 2024-03-07T07:31:54.694 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.733+0000 7fbeca59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbecc106ea0 0x7fbecc1072c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:54.694 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.733+0000 7fbeca59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbecc106ea0 0x7fbecc1072c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44054/0 (socket says 172.21.15.160:44054) 2024-03-07T07:31:54.694 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.733+0000 7fbeca59c700 1 -- 172.21.15.160:0/3306490055 learned_addr learned my addr 172.21.15.160:0/3306490055 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:31:54.694 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.733+0000 7fbeca59c700 1 -- 172.21.15.160:0/3306490055 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbecc107bc0 con 0x7fbecc106ea0 2024-03-07T07:31:54.694 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.733+0000 7fbeca59c700 1 --2- 172.21.15.160:0/3306490055 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbecc106ea0 0x7fbecc1072c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbeb400bb40 tx=0x7fbeb400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d90b976fabed78f server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:54.694 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.733+0000 7fbec959a700 1 -- 172.21.15.160:0/3306490055 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbeb400dbe0 con 0x7fbecc106ea0 2024-03-07T07:31:54.694 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.734+0000 7fbec959a700 1 -- 172.21.15.160:0/3306490055 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbeb40044f0 con 0x7fbecc106ea0 2024-03-07T07:31:54.694 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.734+0000 7fbec959a700 1 -- 172.21.15.160:0/3306490055 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbeb4010470 con 0x7fbecc106ea0 2024-03-07T07:31:54.695 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.734+0000 7fbed0969700 1 -- 172.21.15.160:0/3306490055 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbecc106ea0 msgr2=0x7fbecc1072c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:54.695 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.734+0000 7fbed0969700 1 --2- 172.21.15.160:0/3306490055 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbecc106ea0 0x7fbecc1072c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbeb400bb40 tx=0x7fbeb400be50 comp rx=0 tx=0).stop 2024-03-07T07:31:54.695 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.734+0000 7fbed0969700 1 -- 172.21.15.160:0/3306490055 shutdown_connections 2024-03-07T07:31:54.695 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.734+0000 7fbed0969700 1 --2- 172.21.15.160:0/3306490055 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbecc106ea0 0x7fbecc1072c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:54.695 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.734+0000 7fbed0969700 1 -- 172.21.15.160:0/3306490055 >> 172.21.15.160:0/3306490055 conn(0x7fbecc101740 msgr2=0x7fbecc103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:31:54.695 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.734+0000 7fbed0969700 1 -- 172.21.15.160:0/3306490055 shutdown_connections 2024-03-07T07:31:54.695 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.734+0000 7fbed0969700 1 -- 172.21.15.160:0/3306490055 wait complete. 2024-03-07T07:31:54.695 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.735+0000 7fbed0969700 1 Processor -- start 2024-03-07T07:31:54.695 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.735+0000 7fbed0969700 1 -- start start 2024-03-07T07:31:54.695 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.735+0000 7fbed0969700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbecc106ea0 0x7fbecc19dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:54.695 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.735+0000 7fbed0969700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc105a00 con 0x7fbecc106ea0 2024-03-07T07:31:54.696 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.735+0000 7fbeca59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbecc106ea0 0x7fbecc19dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:54.696 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.735+0000 7fbeca59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbecc106ea0 0x7fbecc19dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44068/0 (socket says 172.21.15.160:44068) 2024-03-07T07:31:54.696 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.735+0000 7fbeca59c700 1 -- 172.21.15.160:0/2262826891 learned_addr learned my addr 172.21.15.160:0/2262826891 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:31:54.696 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.736+0000 7fbeca59c700 1 -- 172.21.15.160:0/2262826891 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbecc19e1f0 con 0x7fbecc106ea0 2024-03-07T07:31:54.696 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.736+0000 7fbeca59c700 1 --2- 172.21.15.160:0/2262826891 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbecc106ea0 0x7fbecc19dcb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbeb400b340 tx=0x7fbeb400dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:54.696 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.736+0000 7fbec37fe700 1 -- 172.21.15.160:0/2262826891 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbeb4010680 con 0x7fbecc106ea0 2024-03-07T07:31:54.696 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.736+0000 7fbed0969700 1 -- 172.21.15.160:0/2262826891 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbecc19e480 con 0x7fbecc106ea0 2024-03-07T07:31:54.696 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.736+0000 7fbec37fe700 1 -- 172.21.15.160:0/2262826891 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbeb401d070 con 0x7fbecc106ea0 2024-03-07T07:31:54.697 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.736+0000 7fbec37fe700 1 -- 172.21.15.160:0/2262826891 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbeb4021af0 con 0x7fbecc106ea0 2024-03-07T07:31:54.697 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.736+0000 7fbed0969700 1 -- 172.21.15.160:0/2262826891 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbecc1a1170 con 0x7fbecc106ea0 2024-03-07T07:31:54.697 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.737+0000 7fbec37fe700 1 -- 172.21.15.160:0/2262826891 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 8) v1 ==== 45927+0+0 (secure 0 0 0) 0x7fbeb401f030 con 0x7fbecc106ea0 2024-03-07T07:31:54.697 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.737+0000 7fbed0969700 1 -- 172.21.15.160:0/2262826891 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbecc063780 con 0x7fbecc106ea0 2024-03-07T07:31:54.697 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.737+0000 7fbec37fe700 1 --2- 172.21.15.160:0/2262826891 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7fbeb8038ba0 0x7fbeb803b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:54.697 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.737+0000 7fbec37fe700 1 -- 172.21.15.160:0/2262826891 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fbeb404cc10 con 0x7fbecc106ea0 2024-03-07T07:31:54.697 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.737+0000 7fbec9d9b700 1 --2- 172.21.15.160:0/2262826891 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7fbeb8038ba0 0x7fbeb803b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:54.697 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.738+0000 7fbec9d9b700 1 --2- 172.21.15.160:0/2262826891 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7fbeb8038ba0 0x7fbeb803b060 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbebc006fd0 tx=0x7fbebc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:54.697 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.742+0000 7fbec37fe700 1 -- 172.21.15.160:0/2262826891 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fbeb4015070 con 0x7fbecc106ea0 2024-03-07T07:31:54.697 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.959+0000 7fbed0969700 1 -- 172.21.15.160:0/2262826891 --> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fbecc062590 con 0x7fbeb8038ba0 2024-03-07T07:31:54.698 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.964+0000 7fbec37fe700 1 -- 172.21.15.160:0/2262826891 <== mgr.14120 v2:172.21.15.160:6800/3859174315 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fbecc062590 con 0x7fbeb8038ba0 2024-03-07T07:31:54.698 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.966+0000 7fbed0969700 1 -- 172.21.15.160:0/2262826891 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7fbeb8038ba0 msgr2=0x7fbeb803b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:54.698 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.966+0000 7fbed0969700 1 --2- 172.21.15.160:0/2262826891 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7fbeb8038ba0 0x7fbeb803b060 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbebc006fd0 tx=0x7fbebc006e40 comp rx=0 tx=0).stop 2024-03-07T07:31:54.698 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.966+0000 7fbed0969700 1 -- 172.21.15.160:0/2262826891 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbecc106ea0 msgr2=0x7fbecc19dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:54.698 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.966+0000 7fbed0969700 1 --2- 172.21.15.160:0/2262826891 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbecc106ea0 0x7fbecc19dcb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbeb400b340 tx=0x7fbeb400dd90 comp rx=0 tx=0).stop 2024-03-07T07:31:54.698 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.966+0000 7fbed0969700 1 -- 172.21.15.160:0/2262826891 shutdown_connections 2024-03-07T07:31:54.698 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.966+0000 7fbed0969700 1 --2- 172.21.15.160:0/2262826891 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7fbeb8038ba0 0x7fbeb803b060 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:54.698 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.966+0000 7fbed0969700 1 --2- 172.21.15.160:0/2262826891 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbecc106ea0 0x7fbecc19dcb0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:54.698 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.966+0000 7fbed0969700 1 -- 172.21.15.160:0/2262826891 >> 172.21.15.160:0/2262826891 conn(0x7fbecc101740 msgr2=0x7fbecc18fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:31:54.698 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.966+0000 7fbed0969700 1 -- 172.21.15.160:0/2262826891 shutdown_connections 2024-03-07T07:31:54.698 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:53.966+0000 7fbed0969700 1 -- 172.21.15.160:0/2262826891 wait complete. 2024-03-07T07:31:54.699 INFO:teuthology.orchestra.run.smithi160.stdout:Deploying unmanaged mgr service... 2024-03-07T07:31:55.221 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:54 smithi160 ceph-mon[31963]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:31:55.222 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:54 smithi160 ceph-mon[31963]: Saving service mon spec with placement count:5 2024-03-07T07:31:55.222 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:54 smithi160 ceph-mon[31963]: from='mgr.14120 172.21.15.160:0/1092429691' entity='mgr.x' 2024-03-07T07:31:56.646 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:56 smithi160 ceph-mon[31963]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:31:56.647 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:56 smithi160 ceph-mon[31963]: Saving service mgr spec with placement count:2 2024-03-07T07:31:56.647 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:56 smithi160 ceph-mon[31963]: from='mgr.14120 172.21.15.160:0/1092429691' entity='mgr.x' 2024-03-07T07:31:56.647 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:56 smithi160 ceph-mon[31963]: from='mgr.14120 172.21.15.160:0/1092429691' entity='mgr.x' 2024-03-07T07:31:56.654 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-07T07:31:56.654 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.091+0000 7f65f41e2700 1 Processor -- start 2024-03-07T07:31:56.654 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.091+0000 7f65f41e2700 1 -- start start 2024-03-07T07:31:56.654 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.091+0000 7f65f41e2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65ec108550 0x7f65ec10a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:56.654 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.091+0000 7f65f41e2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ec07be50 con 0x7f65ec108550 2024-03-07T07:31:56.654 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.091+0000 7f65f1f7e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65ec108550 0x7f65ec10a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:56.654 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.091+0000 7f65f1f7e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65ec108550 0x7f65ec10a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44084/0 (socket says 172.21.15.160:44084) 2024-03-07T07:31:56.654 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.091+0000 7f65f1f7e700 1 -- 172.21.15.160:0/4158522001 learned_addr learned my addr 172.21.15.160:0/4158522001 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:31:56.654 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.092+0000 7f65f1f7e700 1 -- 172.21.15.160:0/4158522001 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65ec10ae80 con 0x7f65ec108550 2024-03-07T07:31:56.655 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.092+0000 7f65f1f7e700 1 --2- 172.21.15.160:0/4158522001 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65ec108550 0x7f65ec10a940 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f65e80098a0 tx=0x7f65e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cce43b0a0432918c server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:56.655 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.092+0000 7f65f0f7c700 1 -- 172.21.15.160:0/4158522001 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65e8004160 con 0x7f65ec108550 2024-03-07T07:31:56.655 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.092+0000 7f65f0f7c700 1 -- 172.21.15.160:0/4158522001 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f65e8004300 con 0x7f65ec108550 2024-03-07T07:31:56.655 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.092+0000 7f65f0f7c700 1 -- 172.21.15.160:0/4158522001 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65e8011b50 con 0x7f65ec108550 2024-03-07T07:31:56.655 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.093+0000 7f65f41e2700 1 -- 172.21.15.160:0/4158522001 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65ec108550 msgr2=0x7f65ec10a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:56.655 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.093+0000 7f65f41e2700 1 --2- 172.21.15.160:0/4158522001 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65ec108550 0x7f65ec10a940 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f65e80098a0 tx=0x7f65e8009bb0 comp rx=0 tx=0).stop 2024-03-07T07:31:56.655 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.093+0000 7f65f41e2700 1 -- 172.21.15.160:0/4158522001 shutdown_connections 2024-03-07T07:31:56.655 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.093+0000 7f65f41e2700 1 --2- 172.21.15.160:0/4158522001 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65ec108550 0x7f65ec10a940 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:56.655 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.093+0000 7f65f41e2700 1 -- 172.21.15.160:0/4158522001 >> 172.21.15.160:0/4158522001 conn(0x7f65ec101f40 msgr2=0x7f65ec104380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:31:56.655 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.093+0000 7f65f41e2700 1 -- 172.21.15.160:0/4158522001 shutdown_connections 2024-03-07T07:31:56.655 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.093+0000 7f65f41e2700 1 -- 172.21.15.160:0/4158522001 wait complete. 2024-03-07T07:31:56.655 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.093+0000 7f65f41e2700 1 Processor -- start 2024-03-07T07:31:56.655 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.093+0000 7f65f41e2700 1 -- start start 2024-03-07T07:31:56.655 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.094+0000 7f65f41e2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65ec108550 0x7f65ec19ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:56.656 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.094+0000 7f65f41e2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ec10bea0 con 0x7f65ec108550 2024-03-07T07:31:56.656 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.094+0000 7f65f1f7e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65ec108550 0x7f65ec19ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:56.656 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.094+0000 7f65f1f7e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65ec108550 0x7f65ec19ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44088/0 (socket says 172.21.15.160:44088) 2024-03-07T07:31:56.656 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.094+0000 7f65f1f7e700 1 -- 172.21.15.160:0/855381395 learned_addr learned my addr 172.21.15.160:0/855381395 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:31:56.656 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.094+0000 7f65f1f7e700 1 -- 172.21.15.160:0/855381395 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65ec19bf90 con 0x7f65ec108550 2024-03-07T07:31:56.656 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.094+0000 7f65f1f7e700 1 --2- 172.21.15.160:0/855381395 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65ec108550 0x7f65ec19ba50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f65e8011670 tx=0x7f65e8011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:56.656 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.094+0000 7f65e2ffd700 1 -- 172.21.15.160:0/855381395 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65e80039c0 con 0x7f65ec108550 2024-03-07T07:31:56.656 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.094+0000 7f65e2ffd700 1 -- 172.21.15.160:0/855381395 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f65e8011e50 con 0x7f65ec108550 2024-03-07T07:31:56.656 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.094+0000 7f65f41e2700 1 -- 172.21.15.160:0/855381395 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65ec19c220 con 0x7f65ec108550 2024-03-07T07:31:56.656 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.094+0000 7f65e2ffd700 1 -- 172.21.15.160:0/855381395 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65e801a5c0 con 0x7f65ec108550 2024-03-07T07:31:56.656 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.094+0000 7f65f41e2700 1 -- 172.21.15.160:0/855381395 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65ec19ef10 con 0x7f65ec108550 2024-03-07T07:31:56.656 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.095+0000 7f65e2ffd700 1 -- 172.21.15.160:0/855381395 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 8) v1 ==== 45927+0+0 (secure 0 0 0) 0x7f65e801a870 con 0x7f65ec108550 2024-03-07T07:31:56.656 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.095+0000 7f65f41e2700 1 -- 172.21.15.160:0/855381395 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65ec063780 con 0x7f65ec108550 2024-03-07T07:31:56.656 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.095+0000 7f65e2ffd700 1 --2- 172.21.15.160:0/855381395 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7f65d8038ba0 0x7f65d803b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:56.657 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.096+0000 7f65e2ffd700 1 -- 172.21.15.160:0/855381395 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f65e804cd10 con 0x7f65ec108550 2024-03-07T07:31:56.657 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.096+0000 7f65f177d700 1 --2- 172.21.15.160:0/855381395 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7f65d8038ba0 0x7f65d803b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:56.657 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.096+0000 7f65f177d700 1 --2- 172.21.15.160:0/855381395 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7f65d8038ba0 0x7f65d803b060 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f65dc006fd0 tx=0x7f65dc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:56.657 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.100+0000 7f65e2ffd700 1 -- 172.21.15.160:0/855381395 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f65e8016260 con 0x7f65ec108550 2024-03-07T07:31:56.657 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.318+0000 7f65f41e2700 1 -- 172.21.15.160:0/855381395 --> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f65ec103140 con 0x7f65d8038ba0 2024-03-07T07:31:56.657 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.326+0000 7f65e2ffd700 1 -- 172.21.15.160:0/855381395 <== mgr.14120 v2:172.21.15.160:6800/3859174315 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f65ec103140 con 0x7f65d8038ba0 2024-03-07T07:31:56.657 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.328+0000 7f65f41e2700 1 -- 172.21.15.160:0/855381395 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7f65d8038ba0 msgr2=0x7f65d803b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:56.657 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.328+0000 7f65f41e2700 1 --2- 172.21.15.160:0/855381395 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7f65d8038ba0 0x7f65d803b060 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f65dc006fd0 tx=0x7f65dc006e40 comp rx=0 tx=0).stop 2024-03-07T07:31:56.657 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.328+0000 7f65f41e2700 1 -- 172.21.15.160:0/855381395 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65ec108550 msgr2=0x7f65ec19ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:56.657 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.328+0000 7f65f41e2700 1 --2- 172.21.15.160:0/855381395 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65ec108550 0x7f65ec19ba50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f65e8011670 tx=0x7f65e8011b60 comp rx=0 tx=0).stop 2024-03-07T07:31:56.657 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.328+0000 7f65f41e2700 1 -- 172.21.15.160:0/855381395 shutdown_connections 2024-03-07T07:31:56.657 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.328+0000 7f65f41e2700 1 --2- 172.21.15.160:0/855381395 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7f65d8038ba0 0x7f65d803b060 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:56.657 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.328+0000 7f65f41e2700 1 --2- 172.21.15.160:0/855381395 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65ec108550 0x7f65ec19ba50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:56.657 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.328+0000 7f65f41e2700 1 -- 172.21.15.160:0/855381395 >> 172.21.15.160:0/855381395 conn(0x7f65ec101f40 msgr2=0x7f65ec102a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:31:56.657 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.328+0000 7f65f41e2700 1 -- 172.21.15.160:0/855381395 shutdown_connections 2024-03-07T07:31:56.658 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:55.328+0000 7f65f41e2700 1 -- 172.21.15.160:0/855381395 wait complete. 2024-03-07T07:31:58.016 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:57 smithi160 ceph-mon[31963]: from='mgr.14120 172.21.15.160:0/1092429691' entity='mgr.x' 2024-03-07T07:31:58.017 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:57 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/2492431495' entity='client.admin' 2024-03-07T07:31:58.547 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.122+0000 7fbe4500a700 1 Processor -- start 2024-03-07T07:31:58.547 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.122+0000 7fbe4500a700 1 -- start start 2024-03-07T07:31:58.547 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.122+0000 7fbe4500a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe401085c0 0x7fbe4010a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:58.548 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.122+0000 7fbe4500a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4007be50 con 0x7fbe401085c0 2024-03-07T07:31:58.548 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.122+0000 7fbe3ed9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe401085c0 0x7fbe4010a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:58.548 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.122+0000 7fbe3ed9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe401085c0 0x7fbe4010a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44090/0 (socket says 172.21.15.160:44090) 2024-03-07T07:31:58.548 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.122+0000 7fbe3ed9d700 1 -- 172.21.15.160:0/3037881206 learned_addr learned my addr 172.21.15.160:0/3037881206 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:31:58.548 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.122+0000 7fbe3ed9d700 1 -- 172.21.15.160:0/3037881206 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe4010aef0 con 0x7fbe401085c0 2024-03-07T07:31:58.548 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.122+0000 7fbe3ed9d700 1 --2- 172.21.15.160:0/3037881206 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe401085c0 0x7fbe4010a9b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbe300098a0 tx=0x7fbe30009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e7bd32bacdaefa4 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:58.548 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.123+0000 7fbe3dd9b700 1 -- 172.21.15.160:0/3037881206 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe30004160 con 0x7fbe401085c0 2024-03-07T07:31:58.548 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.123+0000 7fbe3dd9b700 1 -- 172.21.15.160:0/3037881206 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbe30004300 con 0x7fbe401085c0 2024-03-07T07:31:58.548 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.123+0000 7fbe3dd9b700 1 -- 172.21.15.160:0/3037881206 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe30011b20 con 0x7fbe401085c0 2024-03-07T07:31:58.548 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.123+0000 7fbe4500a700 1 -- 172.21.15.160:0/3037881206 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe401085c0 msgr2=0x7fbe4010a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:58.548 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.123+0000 7fbe4500a700 1 --2- 172.21.15.160:0/3037881206 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe401085c0 0x7fbe4010a9b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbe300098a0 tx=0x7fbe30009bb0 comp rx=0 tx=0).stop 2024-03-07T07:31:58.549 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.123+0000 7fbe4500a700 1 -- 172.21.15.160:0/3037881206 shutdown_connections 2024-03-07T07:31:58.549 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.123+0000 7fbe4500a700 1 --2- 172.21.15.160:0/3037881206 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe401085c0 0x7fbe4010a9b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:58.549 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.123+0000 7fbe4500a700 1 -- 172.21.15.160:0/3037881206 >> 172.21.15.160:0/3037881206 conn(0x7fbe40101ff0 msgr2=0x7fbe40104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:31:58.549 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.123+0000 7fbe4500a700 1 -- 172.21.15.160:0/3037881206 shutdown_connections 2024-03-07T07:31:58.549 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.123+0000 7fbe4500a700 1 -- 172.21.15.160:0/3037881206 wait complete. 2024-03-07T07:31:58.549 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.124+0000 7fbe4500a700 1 Processor -- start 2024-03-07T07:31:58.549 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.124+0000 7fbe4500a700 1 -- start start 2024-03-07T07:31:58.549 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.124+0000 7fbe4500a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe401085c0 0x7fbe4019ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:58.549 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.124+0000 7fbe4500a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4010bf50 con 0x7fbe401085c0 2024-03-07T07:31:58.549 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.125+0000 7fbe3ed9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe401085c0 0x7fbe4019ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:58.549 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.125+0000 7fbe3ed9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe401085c0 0x7fbe4019ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44102/0 (socket says 172.21.15.160:44102) 2024-03-07T07:31:58.549 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.125+0000 7fbe3ed9d700 1 -- 172.21.15.160:0/2492431495 learned_addr learned my addr 172.21.15.160:0/2492431495 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:31:58.549 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.125+0000 7fbe3ed9d700 1 -- 172.21.15.160:0/2492431495 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe4019bfb0 con 0x7fbe401085c0 2024-03-07T07:31:58.549 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.125+0000 7fbe3ed9d700 1 --2- 172.21.15.160:0/2492431495 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe401085c0 0x7fbe4019ba70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbe30011b60 tx=0x7fbe30011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:58.550 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.125+0000 7fbe27fff700 1 -- 172.21.15.160:0/2492431495 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe30003b60 con 0x7fbe401085c0 2024-03-07T07:31:58.550 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.125+0000 7fbe4500a700 1 -- 172.21.15.160:0/2492431495 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe4019c240 con 0x7fbe401085c0 2024-03-07T07:31:58.550 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.125+0000 7fbe27fff700 1 -- 172.21.15.160:0/2492431495 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbe3001a910 con 0x7fbe401085c0 2024-03-07T07:31:58.550 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.125+0000 7fbe27fff700 1 -- 172.21.15.160:0/2492431495 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe30024930 con 0x7fbe401085c0 2024-03-07T07:31:58.550 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.125+0000 7fbe4500a700 1 -- 172.21.15.160:0/2492431495 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe4019c6a0 con 0x7fbe401085c0 2024-03-07T07:31:58.550 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.126+0000 7fbe27fff700 1 -- 172.21.15.160:0/2492431495 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 8) v1 ==== 45927+0+0 (secure 0 0 0) 0x7fbe30022020 con 0x7fbe401085c0 2024-03-07T07:31:58.550 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.126+0000 7fbe4500a700 1 -- 172.21.15.160:0/2492431495 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe40063780 con 0x7fbe401085c0 2024-03-07T07:31:58.550 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.127+0000 7fbe27fff700 1 --2- 172.21.15.160:0/2492431495 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7fbe28038bf0 0x7fbe2803b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:58.550 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.127+0000 7fbe27fff700 1 -- 172.21.15.160:0/2492431495 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fbe3004ce70 con 0x7fbe401085c0 2024-03-07T07:31:58.550 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.127+0000 7fbe3e59c700 1 --2- 172.21.15.160:0/2492431495 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7fbe28038bf0 0x7fbe2803b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:58.550 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.127+0000 7fbe3e59c700 1 --2- 172.21.15.160:0/2492431495 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7fbe28038bf0 0x7fbe2803b0b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbe34006fd0 tx=0x7fbe34006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:58.550 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.136+0000 7fbe27fff700 1 -- 172.21.15.160:0/2492431495 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fbe3000bc70 con 0x7fbe401085c0 2024-03-07T07:31:58.550 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.370+0000 7fbe4500a700 1 -- 172.21.15.160:0/2492431495 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7fbe4019ef30 con 0x7fbe401085c0 2024-03-07T07:31:58.550 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.373+0000 7fbe27fff700 1 -- 172.21.15.160:0/2492431495 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7fbe3001ab30 con 0x7fbe401085c0 2024-03-07T07:31:58.550 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.375+0000 7fbe4500a700 1 -- 172.21.15.160:0/2492431495 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7fbe28038bf0 msgr2=0x7fbe2803b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:58.551 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.375+0000 7fbe4500a700 1 --2- 172.21.15.160:0/2492431495 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7fbe28038bf0 0x7fbe2803b0b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbe34006fd0 tx=0x7fbe34006e40 comp rx=0 tx=0).stop 2024-03-07T07:31:58.551 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.375+0000 7fbe4500a700 1 -- 172.21.15.160:0/2492431495 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe401085c0 msgr2=0x7fbe4019ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:58.551 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.375+0000 7fbe4500a700 1 --2- 172.21.15.160:0/2492431495 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe401085c0 0x7fbe4019ba70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbe30011b60 tx=0x7fbe30011c40 comp rx=0 tx=0).stop 2024-03-07T07:31:58.551 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.375+0000 7fbe4500a700 1 -- 172.21.15.160:0/2492431495 shutdown_connections 2024-03-07T07:31:58.551 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.375+0000 7fbe4500a700 1 --2- 172.21.15.160:0/2492431495 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7fbe28038bf0 0x7fbe2803b0b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:58.551 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.375+0000 7fbe4500a700 1 --2- 172.21.15.160:0/2492431495 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe401085c0 0x7fbe4019ba70 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:58.551 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.375+0000 7fbe4500a700 1 -- 172.21.15.160:0/2492431495 >> 172.21.15.160:0/2492431495 conn(0x7fbe40101ff0 msgr2=0x7fbe40102ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:31:58.551 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.376+0000 7fbe4500a700 1 -- 172.21.15.160:0/2492431495 shutdown_connections 2024-03-07T07:31:58.551 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:57.376+0000 7fbe4500a700 1 -- 172.21.15.160:0/2492431495 wait complete. 2024-03-07T07:31:59.177 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:31:59 smithi160 ceph-mon[31963]: from='mgr.14120 172.21.15.160:0/1092429691' entity='mgr.x' 2024-03-07T07:31:59.803 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.013+0000 7fea028d9700 1 Processor -- start 2024-03-07T07:31:59.803 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.013+0000 7fea028d9700 1 -- start start 2024-03-07T07:31:59.803 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.014+0000 7fea028d9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9fc1085f0 0x7fe9fc10a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:59.803 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.014+0000 7fea028d9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc07bdb0 con 0x7fe9fc1085f0 2024-03-07T07:31:59.803 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.014+0000 7fe9fbfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9fc1085f0 0x7fe9fc10a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:59.803 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.014+0000 7fe9fbfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9fc1085f0 0x7fe9fc10a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44106/0 (socket says 172.21.15.160:44106) 2024-03-07T07:31:59.803 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.014+0000 7fe9fbfff700 1 -- 172.21.15.160:0/2157475093 learned_addr learned my addr 172.21.15.160:0/2157475093 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:31:59.803 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.014+0000 7fe9fbfff700 1 -- 172.21.15.160:0/2157475093 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9fc10af20 con 0x7fe9fc1085f0 2024-03-07T07:31:59.804 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.014+0000 7fe9fbfff700 1 --2- 172.21.15.160:0/2157475093 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9fc1085f0 0x7fe9fc10a9e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe9f00098a0 tx=0x7fe9f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc38e0f0838febdf server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:59.804 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.015+0000 7fe9faffd700 1 -- 172.21.15.160:0/2157475093 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe9f0004160 con 0x7fe9fc1085f0 2024-03-07T07:31:59.804 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.015+0000 7fe9faffd700 1 -- 172.21.15.160:0/2157475093 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe9f0004300 con 0x7fe9fc1085f0 2024-03-07T07:31:59.804 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.015+0000 7fe9faffd700 1 -- 172.21.15.160:0/2157475093 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe9f0011b20 con 0x7fe9fc1085f0 2024-03-07T07:31:59.804 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.015+0000 7fea028d9700 1 -- 172.21.15.160:0/2157475093 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9fc1085f0 msgr2=0x7fe9fc10a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:59.804 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.015+0000 7fea028d9700 1 --2- 172.21.15.160:0/2157475093 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9fc1085f0 0x7fe9fc10a9e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe9f00098a0 tx=0x7fe9f0009bb0 comp rx=0 tx=0).stop 2024-03-07T07:31:59.804 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.015+0000 7fea028d9700 1 -- 172.21.15.160:0/2157475093 shutdown_connections 2024-03-07T07:31:59.804 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.015+0000 7fea028d9700 1 --2- 172.21.15.160:0/2157475093 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9fc1085f0 0x7fe9fc10a9e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:59.804 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.015+0000 7fea028d9700 1 -- 172.21.15.160:0/2157475093 >> 172.21.15.160:0/2157475093 conn(0x7fe9fc101f40 msgr2=0x7fe9fc104380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:31:59.804 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.015+0000 7fea028d9700 1 -- 172.21.15.160:0/2157475093 shutdown_connections 2024-03-07T07:31:59.804 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.015+0000 7fea028d9700 1 -- 172.21.15.160:0/2157475093 wait complete. 2024-03-07T07:31:59.804 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.016+0000 7fea028d9700 1 Processor -- start 2024-03-07T07:31:59.805 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.016+0000 7fea028d9700 1 -- start start 2024-03-07T07:31:59.805 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.016+0000 7fea028d9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9fc1085f0 0x7fe9fc19b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:59.805 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.016+0000 7fea028d9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc10bec0 con 0x7fe9fc1085f0 2024-03-07T07:31:59.805 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.016+0000 7fe9fbfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9fc1085f0 0x7fe9fc19b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:59.805 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.017+0000 7fe9fbfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9fc1085f0 0x7fe9fc19b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44122/0 (socket says 172.21.15.160:44122) 2024-03-07T07:31:59.805 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.017+0000 7fe9fbfff700 1 -- 172.21.15.160:0/1041072210 learned_addr learned my addr 172.21.15.160:0/1041072210 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:31:59.805 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.017+0000 7fe9fbfff700 1 -- 172.21.15.160:0/1041072210 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9fc19bf20 con 0x7fe9fc1085f0 2024-03-07T07:31:59.805 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.017+0000 7fe9fbfff700 1 --2- 172.21.15.160:0/1041072210 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9fc1085f0 0x7fe9fc19b9e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe9f00117e0 tx=0x7fe9f0011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:59.805 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.017+0000 7fe9f97fa700 1 -- 172.21.15.160:0/1041072210 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe9f0003bf0 con 0x7fe9fc1085f0 2024-03-07T07:31:59.805 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.017+0000 7fe9f97fa700 1 -- 172.21.15.160:0/1041072210 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe9f001a910 con 0x7fe9fc1085f0 2024-03-07T07:31:59.805 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.017+0000 7fea028d9700 1 -- 172.21.15.160:0/1041072210 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9fc19c1b0 con 0x7fe9fc1085f0 2024-03-07T07:31:59.805 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.017+0000 7fe9f97fa700 1 -- 172.21.15.160:0/1041072210 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe9f0024970 con 0x7fe9fc1085f0 2024-03-07T07:31:59.805 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.017+0000 7fea028d9700 1 -- 172.21.15.160:0/1041072210 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9fc19c610 con 0x7fe9fc1085f0 2024-03-07T07:31:59.806 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.018+0000 7fe9f97fa700 1 -- 172.21.15.160:0/1041072210 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 8) v1 ==== 45927+0+0 (secure 0 0 0) 0x7fe9f0022020 con 0x7fe9fc1085f0 2024-03-07T07:31:59.806 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.018+0000 7fea028d9700 1 -- 172.21.15.160:0/1041072210 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9fc063780 con 0x7fe9fc1085f0 2024-03-07T07:31:59.806 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.018+0000 7fe9f97fa700 1 --2- 172.21.15.160:0/1041072210 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7fe9e4038ba0 0x7fe9e403b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:31:59.806 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.018+0000 7fe9f97fa700 1 -- 172.21.15.160:0/1041072210 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe9f004cd70 con 0x7fe9fc1085f0 2024-03-07T07:31:59.806 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.019+0000 7fe9fb7fe700 1 --2- 172.21.15.160:0/1041072210 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7fe9e4038ba0 0x7fe9e403b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:31:59.806 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.019+0000 7fe9fb7fe700 1 --2- 172.21.15.160:0/1041072210 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7fe9e4038ba0 0x7fe9e403b060 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec006fd0 tx=0x7fe9ec006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:31:59.806 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.028+0000 7fe9f97fa700 1 -- 172.21.15.160:0/1041072210 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fe9f0051070 con 0x7fe9fc1085f0 2024-03-07T07:31:59.806 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.227+0000 7fea028d9700 1 -- 172.21.15.160:0/1041072210 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7fe9fc19efe0 con 0x7fe9fc1085f0 2024-03-07T07:31:59.806 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.231+0000 7fe9f97fa700 1 -- 172.21.15.160:0/1041072210 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7fe9f0011e20 con 0x7fe9fc1085f0 2024-03-07T07:31:59.806 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.233+0000 7fea028d9700 1 -- 172.21.15.160:0/1041072210 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7fe9e4038ba0 msgr2=0x7fe9e403b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:59.806 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.233+0000 7fea028d9700 1 --2- 172.21.15.160:0/1041072210 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7fe9e4038ba0 0x7fe9e403b060 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec006fd0 tx=0x7fe9ec006e40 comp rx=0 tx=0).stop 2024-03-07T07:31:59.806 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.233+0000 7fea028d9700 1 -- 172.21.15.160:0/1041072210 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9fc1085f0 msgr2=0x7fe9fc19b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:31:59.806 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.233+0000 7fea028d9700 1 --2- 172.21.15.160:0/1041072210 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9fc1085f0 0x7fe9fc19b9e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe9f00117e0 tx=0x7fe9f0011c10 comp rx=0 tx=0).stop 2024-03-07T07:31:59.807 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.234+0000 7fea028d9700 1 -- 172.21.15.160:0/1041072210 shutdown_connections 2024-03-07T07:31:59.807 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.234+0000 7fea028d9700 1 --2- 172.21.15.160:0/1041072210 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7fe9e4038ba0 0x7fe9e403b060 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:59.807 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.234+0000 7fea028d9700 1 --2- 172.21.15.160:0/1041072210 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9fc1085f0 0x7fe9fc19b9e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:31:59.807 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.234+0000 7fea028d9700 1 -- 172.21.15.160:0/1041072210 >> 172.21.15.160:0/1041072210 conn(0x7fe9fc101f40 msgr2=0x7fe9fc1029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:31:59.807 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.234+0000 7fea028d9700 1 -- 172.21.15.160:0/1041072210 shutdown_connections 2024-03-07T07:31:59.807 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:31:59.234+0000 7fea028d9700 1 -- 172.21.15.160:0/1041072210 wait complete. 2024-03-07T07:31:59.807 INFO:teuthology.orchestra.run.smithi160.stdout:Enabling the dashboard module... 2024-03-07T07:32:00.374 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:00 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/1041072210' entity='client.admin' 2024-03-07T07:32:00.375 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:00 smithi160 ceph-mon[31963]: from='mgr.14120 172.21.15.160:0/1092429691' entity='mgr.x' 2024-03-07T07:32:01.522 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:01 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/3151440106' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-07T07:32:01.522 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:32:01 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: ignoring --setuser ceph since I am not root 2024-03-07T07:32:01.522 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:32:01 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: ignoring --setgroup ceph since I am not root 2024-03-07T07:32:02.017 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:00.206+0000 7f58e20c9700 1 Processor -- start 2024-03-07T07:32:02.017 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:00.206+0000 7f58e20c9700 1 -- start start 2024-03-07T07:32:02.017 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:00.206+0000 7f58e20c9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58dc106f10 0x7f58dc107330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:32:02.017 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:00.206+0000 7f58e20c9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc107900 con 0x7f58dc106f10 2024-03-07T07:32:02.017 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:00.206+0000 7f58db7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58dc106f10 0x7f58dc107330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:32:02.017 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:00.206+0000 7f58db7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58dc106f10 0x7f58dc107330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:56840/0 (socket says 172.21.15.160:56840) 2024-03-07T07:32:02.017 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:00.206+0000 7f58db7fe700 1 -- 172.21.15.160:0/893064145 learned_addr learned my addr 172.21.15.160:0/893064145 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:32:02.017 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:00.207+0000 7f58db7fe700 1 -- 172.21.15.160:0/893064145 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58dc107c30 con 0x7f58dc106f10 2024-03-07T07:32:02.017 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:00.207+0000 7f58db7fe700 1 --2- 172.21.15.160:0/893064145 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58dc106f10 0x7f58dc107330 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f58cc0098a0 tx=0x7f58cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b598bbf1a656f81 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:32:02.018 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:00.207+0000 7f58da7fc700 1 -- 172.21.15.160:0/893064145 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58cc004160 con 0x7f58dc106f10 2024-03-07T07:32:02.018 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:00.207+0000 7f58da7fc700 1 -- 172.21.15.160:0/893064145 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f58cc004300 con 0x7f58dc106f10 2024-03-07T07:32:02.018 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:00.207+0000 7f58da7fc700 1 -- 172.21.15.160:0/893064145 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58cc011b20 con 0x7f58dc106f10 2024-03-07T07:32:02.018 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:00.207+0000 7f58e20c9700 1 -- 172.21.15.160:0/893064145 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58dc106f10 msgr2=0x7f58dc107330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:02.018 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:00.207+0000 7f58e20c9700 1 --2- 172.21.15.160:0/893064145 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58dc106f10 0x7f58dc107330 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f58cc0098a0 tx=0x7f58cc009bb0 comp rx=0 tx=0).stop 2024-03-07T07:32:02.018 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:00.207+0000 7f58e20c9700 1 -- 172.21.15.160:0/893064145 shutdown_connections 2024-03-07T07:32:02.018 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:00.207+0000 7f58e20c9700 1 --2- 172.21.15.160:0/893064145 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58dc106f10 0x7f58dc107330 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:02.018 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:00.207+0000 7f58e20c9700 1 -- 172.21.15.160:0/893064145 >> 172.21.15.160:0/893064145 conn(0x7f58dc101790 msgr2=0x7f58dc103bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:32:02.018 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:00.207+0000 7f58e20c9700 1 -- 172.21.15.160:0/893064145 shutdown_connections 2024-03-07T07:32:02.018 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:00.207+0000 7f58e20c9700 1 -- 172.21.15.160:0/893064145 wait complete. 2024-03-07T07:32:02.018 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:00.208+0000 7f58e20c9700 1 Processor -- start 2024-03-07T07:32:02.019 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:00.208+0000 7f58e20c9700 1 -- start start 2024-03-07T07:32:02.019 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:00.208+0000 7f58e20c9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58dc106f10 0x7f58dc19a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:32:02.019 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:00.208+0000 7f58e20c9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc10afe0 con 0x7f58dc106f10 2024-03-07T07:32:02.019 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:00.208+0000 7f58db7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58dc106f10 0x7f58dc19a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:32:02.019 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:00.208+0000 7f58db7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58dc106f10 0x7f58dc19a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:56848/0 (socket says 172.21.15.160:56848) 2024-03-07T07:32:02.019 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:00.208+0000 7f58db7fe700 1 -- 172.21.15.160:0/3151440106 learned_addr learned my addr 172.21.15.160:0/3151440106 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:32:02.019 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:00.209+0000 7f58db7fe700 1 -- 172.21.15.160:0/3151440106 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58dc19aed0 con 0x7f58dc106f10 2024-03-07T07:32:02.019 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:00.209+0000 7f58db7fe700 1 --2- 172.21.15.160:0/3151440106 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58dc106f10 0x7f58dc19a990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f58cc011b60 tx=0x7f58cc011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:32:02.019 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:00.209+0000 7f58d8ff9700 1 -- 172.21.15.160:0/3151440106 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58cc003b60 con 0x7f58dc106f10 2024-03-07T07:32:02.019 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:00.209+0000 7f58d8ff9700 1 -- 172.21.15.160:0/3151440106 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f58cc01a910 con 0x7f58dc106f10 2024-03-07T07:32:02.019 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:00.209+0000 7f58d8ff9700 1 -- 172.21.15.160:0/3151440106 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58cc024930 con 0x7f58dc106f10 2024-03-07T07:32:02.019 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:00.209+0000 7f58e20c9700 1 -- 172.21.15.160:0/3151440106 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58dc19b160 con 0x7f58dc106f10 2024-03-07T07:32:02.020 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:00.209+0000 7f58e20c9700 1 -- 172.21.15.160:0/3151440106 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58dc19b5c0 con 0x7f58dc106f10 2024-03-07T07:32:02.020 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:00.210+0000 7f58d8ff9700 1 -- 172.21.15.160:0/3151440106 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 8) v1 ==== 45927+0+0 (secure 0 0 0) 0x7f58cc022020 con 0x7f58dc106f10 2024-03-07T07:32:02.020 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:00.210+0000 7f58e20c9700 1 -- 172.21.15.160:0/3151440106 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58dc063780 con 0x7f58dc106f10 2024-03-07T07:32:02.020 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:00.210+0000 7f58d8ff9700 1 --2- 172.21.15.160:0/3151440106 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7f58c4038bf0 0x7f58c403b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:32:02.020 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:00.210+0000 7f58d8ff9700 1 -- 172.21.15.160:0/3151440106 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f58cc04cc80 con 0x7f58dc106f10 2024-03-07T07:32:02.020 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:00.210+0000 7f58daffd700 1 --2- 172.21.15.160:0/3151440106 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7f58c4038bf0 0x7f58c403b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:32:02.020 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:00.210+0000 7f58daffd700 1 --2- 172.21.15.160:0/3151440106 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7f58c4038bf0 0x7f58c403b0b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f58d0006fd0 tx=0x7f58d0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:32:02.020 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:00.215+0000 7f58d8ff9700 1 -- 172.21.15.160:0/3151440106 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f58cc00bc70 con 0x7f58dc106f10 2024-03-07T07:32:02.020 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:00.481+0000 7f58e20c9700 1 -- 172.21.15.160:0/3151440106 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f58dc19df70 con 0x7f58dc106f10 2024-03-07T07:32:02.020 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:01.234+0000 7f58d8ff9700 1 -- 172.21.15.160:0/3151440106 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f58cc02d770 con 0x7f58dc106f10 2024-03-07T07:32:02.021 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:01.236+0000 7f58d8ff9700 1 -- 172.21.15.160:0/3151440106 <== mon.0 v2:172.21.15.160:3300/0 8 ==== mgrmap(e 9) v1 ==== 45940+0+0 (secure 0 0 0) 0x7f58cc01aac0 con 0x7f58dc106f10 2024-03-07T07:32:02.021 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:01.238+0000 7f58e20c9700 1 -- 172.21.15.160:0/3151440106 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7f58c4038bf0 msgr2=0x7f58c403b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:02.021 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:01.238+0000 7f58e20c9700 1 --2- 172.21.15.160:0/3151440106 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7f58c4038bf0 0x7f58c403b0b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f58d0006fd0 tx=0x7f58d0006e40 comp rx=0 tx=0).stop 2024-03-07T07:32:02.021 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:01.238+0000 7f58e20c9700 1 -- 172.21.15.160:0/3151440106 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58dc106f10 msgr2=0x7f58dc19a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:02.021 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:01.238+0000 7f58e20c9700 1 --2- 172.21.15.160:0/3151440106 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58dc106f10 0x7f58dc19a990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f58cc011b60 tx=0x7f58cc011c40 comp rx=0 tx=0).stop 2024-03-07T07:32:02.021 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:01.238+0000 7f58e20c9700 1 -- 172.21.15.160:0/3151440106 shutdown_connections 2024-03-07T07:32:02.021 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:01.238+0000 7f58e20c9700 1 --2- 172.21.15.160:0/3151440106 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7f58c4038bf0 0x7f58c403b0b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:02.021 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:01.238+0000 7f58e20c9700 1 --2- 172.21.15.160:0/3151440106 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58dc106f10 0x7f58dc19a990 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:02.021 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:01.238+0000 7f58e20c9700 1 -- 172.21.15.160:0/3151440106 >> 172.21.15.160:0/3151440106 conn(0x7f58dc101790 msgr2=0x7f58dc1022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:32:02.021 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:01.238+0000 7f58e20c9700 1 -- 172.21.15.160:0/3151440106 shutdown_connections 2024-03-07T07:32:02.021 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:01.238+0000 7f58e20c9700 1 -- 172.21.15.160:0/3151440106 wait complete. 2024-03-07T07:32:02.126 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:32:01 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:32:01.805+0000 7fc5a454b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-07T07:32:02.126 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:32:01 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:32:01.899+0000 7fc5a454b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-07T07:32:02.394 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:02 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/3151440106' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-07T07:32:02.394 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:02 smithi160 ceph-mon[31963]: mgrmap e9: x(active, since 20s) 2024-03-07T07:32:02.662 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:32:02 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:32:02.568+0000 7fc5a454b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-07T07:32:02.927 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:32:02 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:32:02.662+0000 7fc5a454b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-07T07:32:02.928 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:32:02 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:32:02.811+0000 7fc5a454b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-07T07:32:02.928 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:32:02 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:32:02.907+0000 7fc5a454b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-07T07:32:03.264 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:03 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/3022213236' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-07T07:32:03.264 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:32:02 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:32:02.994+0000 7fc5a454b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-07T07:32:03.272 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout { 2024-03-07T07:32:03.272 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-07T07:32:03.272 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-07T07:32:03.272 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-07T07:32:03.273 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-07T07:32:03.273 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout } 2024-03-07T07:32:03.273 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.428+0000 7f77d0bf9700 1 Processor -- start 2024-03-07T07:32:03.273 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.428+0000 7f77d0bf9700 1 -- start start 2024-03-07T07:32:03.273 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.429+0000 7f77d0bf9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f77cc1085e0 0x7f77cc10a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:32:03.273 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.429+0000 7f77d0bf9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77cc07be30 con 0x7f77cc1085e0 2024-03-07T07:32:03.273 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.429+0000 7f77ca59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f77cc1085e0 0x7f77cc10a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:32:03.273 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.429+0000 7f77ca59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f77cc1085e0 0x7f77cc10a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:56882/0 (socket says 172.21.15.160:56882) 2024-03-07T07:32:03.273 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.429+0000 7f77ca59c700 1 -- 172.21.15.160:0/2967198467 learned_addr learned my addr 172.21.15.160:0/2967198467 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:32:03.273 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.429+0000 7f77ca59c700 1 -- 172.21.15.160:0/2967198467 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77cc10af10 con 0x7f77cc1085e0 2024-03-07T07:32:03.273 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.429+0000 7f77ca59c700 1 --2- 172.21.15.160:0/2967198467 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f77cc1085e0 0x7f77cc10a9d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f77c00098a0 tx=0x7f77c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4edee8bcdd895522 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:32:03.273 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.430+0000 7f77c959a700 1 -- 172.21.15.160:0/2967198467 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77c0004160 con 0x7f77cc1085e0 2024-03-07T07:32:03.273 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.430+0000 7f77c959a700 1 -- 172.21.15.160:0/2967198467 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77c0004300 con 0x7f77cc1085e0 2024-03-07T07:32:03.274 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.430+0000 7f77c959a700 1 -- 172.21.15.160:0/2967198467 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77c0011b20 con 0x7f77cc1085e0 2024-03-07T07:32:03.274 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.430+0000 7f77d0bf9700 1 -- 172.21.15.160:0/2967198467 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f77cc1085e0 msgr2=0x7f77cc10a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:03.274 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.430+0000 7f77d0bf9700 1 --2- 172.21.15.160:0/2967198467 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f77cc1085e0 0x7f77cc10a9d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f77c00098a0 tx=0x7f77c0009bb0 comp rx=0 tx=0).stop 2024-03-07T07:32:03.274 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.430+0000 7f77d0bf9700 1 -- 172.21.15.160:0/2967198467 shutdown_connections 2024-03-07T07:32:03.274 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.430+0000 7f77d0bf9700 1 --2- 172.21.15.160:0/2967198467 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f77cc1085e0 0x7f77cc10a9d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:03.274 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.430+0000 7f77d0bf9700 1 -- 172.21.15.160:0/2967198467 >> 172.21.15.160:0/2967198467 conn(0x7f77cc101ff0 msgr2=0x7f77cc104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:32:03.274 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.430+0000 7f77d0bf9700 1 -- 172.21.15.160:0/2967198467 shutdown_connections 2024-03-07T07:32:03.274 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.430+0000 7f77d0bf9700 1 -- 172.21.15.160:0/2967198467 wait complete. 2024-03-07T07:32:03.274 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.431+0000 7f77d0bf9700 1 Processor -- start 2024-03-07T07:32:03.274 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.431+0000 7f77d0bf9700 1 -- start start 2024-03-07T07:32:03.274 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.431+0000 7f77d0bf9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f77cc1085e0 0x7f77cc19baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:32:03.274 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.431+0000 7f77d0bf9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77cc10bf20 con 0x7f77cc1085e0 2024-03-07T07:32:03.274 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.432+0000 7f77ca59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f77cc1085e0 0x7f77cc19baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:32:03.275 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.432+0000 7f77ca59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f77cc1085e0 0x7f77cc19baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:56884/0 (socket says 172.21.15.160:56884) 2024-03-07T07:32:03.275 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.432+0000 7f77ca59c700 1 -- 172.21.15.160:0/3022213236 learned_addr learned my addr 172.21.15.160:0/3022213236 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:32:03.275 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.432+0000 7f77ca59c700 1 -- 172.21.15.160:0/3022213236 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77cc19bfe0 con 0x7f77cc1085e0 2024-03-07T07:32:03.275 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.432+0000 7f77ca59c700 1 --2- 172.21.15.160:0/3022213236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f77cc1085e0 0x7f77cc19baa0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f77c0011b60 tx=0x7f77c0011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:32:03.275 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.432+0000 7f77bb7fe700 1 -- 172.21.15.160:0/3022213236 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77c0003b60 con 0x7f77cc1085e0 2024-03-07T07:32:03.275 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.432+0000 7f77bb7fe700 1 -- 172.21.15.160:0/3022213236 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77c001a910 con 0x7f77cc1085e0 2024-03-07T07:32:03.275 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.432+0000 7f77bb7fe700 1 -- 172.21.15.160:0/3022213236 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77c0024930 con 0x7f77cc1085e0 2024-03-07T07:32:03.275 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.432+0000 7f77d0bf9700 1 -- 172.21.15.160:0/3022213236 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77cc19c270 con 0x7f77cc1085e0 2024-03-07T07:32:03.275 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.432+0000 7f77d0bf9700 1 -- 172.21.15.160:0/3022213236 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77cc19c6d0 con 0x7f77cc1085e0 2024-03-07T07:32:03.275 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.433+0000 7f77bb7fe700 1 -- 172.21.15.160:0/3022213236 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 9) v1 ==== 45940+0+0 (secure 0 0 0) 0x7f77c0022020 con 0x7f77cc1085e0 2024-03-07T07:32:03.275 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.433+0000 7f77d0bf9700 1 -- 172.21.15.160:0/3022213236 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77cc063780 con 0x7f77cc1085e0 2024-03-07T07:32:03.275 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.433+0000 7f77bb7fe700 1 --2- 172.21.15.160:0/3022213236 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7f77b4038bf0 0x7f77b403b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:32:03.275 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.433+0000 7f77bb7fe700 1 -- 172.21.15.160:0/3022213236 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f77c004ccc0 con 0x7f77cc1085e0 2024-03-07T07:32:03.275 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.433+0000 7f77c9d9b700 1 -- 172.21.15.160:0/3022213236 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7f77b4038bf0 msgr2=0x7f77b403b0b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6800/3859174315 2024-03-07T07:32:03.276 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.433+0000 7f77c9d9b700 1 --2- 172.21.15.160:0/3022213236 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7f77b4038bf0 0x7f77b403b0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-07T07:32:03.276 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.442+0000 7f77bb7fe700 1 -- 172.21.15.160:0/3022213236 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f77c001abe0 con 0x7f77cc1085e0 2024-03-07T07:32:03.276 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.634+0000 7f77c9d9b700 1 -- 172.21.15.160:0/3022213236 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7f77b4038bf0 msgr2=0x7f77b403b0b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6800/3859174315 2024-03-07T07:32:03.276 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.634+0000 7f77c9d9b700 1 --2- 172.21.15.160:0/3022213236 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7f77b4038bf0 0x7f77b403b0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-07T07:32:03.276 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.695+0000 7f77d0bf9700 1 -- 172.21.15.160:0/3022213236 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f77cc19f080 con 0x7f77cc1085e0 2024-03-07T07:32:03.276 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.696+0000 7f77bb7fe700 1 -- 172.21.15.160:0/3022213236 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f77c002d910 con 0x7f77cc1085e0 2024-03-07T07:32:03.276 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.698+0000 7f77d0bf9700 1 -- 172.21.15.160:0/3022213236 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7f77b4038bf0 msgr2=0x7f77b403b0b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-07T07:32:03.276 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.698+0000 7f77d0bf9700 1 --2- 172.21.15.160:0/3022213236 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7f77b4038bf0 0x7f77b403b0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:03.276 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.698+0000 7f77d0bf9700 1 -- 172.21.15.160:0/3022213236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f77cc1085e0 msgr2=0x7f77cc19baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:03.276 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.698+0000 7f77d0bf9700 1 --2- 172.21.15.160:0/3022213236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f77cc1085e0 0x7f77cc19baa0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f77c0011b60 tx=0x7f77c0011c40 comp rx=0 tx=0).stop 2024-03-07T07:32:03.276 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.698+0000 7f77d0bf9700 1 -- 172.21.15.160:0/3022213236 shutdown_connections 2024-03-07T07:32:03.276 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.698+0000 7f77d0bf9700 1 --2- 172.21.15.160:0/3022213236 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7f77b4038bf0 0x7f77b403b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:03.276 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.698+0000 7f77d0bf9700 1 --2- 172.21.15.160:0/3022213236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f77cc1085e0 0x7f77cc19baa0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:03.277 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.698+0000 7f77d0bf9700 1 -- 172.21.15.160:0/3022213236 >> 172.21.15.160:0/3022213236 conn(0x7f77cc101ff0 msgr2=0x7f77cc102b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:32:03.277 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.698+0000 7f77d0bf9700 1 -- 172.21.15.160:0/3022213236 shutdown_connections 2024-03-07T07:32:03.277 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:02.698+0000 7f77d0bf9700 1 -- 172.21.15.160:0/3022213236 wait complete. 2024-03-07T07:32:03.277 INFO:teuthology.orchestra.run.smithi160.stdout:Waiting for the mgr to restart... 2024-03-07T07:32:03.277 INFO:teuthology.orchestra.run.smithi160.stdout:Waiting for mgr epoch 9... 2024-03-07T07:32:03.515 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:32:03 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:32:03.265+0000 7fc5a454b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-07T07:32:03.515 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:32:03 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:32:03.380+0000 7fc5a454b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-07T07:32:03.515 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:32:03 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:32:03.468+0000 7fc5a454b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-07T07:32:03.910 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:32:03 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:32:03.568+0000 7fc5a454b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-07T07:32:03.911 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:32:03 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:32:03.753+0000 7fc5a454b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-07T07:32:05.112 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:32:04 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:32:04.719+0000 7fc5a454b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-07T07:32:05.112 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:32:04 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:32:04.849+0000 7fc5a454b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-07T07:32:05.427 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:32:05 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:32:05.188+0000 7fc5a454b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-07T07:32:05.428 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:32:05 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:32:05.279+0000 7fc5a454b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-07T07:32:05.928 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:32:05 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:32:05.477+0000 7fc5a454b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-07T07:32:05.928 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:32:05 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:32:05.669+0000 7fc5a454b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-07T07:32:06.225 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:32:05 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:32:05.958+0000 7fc5a454b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-07T07:32:06.226 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:32:06 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:32:06.225+0000 7fc5a454b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-07T07:32:06.677 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:32:06 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:32:06.464+0000 7fc5a454b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-07T07:32:07.677 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:32:07 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:32:07.279+0000 7fc5a454b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-07T07:32:08.177 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:32:07 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:32:07.853+0000 7fc5a454b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-07T07:32:08.178 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:32:07 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:32:07.954+0000 7fc5a454b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-07T07:32:08.178 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:32:08 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:32:08.177+0000 7fc5a454b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-07T07:32:08.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:08 smithi160 ceph-mon[31963]: Active manager daemon x restarted 2024-03-07T07:32:08.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:08 smithi160 ceph-mon[31963]: Activating manager daemon x 2024-03-07T07:32:08.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:08 smithi160 ceph-mon[31963]: osdmap e3: 0 total, 0 up, 0 in 2024-03-07T07:32:08.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:08 smithi160 ceph-mon[31963]: mgrmap e10: x(active, starting, since 0.00354213s) 2024-03-07T07:32:08.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:08 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-07T07:32:08.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:08 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-07T07:32:08.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:08 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-07T07:32:08.679 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:08 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-07T07:32:08.679 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:08 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-07T07:32:08.679 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:08 smithi160 ceph-mon[31963]: Manager daemon x is now available 2024-03-07T07:32:08.679 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:08 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:32:08.679 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:08 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:32:09.407 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:09 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-07T07:32:09.407 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:09 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-07T07:32:09.408 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:09 smithi160 ceph-mon[31963]: mgrmap e11: x(active, since 1.00584s) 2024-03-07T07:32:10.133 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout { 2024-03-07T07:32:10.133 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-07T07:32:10.133 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-07T07:32:10.134 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout } 2024-03-07T07:32:10.134 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:03.704+0000 7f8f4e7d1700 1 Processor -- start 2024-03-07T07:32:10.134 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:03.704+0000 7f8f4e7d1700 1 -- start start 2024-03-07T07:32:10.134 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:03.704+0000 7f8f4e7d1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f48108590 0x7f8f4810a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:32:10.134 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:03.704+0000 7f8f4e7d1700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4807be50 con 0x7f8f48108590 2024-03-07T07:32:10.134 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:03.705+0000 7f8f47fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f48108590 0x7f8f4810a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:32:10.134 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:03.705+0000 7f8f47fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f48108590 0x7f8f4810a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:56898/0 (socket says 172.21.15.160:56898) 2024-03-07T07:32:10.134 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:03.705+0000 7f8f47fff700 1 -- 172.21.15.160:0/1428421396 learned_addr learned my addr 172.21.15.160:0/1428421396 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:32:10.134 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:03.705+0000 7f8f47fff700 1 -- 172.21.15.160:0/1428421396 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f4810aec0 con 0x7f8f48108590 2024-03-07T07:32:10.134 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:03.705+0000 7f8f47fff700 1 --2- 172.21.15.160:0/1428421396 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f48108590 0x7f8f4810a980 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c0098a0 tx=0x7f8f3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c17dc0994820a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:32:10.134 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:03.705+0000 7f8f46ffd700 1 -- 172.21.15.160:0/1428421396 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f3c004160 con 0x7f8f48108590 2024-03-07T07:32:10.134 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:03.705+0000 7f8f46ffd700 1 -- 172.21.15.160:0/1428421396 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8f3c004300 con 0x7f8f48108590 2024-03-07T07:32:10.135 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:03.705+0000 7f8f46ffd700 1 -- 172.21.15.160:0/1428421396 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f3c011b20 con 0x7f8f48108590 2024-03-07T07:32:10.135 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:03.706+0000 7f8f4e7d1700 1 -- 172.21.15.160:0/1428421396 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f48108590 msgr2=0x7f8f4810a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:10.135 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:03.706+0000 7f8f4e7d1700 1 --2- 172.21.15.160:0/1428421396 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f48108590 0x7f8f4810a980 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c0098a0 tx=0x7f8f3c009bb0 comp rx=0 tx=0).stop 2024-03-07T07:32:10.135 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:03.706+0000 7f8f4e7d1700 1 -- 172.21.15.160:0/1428421396 shutdown_connections 2024-03-07T07:32:10.135 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:03.706+0000 7f8f4e7d1700 1 --2- 172.21.15.160:0/1428421396 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f48108590 0x7f8f4810a980 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:10.135 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:03.706+0000 7f8f4e7d1700 1 -- 172.21.15.160:0/1428421396 >> 172.21.15.160:0/1428421396 conn(0x7f8f48101f60 msgr2=0x7f8f481043c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:32:10.135 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:03.706+0000 7f8f4e7d1700 1 -- 172.21.15.160:0/1428421396 shutdown_connections 2024-03-07T07:32:10.135 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:03.706+0000 7f8f4e7d1700 1 -- 172.21.15.160:0/1428421396 wait complete. 2024-03-07T07:32:10.135 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:03.706+0000 7f8f4e7d1700 1 Processor -- start 2024-03-07T07:32:10.135 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:03.706+0000 7f8f4e7d1700 1 -- start start 2024-03-07T07:32:10.135 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:03.707+0000 7f8f4e7d1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f48108590 0x7f8f4819ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:32:10.135 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:03.707+0000 7f8f4e7d1700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4810bf20 con 0x7f8f48108590 2024-03-07T07:32:10.135 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:03.707+0000 7f8f47fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f48108590 0x7f8f4819ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:32:10.135 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:03.707+0000 7f8f47fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f48108590 0x7f8f4819ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:56908/0 (socket says 172.21.15.160:56908) 2024-03-07T07:32:10.136 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:03.707+0000 7f8f47fff700 1 -- 172.21.15.160:0/1762610156 learned_addr learned my addr 172.21.15.160:0/1762610156 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:32:10.136 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:03.707+0000 7f8f47fff700 1 -- 172.21.15.160:0/1762610156 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f4819bf50 con 0x7f8f48108590 2024-03-07T07:32:10.136 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:03.707+0000 7f8f47fff700 1 --2- 172.21.15.160:0/1762610156 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f48108590 0x7f8f4819ba10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c0117e0 tx=0x7f8f3c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:32:10.136 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:03.707+0000 7f8f457fa700 1 -- 172.21.15.160:0/1762610156 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f3c003ba0 con 0x7f8f48108590 2024-03-07T07:32:10.136 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:03.707+0000 7f8f457fa700 1 -- 172.21.15.160:0/1762610156 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8f3c01a910 con 0x7f8f48108590 2024-03-07T07:32:10.136 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:03.708+0000 7f8f4e7d1700 1 -- 172.21.15.160:0/1762610156 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f4819c1e0 con 0x7f8f48108590 2024-03-07T07:32:10.136 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:03.708+0000 7f8f457fa700 1 -- 172.21.15.160:0/1762610156 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f3c0248f0 con 0x7f8f48108590 2024-03-07T07:32:10.136 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:03.708+0000 7f8f4e7d1700 1 -- 172.21.15.160:0/1762610156 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f4819c640 con 0x7f8f48108590 2024-03-07T07:32:10.136 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:03.709+0000 7f8f457fa700 1 -- 172.21.15.160:0/1762610156 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 9) v1 ==== 45940+0+0 (secure 0 0 0) 0x7f8f3c022020 con 0x7f8f48108590 2024-03-07T07:32:10.136 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:03.709+0000 7f8f457fa700 1 --2- 172.21.15.160:0/1762610156 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7f8f30038bf0 0x7f8f3003b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:32:10.136 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:03.709+0000 7f8f457fa700 1 -- 172.21.15.160:0/1762610156 --> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8f3003b7c0 con 0x7f8f30038bf0 2024-03-07T07:32:10.136 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:03.709+0000 7f8f457fa700 1 -- 172.21.15.160:0/1762610156 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f8f3c04d2d0 con 0x7f8f48108590 2024-03-07T07:32:10.136 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:03.709+0000 7f8f477fe700 1 -- 172.21.15.160:0/1762610156 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7f8f30038bf0 msgr2=0x7f8f3003b0b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6800/3859174315 2024-03-07T07:32:10.136 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:03.709+0000 7f8f477fe700 1 --2- 172.21.15.160:0/1762610156 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7f8f30038bf0 0x7f8f3003b0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-07T07:32:10.137 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:03.910+0000 7f8f477fe700 1 -- 172.21.15.160:0/1762610156 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7f8f30038bf0 msgr2=0x7f8f3003b0b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6800/3859174315 2024-03-07T07:32:10.137 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:03.910+0000 7f8f477fe700 1 --2- 172.21.15.160:0/1762610156 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7f8f30038bf0 0x7f8f3003b0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-07T07:32:10.137 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:04.311+0000 7f8f477fe700 1 -- 172.21.15.160:0/1762610156 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7f8f30038bf0 msgr2=0x7f8f3003b0b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6800/3859174315 2024-03-07T07:32:10.137 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:04.311+0000 7f8f477fe700 1 --2- 172.21.15.160:0/1762610156 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7f8f30038bf0 0x7f8f3003b0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-07T07:32:10.137 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:05.112+0000 7f8f477fe700 1 -- 172.21.15.160:0/1762610156 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7f8f30038bf0 msgr2=0x7f8f3003b0b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6800/3859174315 2024-03-07T07:32:10.137 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:05.112+0000 7f8f477fe700 1 --2- 172.21.15.160:0/1762610156 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7f8f30038bf0 0x7f8f3003b0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-07T07:32:10.137 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:06.713+0000 7f8f477fe700 1 -- 172.21.15.160:0/1762610156 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7f8f30038bf0 msgr2=0x7f8f3003b0b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6800/3859174315 2024-03-07T07:32:10.137 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:06.713+0000 7f8f477fe700 1 --2- 172.21.15.160:0/1762610156 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7f8f30038bf0 0x7f8f3003b0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-07T07:32:10.137 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:08.192+0000 7f8f457fa700 1 -- 172.21.15.160:0/1762610156 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mgrmap(e 10) v1 ==== 45707+0+0 (secure 0 0 0) 0x7f8f3c02d3f0 con 0x7f8f48108590 2024-03-07T07:32:10.137 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:08.192+0000 7f8f457fa700 1 -- 172.21.15.160:0/1762610156 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7f8f30038bf0 msgr2=0x7f8f3003b0b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-07T07:32:10.137 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:08.192+0000 7f8f457fa700 1 --2- 172.21.15.160:0/1762610156 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7f8f30038bf0 0x7f8f3003b0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:10.137 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:09.193+0000 7f8f457fa700 1 -- 172.21.15.160:0/1762610156 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mgrmap(e 11) v1 ==== 45834+0+0 (secure 0 0 0) 0x7f8f3c019d00 con 0x7f8f48108590 2024-03-07T07:32:10.137 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:09.193+0000 7f8f457fa700 1 --2- 172.21.15.160:0/1762610156 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f8f3003c6b0 0x7f8f3003eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:32:10.137 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:09.193+0000 7f8f457fa700 1 -- 172.21.15.160:0/1762610156 --> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8f3003b7c0 con 0x7f8f3003c6b0 2024-03-07T07:32:10.138 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:09.194+0000 7f8f477fe700 1 --2- 172.21.15.160:0/1762610156 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f8f3003c6b0 0x7f8f3003eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:32:10.138 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:09.194+0000 7f8f477fe700 1 --2- 172.21.15.160:0/1762610156 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f8f3003c6b0 0x7f8f3003eaa0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8f38003a10 tx=0x7f8f380092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:32:10.138 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:09.195+0000 7f8f457fa700 1 -- 172.21.15.160:0/1762610156 <== mgr.14152 v2:172.21.15.160:6800/419630619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f8f3003b7c0 con 0x7f8f3003c6b0 2024-03-07T07:32:10.138 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:09.201+0000 7f8f4e7d1700 1 -- 172.21.15.160:0/1762610156 --> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f8f48063780 con 0x7f8f3003c6b0 2024-03-07T07:32:10.138 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:09.202+0000 7f8f457fa700 1 -- 172.21.15.160:0/1762610156 <== mgr.14152 v2:172.21.15.160:6800/419630619 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f8f48063780 con 0x7f8f3003c6b0 2024-03-07T07:32:10.138 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:09.202+0000 7f8f4e7d1700 1 -- 172.21.15.160:0/1762610156 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f8f3003c6b0 msgr2=0x7f8f3003eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:10.138 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:09.202+0000 7f8f4e7d1700 1 --2- 172.21.15.160:0/1762610156 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f8f3003c6b0 0x7f8f3003eaa0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8f38003a10 tx=0x7f8f380092b0 comp rx=0 tx=0).stop 2024-03-07T07:32:10.138 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:09.202+0000 7f8f4e7d1700 1 -- 172.21.15.160:0/1762610156 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f48108590 msgr2=0x7f8f4819ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:10.138 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:09.202+0000 7f8f4e7d1700 1 --2- 172.21.15.160:0/1762610156 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f48108590 0x7f8f4819ba10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c0117e0 tx=0x7f8f3c011ba0 comp rx=0 tx=0).stop 2024-03-07T07:32:10.138 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:09.202+0000 7f8f4e7d1700 1 -- 172.21.15.160:0/1762610156 shutdown_connections 2024-03-07T07:32:10.138 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:09.202+0000 7f8f4e7d1700 1 --2- 172.21.15.160:0/1762610156 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f8f3003c6b0 0x7f8f3003eaa0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:10.138 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:09.202+0000 7f8f4e7d1700 1 --2- 172.21.15.160:0/1762610156 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f48108590 0x7f8f4819ba10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:10.138 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:09.202+0000 7f8f4e7d1700 1 --2- 172.21.15.160:0/1762610156 >> [v2:172.21.15.160:6800/3859174315,v1:172.21.15.160:6801/3859174315] conn(0x7f8f30038bf0 0x7f8f3003b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:10.139 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:09.202+0000 7f8f4e7d1700 1 -- 172.21.15.160:0/1762610156 >> 172.21.15.160:0/1762610156 conn(0x7f8f48101f60 msgr2=0x7f8f48102a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:32:10.139 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:09.202+0000 7f8f4e7d1700 1 -- 172.21.15.160:0/1762610156 shutdown_connections 2024-03-07T07:32:10.139 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:09.202+0000 7f8f4e7d1700 1 -- 172.21.15.160:0/1762610156 wait complete. 2024-03-07T07:32:10.139 INFO:teuthology.orchestra.run.smithi160.stdout:mgr epoch 9 is available 2024-03-07T07:32:10.139 INFO:teuthology.orchestra.run.smithi160.stdout:Generating a dashboard self-signed certificate... 2024-03-07T07:32:10.387 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:10 smithi160 ceph-mon[31963]: [07/Mar/2024:07:32:08] ENGINE Bus STARTING 2024-03-07T07:32:10.387 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:10 smithi160 ceph-mon[31963]: [07/Mar/2024:07:32:08] ENGINE Serving on https://172.21.15.160:7150 2024-03-07T07:32:10.387 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:10 smithi160 ceph-mon[31963]: [07/Mar/2024:07:32:08] ENGINE Serving on http://172.21.15.160:8765 2024-03-07T07:32:10.387 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:10 smithi160 ceph-mon[31963]: [07/Mar/2024:07:32:08] ENGINE Bus STARTED 2024-03-07T07:32:10.387 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:10 smithi160 ceph-mon[31963]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-07T07:32:10.387 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:10 smithi160 ceph-mon[31963]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-07T07:32:10.387 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:10 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:11.404 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:11 smithi160 ceph-mon[31963]: mgrmap e12: x(active, since 2s) 2024-03-07T07:32:11.404 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:11 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:11.404 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:11 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:11.680 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-07T07:32:11.680 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:10.593+0000 7f9a9edcf700 1 Processor -- start 2024-03-07T07:32:11.680 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:10.593+0000 7f9a9edcf700 1 -- start start 2024-03-07T07:32:11.680 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:10.593+0000 7f9a9edcf700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a98108560 0x7f9a9810a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:32:11.680 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:10.593+0000 7f9a9edcf700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a98075880 con 0x7f9a98108560 2024-03-07T07:32:11.680 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:10.593+0000 7f9a9cb6b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a98108560 0x7f9a9810a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:32:11.681 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:10.593+0000 7f9a9cb6b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a98108560 0x7f9a9810a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:36052/0 (socket says 172.21.15.160:36052) 2024-03-07T07:32:11.681 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:10.593+0000 7f9a9cb6b700 1 -- 172.21.15.160:0/481064254 learned_addr learned my addr 172.21.15.160:0/481064254 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:32:11.681 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:10.593+0000 7f9a9cb6b700 1 -- 172.21.15.160:0/481064254 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a9810ae90 con 0x7f9a98108560 2024-03-07T07:32:11.681 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:10.594+0000 7f9a9cb6b700 1 --2- 172.21.15.160:0/481064254 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a98108560 0x7f9a9810a950 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c0098a0 tx=0x7f9a8c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ee036efa8ee194b server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:32:11.681 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:10.594+0000 7f9a977fe700 1 -- 172.21.15.160:0/481064254 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a8c004160 con 0x7f9a98108560 2024-03-07T07:32:11.681 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:10.594+0000 7f9a977fe700 1 -- 172.21.15.160:0/481064254 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a8c004300 con 0x7f9a98108560 2024-03-07T07:32:11.681 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:10.594+0000 7f9a977fe700 1 -- 172.21.15.160:0/481064254 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a8c011b20 con 0x7f9a98108560 2024-03-07T07:32:11.681 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:10.594+0000 7f9a9edcf700 1 -- 172.21.15.160:0/481064254 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a98108560 msgr2=0x7f9a9810a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:11.681 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:10.594+0000 7f9a9edcf700 1 --2- 172.21.15.160:0/481064254 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a98108560 0x7f9a9810a950 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c0098a0 tx=0x7f9a8c009bb0 comp rx=0 tx=0).stop 2024-03-07T07:32:11.681 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:10.594+0000 7f9a9edcf700 1 -- 172.21.15.160:0/481064254 shutdown_connections 2024-03-07T07:32:11.681 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:10.594+0000 7f9a9edcf700 1 --2- 172.21.15.160:0/481064254 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a98108560 0x7f9a9810a950 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:11.681 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:10.594+0000 7f9a9edcf700 1 -- 172.21.15.160:0/481064254 >> 172.21.15.160:0/481064254 conn(0x7f9a98101f70 msgr2=0x7f9a98104390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:32:11.681 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:10.595+0000 7f9a9edcf700 1 -- 172.21.15.160:0/481064254 shutdown_connections 2024-03-07T07:32:11.682 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:10.595+0000 7f9a9edcf700 1 -- 172.21.15.160:0/481064254 wait complete. 2024-03-07T07:32:11.682 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:10.595+0000 7f9a9edcf700 1 Processor -- start 2024-03-07T07:32:11.682 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:10.595+0000 7f9a9edcf700 1 -- start start 2024-03-07T07:32:11.682 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:10.595+0000 7f9a9edcf700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a98108560 0x7f9a9819ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:32:11.682 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:10.595+0000 7f9a9edcf700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9810bef0 con 0x7f9a98108560 2024-03-07T07:32:11.682 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:10.595+0000 7f9a9cb6b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a98108560 0x7f9a9819ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:32:11.682 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:10.595+0000 7f9a9cb6b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a98108560 0x7f9a9819ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:36054/0 (socket says 172.21.15.160:36054) 2024-03-07T07:32:11.682 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:10.595+0000 7f9a9cb6b700 1 -- 172.21.15.160:0/2424450326 learned_addr learned my addr 172.21.15.160:0/2424450326 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:32:11.682 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:10.596+0000 7f9a9cb6b700 1 -- 172.21.15.160:0/2424450326 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a9819bfa0 con 0x7f9a98108560 2024-03-07T07:32:11.682 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:10.596+0000 7f9a9cb6b700 1 --2- 172.21.15.160:0/2424450326 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a98108560 0x7f9a9819ba60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c0116a0 tx=0x7f9a8c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:32:11.682 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:10.596+0000 7f9a95ffb700 1 -- 172.21.15.160:0/2424450326 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a8c003a40 con 0x7f9a98108560 2024-03-07T07:32:11.682 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:10.596+0000 7f9a9edcf700 1 -- 172.21.15.160:0/2424450326 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a9819c230 con 0x7f9a98108560 2024-03-07T07:32:11.682 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:10.596+0000 7f9a95ffb700 1 -- 172.21.15.160:0/2424450326 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a8c003be0 con 0x7f9a98108560 2024-03-07T07:32:11.682 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:10.596+0000 7f9a95ffb700 1 -- 172.21.15.160:0/2424450326 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a8c01a750 con 0x7f9a98108560 2024-03-07T07:32:11.683 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:10.596+0000 7f9a9edcf700 1 -- 172.21.15.160:0/2424450326 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a9819c690 con 0x7f9a98108560 2024-03-07T07:32:11.683 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:10.597+0000 7f9a95ffb700 1 -- 172.21.15.160:0/2424450326 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 12) v1 ==== 45940+0+0 (secure 0 0 0) 0x7f9a8c024aa0 con 0x7f9a98108560 2024-03-07T07:32:11.683 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:10.597+0000 7f9a9edcf700 1 -- 172.21.15.160:0/2424450326 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a98063780 con 0x7f9a98108560 2024-03-07T07:32:11.683 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:10.598+0000 7f9a95ffb700 1 --2- 172.21.15.160:0/2424450326 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f9a88038ba0 0x7f9a8803b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:32:11.683 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:10.598+0000 7f9a95ffb700 1 -- 172.21.15.160:0/2424450326 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9a8c04d210 con 0x7f9a98108560 2024-03-07T07:32:11.683 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:10.598+0000 7f9a97fff700 1 --2- 172.21.15.160:0/2424450326 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f9a88038ba0 0x7f9a8803b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:32:11.683 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:10.598+0000 7f9a97fff700 1 --2- 172.21.15.160:0/2424450326 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f9a88038ba0 0x7f9a8803b060 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9a84006fd0 tx=0x7f9a84006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:32:11.683 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:10.603+0000 7f9a95ffb700 1 -- 172.21.15.160:0/2424450326 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9a8c011e20 con 0x7f9a98108560 2024-03-07T07:32:11.683 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:10.811+0000 7f9a9edcf700 1 -- 172.21.15.160:0/2424450326 --> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f9a98103120 con 0x7f9a88038ba0 2024-03-07T07:32:11.683 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:11.064+0000 7f9a95ffb700 1 -- 172.21.15.160:0/2424450326 <== mgr.14152 v2:172.21.15.160:6800/419630619 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f9a98103120 con 0x7f9a88038ba0 2024-03-07T07:32:11.683 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:11.066+0000 7f9a9edcf700 1 -- 172.21.15.160:0/2424450326 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f9a88038ba0 msgr2=0x7f9a8803b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:11.683 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:11.066+0000 7f9a9edcf700 1 --2- 172.21.15.160:0/2424450326 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f9a88038ba0 0x7f9a8803b060 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9a84006fd0 tx=0x7f9a84006e40 comp rx=0 tx=0).stop 2024-03-07T07:32:11.683 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:11.066+0000 7f9a9edcf700 1 -- 172.21.15.160:0/2424450326 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a98108560 msgr2=0x7f9a9819ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:11.683 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:11.066+0000 7f9a9edcf700 1 --2- 172.21.15.160:0/2424450326 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a98108560 0x7f9a9819ba60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c0116a0 tx=0x7f9a8c011ba0 comp rx=0 tx=0).stop 2024-03-07T07:32:11.684 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:11.066+0000 7f9a9edcf700 1 -- 172.21.15.160:0/2424450326 shutdown_connections 2024-03-07T07:32:11.684 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:11.066+0000 7f9a9edcf700 1 --2- 172.21.15.160:0/2424450326 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f9a88038ba0 0x7f9a8803b060 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:11.684 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:11.066+0000 7f9a9edcf700 1 --2- 172.21.15.160:0/2424450326 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a98108560 0x7f9a9819ba60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:11.684 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:11.066+0000 7f9a9edcf700 1 -- 172.21.15.160:0/2424450326 >> 172.21.15.160:0/2424450326 conn(0x7f9a98101f70 msgr2=0x7f9a98102a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:32:11.684 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:11.066+0000 7f9a9edcf700 1 -- 172.21.15.160:0/2424450326 shutdown_connections 2024-03-07T07:32:11.684 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:11.066+0000 7f9a9edcf700 1 -- 172.21.15.160:0/2424450326 wait complete. 2024-03-07T07:32:11.684 INFO:teuthology.orchestra.run.smithi160.stdout:Creating initial admin user... 2024-03-07T07:32:12.270 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:12 smithi160 ceph-mon[31963]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:32:12.270 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:12 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:13.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:13 smithi160 ceph-mon[31963]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:32:13.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:13 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:13.862 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$Sr2ZSJW5EbgoasAo5.M3M.ggzBaodZssqGVpkRMjDetQ.CdJimA6W", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1709796732, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-07T07:32:13.862 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.117+0000 7fd7c1957700 1 Processor -- start 2024-03-07T07:32:13.862 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.117+0000 7fd7c1957700 1 -- start start 2024-03-07T07:32:13.863 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.117+0000 7fd7c1957700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7bc107ae0 0x7fd7bc107f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:32:13.863 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.117+0000 7fd7c1957700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc1084d0 con 0x7fd7bc107ae0 2024-03-07T07:32:13.863 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.117+0000 7fd7baffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7bc107ae0 0x7fd7bc107f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:32:13.863 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.117+0000 7fd7baffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7bc107ae0 0x7fd7bc107f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:36060/0 (socket says 172.21.15.160:36060) 2024-03-07T07:32:13.863 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.117+0000 7fd7baffd700 1 -- 172.21.15.160:0/4111424897 learned_addr learned my addr 172.21.15.160:0/4111424897 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:32:13.863 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.118+0000 7fd7baffd700 1 -- 172.21.15.160:0/4111424897 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7bc108ce0 con 0x7fd7bc107ae0 2024-03-07T07:32:13.863 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.118+0000 7fd7baffd700 1 --2- 172.21.15.160:0/4111424897 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7bc107ae0 0x7fd7bc107f00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac0098a0 tx=0x7fd7ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=437110975a2d5c6c server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:32:13.863 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.118+0000 7fd7b9ffb700 1 -- 172.21.15.160:0/4111424897 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7ac004160 con 0x7fd7bc107ae0 2024-03-07T07:32:13.863 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.118+0000 7fd7b9ffb700 1 -- 172.21.15.160:0/4111424897 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd7ac004300 con 0x7fd7bc107ae0 2024-03-07T07:32:13.863 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.118+0000 7fd7b9ffb700 1 -- 172.21.15.160:0/4111424897 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7ac011b20 con 0x7fd7bc107ae0 2024-03-07T07:32:13.863 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.119+0000 7fd7c1957700 1 -- 172.21.15.160:0/4111424897 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7bc107ae0 msgr2=0x7fd7bc107f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:13.863 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.119+0000 7fd7c1957700 1 --2- 172.21.15.160:0/4111424897 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7bc107ae0 0x7fd7bc107f00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac0098a0 tx=0x7fd7ac009bb0 comp rx=0 tx=0).stop 2024-03-07T07:32:13.863 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.119+0000 7fd7c1957700 1 -- 172.21.15.160:0/4111424897 shutdown_connections 2024-03-07T07:32:13.863 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.119+0000 7fd7c1957700 1 --2- 172.21.15.160:0/4111424897 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7bc107ae0 0x7fd7bc107f00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:13.864 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.119+0000 7fd7c1957700 1 -- 172.21.15.160:0/4111424897 >> 172.21.15.160:0/4111424897 conn(0x7fd7bc103080 msgr2=0x7fd7bc1054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:32:13.864 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.119+0000 7fd7c1957700 1 -- 172.21.15.160:0/4111424897 shutdown_connections 2024-03-07T07:32:13.864 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.119+0000 7fd7c1957700 1 -- 172.21.15.160:0/4111424897 wait complete. 2024-03-07T07:32:13.864 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.120+0000 7fd7c1957700 1 Processor -- start 2024-03-07T07:32:13.864 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.120+0000 7fd7c1957700 1 -- start start 2024-03-07T07:32:13.864 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.120+0000 7fd7c1957700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7bc107ae0 0x7fd7bc07c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:32:13.864 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.120+0000 7fd7c1957700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc10c090 con 0x7fd7bc107ae0 2024-03-07T07:32:13.864 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.120+0000 7fd7baffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7bc107ae0 0x7fd7bc07c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:32:13.864 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.120+0000 7fd7baffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7bc107ae0 0x7fd7bc07c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:36070/0 (socket says 172.21.15.160:36070) 2024-03-07T07:32:13.865 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.120+0000 7fd7baffd700 1 -- 172.21.15.160:0/1479893369 learned_addr learned my addr 172.21.15.160:0/1479893369 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:32:13.865 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.120+0000 7fd7baffd700 1 -- 172.21.15.160:0/1479893369 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7bc07cf30 con 0x7fd7bc107ae0 2024-03-07T07:32:13.865 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.121+0000 7fd7baffd700 1 --2- 172.21.15.160:0/1479893369 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7bc107ae0 0x7fd7bc07c9f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac0117e0 tx=0x7fd7ac011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:32:13.865 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.121+0000 7fd7c0955700 1 -- 172.21.15.160:0/1479893369 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7ac003bf0 con 0x7fd7bc107ae0 2024-03-07T07:32:13.865 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.121+0000 7fd7c1957700 1 -- 172.21.15.160:0/1479893369 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7bc07b170 con 0x7fd7bc107ae0 2024-03-07T07:32:13.865 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.121+0000 7fd7c0955700 1 -- 172.21.15.160:0/1479893369 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd7ac01a910 con 0x7fd7bc107ae0 2024-03-07T07:32:13.865 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.121+0000 7fd7c0955700 1 -- 172.21.15.160:0/1479893369 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7ac0249a0 con 0x7fd7bc107ae0 2024-03-07T07:32:13.865 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.121+0000 7fd7c1957700 1 -- 172.21.15.160:0/1479893369 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7bc07b5d0 con 0x7fd7bc107ae0 2024-03-07T07:32:13.865 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.122+0000 7fd7c0955700 1 -- 172.21.15.160:0/1479893369 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 12) v1 ==== 45940+0+0 (secure 0 0 0) 0x7fd7ac022020 con 0x7fd7bc107ae0 2024-03-07T07:32:13.865 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.122+0000 7fd7c1957700 1 -- 172.21.15.160:0/1479893369 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7bc063780 con 0x7fd7bc107ae0 2024-03-07T07:32:13.865 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.122+0000 7fd7c0955700 1 --2- 172.21.15.160:0/1479893369 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fd7a4038ba0 0x7fd7a403b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:32:13.865 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.122+0000 7fd7c0955700 1 -- 172.21.15.160:0/1479893369 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd7ac04deb0 con 0x7fd7bc107ae0 2024-03-07T07:32:13.866 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.122+0000 7fd7ba7fc700 1 --2- 172.21.15.160:0/1479893369 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fd7a4038ba0 0x7fd7a403b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:32:13.866 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.123+0000 7fd7ba7fc700 1 --2- 172.21.15.160:0/1479893369 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fd7a4038ba0 0x7fd7a403b060 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0006fd0 tx=0x7fd7b0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:32:13.866 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.129+0000 7fd7c0955700 1 -- 172.21.15.160:0/1479893369 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd7ac011e20 con 0x7fd7bc107ae0 2024-03-07T07:32:13.866 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.360+0000 7fd7c1957700 1 -- 172.21.15.160:0/1479893369 --> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fd7bc07bd30 con 0x7fd7a4038ba0 2024-03-07T07:32:13.866 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.585+0000 7fd7c0955700 1 -- 172.21.15.160:0/1479893369 <== mgr.14152 v2:172.21.15.160:6800/419630619 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7fd7bc07bd30 con 0x7fd7a4038ba0 2024-03-07T07:32:13.866 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.588+0000 7fd7c1957700 1 -- 172.21.15.160:0/1479893369 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fd7a4038ba0 msgr2=0x7fd7a403b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:13.866 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.588+0000 7fd7c1957700 1 --2- 172.21.15.160:0/1479893369 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fd7a4038ba0 0x7fd7a403b060 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0006fd0 tx=0x7fd7b0006e40 comp rx=0 tx=0).stop 2024-03-07T07:32:13.866 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.588+0000 7fd7c1957700 1 -- 172.21.15.160:0/1479893369 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7bc107ae0 msgr2=0x7fd7bc07c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:13.866 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.588+0000 7fd7c1957700 1 --2- 172.21.15.160:0/1479893369 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7bc107ae0 0x7fd7bc07c9f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac0117e0 tx=0x7fd7ac011c10 comp rx=0 tx=0).stop 2024-03-07T07:32:13.866 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.588+0000 7fd7c1957700 1 -- 172.21.15.160:0/1479893369 shutdown_connections 2024-03-07T07:32:13.866 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.588+0000 7fd7c1957700 1 --2- 172.21.15.160:0/1479893369 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fd7a4038ba0 0x7fd7a403b060 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:13.866 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.588+0000 7fd7c1957700 1 --2- 172.21.15.160:0/1479893369 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7bc107ae0 0x7fd7bc07c9f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:13.866 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.588+0000 7fd7c1957700 1 -- 172.21.15.160:0/1479893369 >> 172.21.15.160:0/1479893369 conn(0x7fd7bc103080 msgr2=0x7fd7bc103b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:32:13.866 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.588+0000 7fd7c1957700 1 -- 172.21.15.160:0/1479893369 shutdown_connections 2024-03-07T07:32:13.867 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:12.588+0000 7fd7c1957700 1 -- 172.21.15.160:0/1479893369 wait complete. 2024-03-07T07:32:13.867 INFO:teuthology.orchestra.run.smithi160.stdout:Fetching dashboard port number... 2024-03-07T07:32:14.826 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:14 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/1458932573' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-07T07:32:15.101 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stdout 8443 2024-03-07T07:32:15.101 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.257+0000 7f96cff20700 1 Processor -- start 2024-03-07T07:32:15.101 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.257+0000 7f96cff20700 1 -- start start 2024-03-07T07:32:15.101 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.257+0000 7f96cff20700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96c8107fb0 0x7f96c81083d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:32:15.101 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.257+0000 7f96cff20700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c81089a0 con 0x7f96c8107fb0 2024-03-07T07:32:15.101 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.258+0000 7f96cdcbc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96c8107fb0 0x7f96c81083d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:32:15.101 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.258+0000 7f96cdcbc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96c8107fb0 0x7f96c81083d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:36080/0 (socket says 172.21.15.160:36080) 2024-03-07T07:32:15.101 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.258+0000 7f96cdcbc700 1 -- 172.21.15.160:0/2310405181 learned_addr learned my addr 172.21.15.160:0/2310405181 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:32:15.102 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.258+0000 7f96cdcbc700 1 -- 172.21.15.160:0/2310405181 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c8106c10 con 0x7f96c8107fb0 2024-03-07T07:32:15.102 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.258+0000 7f96cdcbc700 1 --2- 172.21.15.160:0/2310405181 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96c8107fb0 0x7f96c81083d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f96c40098a0 tx=0x7f96c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e31aa96c4a705af server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:32:15.102 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.258+0000 7f96cccba700 1 -- 172.21.15.160:0/2310405181 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96c4004160 con 0x7f96c8107fb0 2024-03-07T07:32:15.102 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.258+0000 7f96cccba700 1 -- 172.21.15.160:0/2310405181 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f96c4004300 con 0x7f96c8107fb0 2024-03-07T07:32:15.102 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.258+0000 7f96cccba700 1 -- 172.21.15.160:0/2310405181 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96c4011b20 con 0x7f96c8107fb0 2024-03-07T07:32:15.102 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.259+0000 7f96cff20700 1 -- 172.21.15.160:0/2310405181 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96c8107fb0 msgr2=0x7f96c81083d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:15.102 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.259+0000 7f96cff20700 1 --2- 172.21.15.160:0/2310405181 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96c8107fb0 0x7f96c81083d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f96c40098a0 tx=0x7f96c4009bb0 comp rx=0 tx=0).stop 2024-03-07T07:32:15.102 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.259+0000 7f96cff20700 1 -- 172.21.15.160:0/2310405181 shutdown_connections 2024-03-07T07:32:15.102 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.259+0000 7f96cff20700 1 --2- 172.21.15.160:0/2310405181 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96c8107fb0 0x7f96c81083d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:15.102 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.259+0000 7f96cff20700 1 -- 172.21.15.160:0/2310405181 >> 172.21.15.160:0/2310405181 conn(0x7f96c8101760 msgr2=0x7f96c8103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:32:15.102 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.259+0000 7f96cff20700 1 -- 172.21.15.160:0/2310405181 shutdown_connections 2024-03-07T07:32:15.102 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.259+0000 7f96cff20700 1 -- 172.21.15.160:0/2310405181 wait complete. 2024-03-07T07:32:15.103 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.259+0000 7f96cff20700 1 Processor -- start 2024-03-07T07:32:15.103 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.259+0000 7f96cff20700 1 -- start start 2024-03-07T07:32:15.103 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.260+0000 7f96cff20700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96c8107fb0 0x7f96c810c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:32:15.103 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.260+0000 7f96cff20700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c8105b00 con 0x7f96c8107fb0 2024-03-07T07:32:15.103 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.260+0000 7f96cdcbc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96c8107fb0 0x7f96c810c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:32:15.103 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.260+0000 7f96cdcbc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96c8107fb0 0x7f96c810c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:36088/0 (socket says 172.21.15.160:36088) 2024-03-07T07:32:15.103 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.260+0000 7f96cdcbc700 1 -- 172.21.15.160:0/1458932573 learned_addr learned my addr 172.21.15.160:0/1458932573 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:32:15.103 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.260+0000 7f96cdcbc700 1 -- 172.21.15.160:0/1458932573 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c810ce20 con 0x7f96c8107fb0 2024-03-07T07:32:15.103 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.260+0000 7f96cdcbc700 1 --2- 172.21.15.160:0/1458932573 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96c8107fb0 0x7f96c810c8e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f96c40116a0 tx=0x7f96c4011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:32:15.103 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.260+0000 7f96beffd700 1 -- 172.21.15.160:0/1458932573 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96c4003a40 con 0x7f96c8107fb0 2024-03-07T07:32:15.103 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.260+0000 7f96beffd700 1 -- 172.21.15.160:0/1458932573 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f96c4003be0 con 0x7f96c8107fb0 2024-03-07T07:32:15.103 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.260+0000 7f96cff20700 1 -- 172.21.15.160:0/1458932573 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96c810b010 con 0x7f96c8107fb0 2024-03-07T07:32:15.104 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.260+0000 7f96beffd700 1 -- 172.21.15.160:0/1458932573 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96c401a750 con 0x7f96c8107fb0 2024-03-07T07:32:15.104 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.260+0000 7f96cff20700 1 -- 172.21.15.160:0/1458932573 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96c810b4d0 con 0x7f96c8107fb0 2024-03-07T07:32:15.104 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.261+0000 7f96beffd700 1 -- 172.21.15.160:0/1458932573 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 12) v1 ==== 45940+0+0 (secure 0 0 0) 0x7f96c401aa00 con 0x7f96c8107fb0 2024-03-07T07:32:15.104 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.261+0000 7f96beffd700 1 --2- 172.21.15.160:0/1458932573 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f96b4038c00 0x7f96b403b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:32:15.104 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.261+0000 7f96beffd700 1 -- 172.21.15.160:0/1458932573 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f96c404d130 con 0x7f96c8107fb0 2024-03-07T07:32:15.104 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.261+0000 7f96cff20700 1 -- 172.21.15.160:0/1458932573 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96c8063780 con 0x7f96c8107fb0 2024-03-07T07:32:15.104 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.261+0000 7f96cd4bb700 1 --2- 172.21.15.160:0/1458932573 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f96b4038c00 0x7f96b403b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:32:15.104 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.262+0000 7f96cd4bb700 1 --2- 172.21.15.160:0/1458932573 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f96b4038c00 0x7f96b403b0c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f96b8006fd0 tx=0x7f96b8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:32:15.104 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.266+0000 7f96beffd700 1 -- 172.21.15.160:0/1458932573 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f96c4011e20 con 0x7f96c8107fb0 2024-03-07T07:32:15.104 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.463+0000 7f96cff20700 1 -- 172.21.15.160:0/1458932573 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f96c81952b0 con 0x7f96c8107fb0 2024-03-07T07:32:15.104 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.463+0000 7f96beffd700 1 -- 172.21.15.160:0/1458932573 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f96c400ba30 con 0x7f96c8107fb0 2024-03-07T07:32:15.104 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.465+0000 7f96cff20700 1 -- 172.21.15.160:0/1458932573 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f96b4038c00 msgr2=0x7f96b403b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:15.104 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.465+0000 7f96cff20700 1 --2- 172.21.15.160:0/1458932573 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f96b4038c00 0x7f96b403b0c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f96b8006fd0 tx=0x7f96b8006e40 comp rx=0 tx=0).stop 2024-03-07T07:32:15.104 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.465+0000 7f96cff20700 1 -- 172.21.15.160:0/1458932573 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96c8107fb0 msgr2=0x7f96c810c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:15.105 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.465+0000 7f96cff20700 1 --2- 172.21.15.160:0/1458932573 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96c8107fb0 0x7f96c810c8e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f96c40116a0 tx=0x7f96c4011ba0 comp rx=0 tx=0).stop 2024-03-07T07:32:15.105 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.465+0000 7f96cff20700 1 -- 172.21.15.160:0/1458932573 shutdown_connections 2024-03-07T07:32:15.105 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.465+0000 7f96cff20700 1 --2- 172.21.15.160:0/1458932573 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f96b4038c00 0x7f96b403b0c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:15.105 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.465+0000 7f96cff20700 1 --2- 172.21.15.160:0/1458932573 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96c8107fb0 0x7f96c810c8e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:15.105 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.465+0000 7f96cff20700 1 -- 172.21.15.160:0/1458932573 >> 172.21.15.160:0/1458932573 conn(0x7f96c8101760 msgr2=0x7f96c81023d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:32:15.105 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.465+0000 7f96cff20700 1 -- 172.21.15.160:0/1458932573 shutdown_connections 2024-03-07T07:32:15.105 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:14.465+0000 7f96cff20700 1 -- 172.21.15.160:0/1458932573 wait complete. 2024-03-07T07:32:15.108 INFO:teuthology.orchestra.run.smithi160.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-07T07:32:15.108 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stdout disabled 2024-03-07T07:32:15.115 INFO:teuthology.orchestra.run.smithi160.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-07T07:32:15.115 INFO:teuthology.orchestra.run.smithi160.stdout:systemctl: stdout inactive 2024-03-07T07:32:15.115 INFO:teuthology.orchestra.run.smithi160.stdout:firewalld.service is not enabled 2024-03-07T07:32:15.115 INFO:teuthology.orchestra.run.smithi160.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-07T07:32:15.117 INFO:teuthology.orchestra.run.smithi160.stdout:Ceph Dashboard is now available at: 2024-03-07T07:32:15.117 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:32:15.118 INFO:teuthology.orchestra.run.smithi160.stdout: URL: https://smithi160.front.sepia.ceph.com:8443/ 2024-03-07T07:32:15.118 INFO:teuthology.orchestra.run.smithi160.stdout: User: admin 2024-03-07T07:32:15.118 INFO:teuthology.orchestra.run.smithi160.stdout: Password: vtoz8qsy8j 2024-03-07T07:32:15.118 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:32:15.118 INFO:teuthology.orchestra.run.smithi160.stdout:Saving cluster configuration to /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/config directory 2024-03-07T07:32:15.118 INFO:teuthology.orchestra.run.smithi160.stdout:Enabling autotune for osd_memory_target 2024-03-07T07:32:15.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:15 smithi160 ceph-mon[31963]: mgrmap e13: x(active, since 6s) 2024-03-07T07:32:16.391 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.572+0000 7f6781f50700 1 Processor -- start 2024-03-07T07:32:16.391 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.572+0000 7f6781f50700 1 -- start start 2024-03-07T07:32:16.391 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.572+0000 7f6781f50700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f677c106a00 0x7f677c106e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:32:16.391 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.572+0000 7f6781f50700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677c1073f0 con 0x7f677c106a00 2024-03-07T07:32:16.391 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.572+0000 7f677b7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f677c106a00 0x7f677c106e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:32:16.391 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.572+0000 7f677b7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f677c106a00 0x7f677c106e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:36102/0 (socket says 172.21.15.160:36102) 2024-03-07T07:32:16.391 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.572+0000 7f677b7fe700 1 -- 172.21.15.160:0/2247696024 learned_addr learned my addr 172.21.15.160:0/2247696024 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:32:16.392 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.573+0000 7f677b7fe700 1 -- 172.21.15.160:0/2247696024 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f677c107c00 con 0x7f677c106a00 2024-03-07T07:32:16.392 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.573+0000 7f677b7fe700 1 --2- 172.21.15.160:0/2247696024 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f677c106a00 0x7f677c106e20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f676c0098a0 tx=0x7f676c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd91fb7bbfe6999c server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:32:16.392 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.573+0000 7f677a7fc700 1 -- 172.21.15.160:0/2247696024 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f676c004160 con 0x7f677c106a00 2024-03-07T07:32:16.392 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.573+0000 7f677a7fc700 1 -- 172.21.15.160:0/2247696024 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f676c004300 con 0x7f677c106a00 2024-03-07T07:32:16.392 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.573+0000 7f677a7fc700 1 -- 172.21.15.160:0/2247696024 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f676c011b20 con 0x7f677c106a00 2024-03-07T07:32:16.392 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.574+0000 7f6781f50700 1 -- 172.21.15.160:0/2247696024 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f677c106a00 msgr2=0x7f677c106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:16.392 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.574+0000 7f6781f50700 1 --2- 172.21.15.160:0/2247696024 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f677c106a00 0x7f677c106e20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f676c0098a0 tx=0x7f676c009bb0 comp rx=0 tx=0).stop 2024-03-07T07:32:16.392 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.574+0000 7f6781f50700 1 -- 172.21.15.160:0/2247696024 shutdown_connections 2024-03-07T07:32:16.392 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.574+0000 7f6781f50700 1 --2- 172.21.15.160:0/2247696024 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f677c106a00 0x7f677c106e20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:16.392 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.574+0000 7f6781f50700 1 -- 172.21.15.160:0/2247696024 >> 172.21.15.160:0/2247696024 conn(0x7f677c101fc0 msgr2=0x7f677c1043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:32:16.392 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.574+0000 7f6781f50700 1 -- 172.21.15.160:0/2247696024 shutdown_connections 2024-03-07T07:32:16.392 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.574+0000 7f6781f50700 1 -- 172.21.15.160:0/2247696024 wait complete. 2024-03-07T07:32:16.393 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.574+0000 7f6781f50700 1 Processor -- start 2024-03-07T07:32:16.393 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.575+0000 7f6781f50700 1 -- start start 2024-03-07T07:32:16.393 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.575+0000 7f6781f50700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f677c106a00 0x7f677c19bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:32:16.393 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.575+0000 7f6781f50700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677c10afb0 con 0x7f677c106a00 2024-03-07T07:32:16.393 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.575+0000 7f677b7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f677c106a00 0x7f677c19bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:32:16.393 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.575+0000 7f677b7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f677c106a00 0x7f677c19bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:36112/0 (socket says 172.21.15.160:36112) 2024-03-07T07:32:16.393 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.575+0000 7f677b7fe700 1 -- 172.21.15.160:0/2360850320 learned_addr learned my addr 172.21.15.160:0/2360850320 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:32:16.393 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.575+0000 7f677b7fe700 1 -- 172.21.15.160:0/2360850320 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f677c19c040 con 0x7f677c106a00 2024-03-07T07:32:16.393 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.575+0000 7f677b7fe700 1 --2- 172.21.15.160:0/2360850320 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f677c106a00 0x7f677c19bb00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f676c0116a0 tx=0x7f676c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:32:16.393 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.575+0000 7f6778ff9700 1 -- 172.21.15.160:0/2360850320 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f676c003a40 con 0x7f677c106a00 2024-03-07T07:32:16.393 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.575+0000 7f6778ff9700 1 -- 172.21.15.160:0/2360850320 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f676c003be0 con 0x7f677c106a00 2024-03-07T07:32:16.393 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.575+0000 7f6781f50700 1 -- 172.21.15.160:0/2360850320 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f677c19c2d0 con 0x7f677c106a00 2024-03-07T07:32:16.393 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.575+0000 7f6778ff9700 1 -- 172.21.15.160:0/2360850320 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f676c01a720 con 0x7f677c106a00 2024-03-07T07:32:16.393 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.575+0000 7f6781f50700 1 -- 172.21.15.160:0/2360850320 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f677c19c730 con 0x7f677c106a00 2024-03-07T07:32:16.394 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.576+0000 7f6778ff9700 1 -- 172.21.15.160:0/2360850320 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7f676c01aa20 con 0x7f677c106a00 2024-03-07T07:32:16.394 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.576+0000 7f6781f50700 1 -- 172.21.15.160:0/2360850320 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f677c063780 con 0x7f677c106a00 2024-03-07T07:32:16.394 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.576+0000 7f6778ff9700 1 --2- 172.21.15.160:0/2360850320 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f6764038c40 0x7f676403b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:32:16.394 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.576+0000 7f6778ff9700 1 -- 172.21.15.160:0/2360850320 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f676c04d3c0 con 0x7f677c106a00 2024-03-07T07:32:16.394 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.577+0000 7f677affd700 1 --2- 172.21.15.160:0/2360850320 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f6764038c40 0x7f676403b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:32:16.394 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.577+0000 7f677affd700 1 --2- 172.21.15.160:0/2360850320 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f6764038c40 0x7f676403b100 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6770006fd0 tx=0x7f6770006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:32:16.394 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.581+0000 7f6778ff9700 1 -- 172.21.15.160:0/2360850320 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f676c011e20 con 0x7f677c106a00 2024-03-07T07:32:16.394 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.779+0000 7f6781f50700 1 -- 172.21.15.160:0/2360850320 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f677c062d60 con 0x7f677c106a00 2024-03-07T07:32:16.394 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.779+0000 7f6778ff9700 1 -- 172.21.15.160:0/2360850320 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f676c02c350 con 0x7f677c106a00 2024-03-07T07:32:16.394 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.781+0000 7f6781f50700 1 -- 172.21.15.160:0/2360850320 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f6764038c40 msgr2=0x7f676403b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:16.394 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.781+0000 7f6781f50700 1 --2- 172.21.15.160:0/2360850320 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f6764038c40 0x7f676403b100 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6770006fd0 tx=0x7f6770006e40 comp rx=0 tx=0).stop 2024-03-07T07:32:16.394 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.781+0000 7f6781f50700 1 -- 172.21.15.160:0/2360850320 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f677c106a00 msgr2=0x7f677c19bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:16.395 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.781+0000 7f6781f50700 1 --2- 172.21.15.160:0/2360850320 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f677c106a00 0x7f677c19bb00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f676c0116a0 tx=0x7f676c011ba0 comp rx=0 tx=0).stop 2024-03-07T07:32:16.395 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.781+0000 7f6781f50700 1 -- 172.21.15.160:0/2360850320 shutdown_connections 2024-03-07T07:32:16.395 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.781+0000 7f6781f50700 1 --2- 172.21.15.160:0/2360850320 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f6764038c40 0x7f676403b100 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:16.395 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.781+0000 7f6781f50700 1 --2- 172.21.15.160:0/2360850320 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f677c106a00 0x7f677c19bb00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:16.395 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.781+0000 7f6781f50700 1 -- 172.21.15.160:0/2360850320 >> 172.21.15.160:0/2360850320 conn(0x7f677c101fc0 msgr2=0x7f677c102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:32:16.395 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.781+0000 7f6781f50700 1 -- 172.21.15.160:0/2360850320 shutdown_connections 2024-03-07T07:32:16.395 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:15.781+0000 7f6781f50700 1 -- 172.21.15.160:0/2360850320 wait complete. 2024-03-07T07:32:17.713 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:16.787+0000 7f57645a3700 1 Processor -- start 2024-03-07T07:32:17.713 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:16.787+0000 7f57645a3700 1 -- start start 2024-03-07T07:32:17.713 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:16.787+0000 7f57645a3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f575c1085c0 0x7f575c10a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:32:17.713 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:16.788+0000 7f57645a3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f575c07be50 con 0x7f575c1085c0 2024-03-07T07:32:17.713 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:16.788+0000 7f576233f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f575c1085c0 0x7f575c10a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:32:17.714 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:16.788+0000 7f576233f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f575c1085c0 0x7f575c10a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:36128/0 (socket says 172.21.15.160:36128) 2024-03-07T07:32:17.714 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:16.788+0000 7f576233f700 1 -- 172.21.15.160:0/1484499665 learned_addr learned my addr 172.21.15.160:0/1484499665 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:32:17.714 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:16.788+0000 7f576233f700 1 -- 172.21.15.160:0/1484499665 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f575c10aef0 con 0x7f575c1085c0 2024-03-07T07:32:17.714 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:16.788+0000 7f576233f700 1 --2- 172.21.15.160:0/1484499665 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f575c1085c0 0x7f575c10a9b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f57580098a0 tx=0x7f5758009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d5c57531d9b3c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:32:17.714 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:16.788+0000 7f576133d700 1 -- 172.21.15.160:0/1484499665 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5758004160 con 0x7f575c1085c0 2024-03-07T07:32:17.714 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:16.788+0000 7f576133d700 1 -- 172.21.15.160:0/1484499665 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5758004300 con 0x7f575c1085c0 2024-03-07T07:32:17.714 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:16.788+0000 7f576133d700 1 -- 172.21.15.160:0/1484499665 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5758011b20 con 0x7f575c1085c0 2024-03-07T07:32:17.714 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:16.789+0000 7f57645a3700 1 -- 172.21.15.160:0/1484499665 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f575c1085c0 msgr2=0x7f575c10a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:17.714 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:16.789+0000 7f57645a3700 1 --2- 172.21.15.160:0/1484499665 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f575c1085c0 0x7f575c10a9b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f57580098a0 tx=0x7f5758009bb0 comp rx=0 tx=0).stop 2024-03-07T07:32:17.714 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:16.789+0000 7f57645a3700 1 -- 172.21.15.160:0/1484499665 shutdown_connections 2024-03-07T07:32:17.714 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:16.789+0000 7f57645a3700 1 --2- 172.21.15.160:0/1484499665 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f575c1085c0 0x7f575c10a9b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:17.714 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:16.789+0000 7f57645a3700 1 -- 172.21.15.160:0/1484499665 >> 172.21.15.160:0/1484499665 conn(0x7f575c101ff0 msgr2=0x7f575c104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:32:17.714 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:16.789+0000 7f57645a3700 1 -- 172.21.15.160:0/1484499665 shutdown_connections 2024-03-07T07:32:17.714 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:16.789+0000 7f57645a3700 1 -- 172.21.15.160:0/1484499665 wait complete. 2024-03-07T07:32:17.715 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:16.790+0000 7f57645a3700 1 Processor -- start 2024-03-07T07:32:17.715 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:16.790+0000 7f57645a3700 1 -- start start 2024-03-07T07:32:17.715 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:16.790+0000 7f57645a3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f575c1085c0 0x7f575c107e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:32:17.715 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:16.790+0000 7f57645a3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f575c10bf50 con 0x7f575c1085c0 2024-03-07T07:32:17.715 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:16.790+0000 7f576233f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f575c1085c0 0x7f575c107e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:32:17.715 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:16.790+0000 7f576233f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f575c1085c0 0x7f575c107e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:36140/0 (socket says 172.21.15.160:36140) 2024-03-07T07:32:17.715 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:16.790+0000 7f576233f700 1 -- 172.21.15.160:0/198146994 learned_addr learned my addr 172.21.15.160:0/198146994 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:32:17.715 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:16.790+0000 7f576233f700 1 -- 172.21.15.160:0/198146994 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f575c108380 con 0x7f575c1085c0 2024-03-07T07:32:17.715 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:16.790+0000 7f576233f700 1 --2- 172.21.15.160:0/198146994 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f575c1085c0 0x7f575c107e40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5758011b60 tx=0x7f5758011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:32:17.715 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:16.790+0000 7f57537fe700 1 -- 172.21.15.160:0/198146994 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5758003b60 con 0x7f575c1085c0 2024-03-07T07:32:17.715 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:16.791+0000 7f57537fe700 1 -- 172.21.15.160:0/198146994 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f575801a910 con 0x7f575c1085c0 2024-03-07T07:32:17.715 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:16.791+0000 7f57645a3700 1 -- 172.21.15.160:0/198146994 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f575c106570 con 0x7f575c1085c0 2024-03-07T07:32:17.715 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:16.791+0000 7f57537fe700 1 -- 172.21.15.160:0/198146994 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5758024930 con 0x7f575c1085c0 2024-03-07T07:32:17.715 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:16.791+0000 7f57645a3700 1 -- 172.21.15.160:0/198146994 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f575c106a30 con 0x7f575c1085c0 2024-03-07T07:32:17.716 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:16.791+0000 7f57537fe700 1 -- 172.21.15.160:0/198146994 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7f5758022020 con 0x7f575c1085c0 2024-03-07T07:32:17.716 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:16.792+0000 7f57645a3700 1 -- 172.21.15.160:0/198146994 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f575c063780 con 0x7f575c1085c0 2024-03-07T07:32:17.716 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:16.792+0000 7f57537fe700 1 --2- 172.21.15.160:0/198146994 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f5748038cf0 0x7f574803b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:32:17.716 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:16.792+0000 7f57537fe700 1 -- 172.21.15.160:0/198146994 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f575804cdd0 con 0x7f575c1085c0 2024-03-07T07:32:17.716 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:16.792+0000 7f5761b3e700 1 --2- 172.21.15.160:0/198146994 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f5748038cf0 0x7f574803b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:32:17.716 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:16.792+0000 7f5761b3e700 1 --2- 172.21.15.160:0/198146994 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f5748038cf0 0x7f574803b1b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f574c006fd0 tx=0x7f574c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:32:17.716 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:16.797+0000 7f57537fe700 1 -- 172.21.15.160:0/198146994 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f57580212c0 con 0x7f575c1085c0 2024-03-07T07:32:17.716 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:17.098+0000 7f57645a3700 1 -- 172.21.15.160:0/198146994 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f575c106d70 con 0x7f575c1085c0 2024-03-07T07:32:17.716 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:17.101+0000 7f57537fe700 1 -- 172.21.15.160:0/198146994 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f575c106d70 con 0x7f575c1085c0 2024-03-07T07:32:17.716 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:17.103+0000 7f57645a3700 1 -- 172.21.15.160:0/198146994 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f5748038cf0 msgr2=0x7f574803b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:17.716 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:17.103+0000 7f57645a3700 1 --2- 172.21.15.160:0/198146994 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f5748038cf0 0x7f574803b1b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f574c006fd0 tx=0x7f574c006e40 comp rx=0 tx=0).stop 2024-03-07T07:32:17.716 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:17.103+0000 7f57645a3700 1 -- 172.21.15.160:0/198146994 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f575c1085c0 msgr2=0x7f575c107e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:17.716 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:17.103+0000 7f57645a3700 1 --2- 172.21.15.160:0/198146994 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f575c1085c0 0x7f575c107e40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5758011b60 tx=0x7f5758011c40 comp rx=0 tx=0).stop 2024-03-07T07:32:17.717 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:17.103+0000 7f57645a3700 1 -- 172.21.15.160:0/198146994 shutdown_connections 2024-03-07T07:32:17.717 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:17.103+0000 7f57645a3700 1 --2- 172.21.15.160:0/198146994 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f5748038cf0 0x7f574803b1b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:17.717 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:17.103+0000 7f57645a3700 1 --2- 172.21.15.160:0/198146994 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f575c1085c0 0x7f575c107e40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:17.717 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:17.103+0000 7f57645a3700 1 -- 172.21.15.160:0/198146994 >> 172.21.15.160:0/198146994 conn(0x7f575c101ff0 msgr2=0x7f575c102ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:32:17.717 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:17.103+0000 7f57645a3700 1 -- 172.21.15.160:0/198146994 shutdown_connections 2024-03-07T07:32:17.717 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr 2024-03-07T07:32:17.103+0000 7f57645a3700 1 -- 172.21.15.160:0/198146994 wait complete. 2024-03-07T07:32:17.717 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-07T07:32:17.717 INFO:teuthology.orchestra.run.smithi160.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-07T07:32:17.717 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:32:17.717 INFO:teuthology.orchestra.run.smithi160.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-07T07:32:17.717 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:32:17.717 INFO:teuthology.orchestra.run.smithi160.stdout:Or, if you are only running a single cluster on this host: 2024-03-07T07:32:17.717 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:32:17.717 INFO:teuthology.orchestra.run.smithi160.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-07T07:32:17.718 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:32:17.718 INFO:teuthology.orchestra.run.smithi160.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-07T07:32:17.718 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:32:17.718 INFO:teuthology.orchestra.run.smithi160.stdout: ceph telemetry on 2024-03-07T07:32:17.718 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:32:17.718 INFO:teuthology.orchestra.run.smithi160.stdout:For more information see: 2024-03-07T07:32:17.718 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:32:17.718 INFO:teuthology.orchestra.run.smithi160.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-07T07:32:17.718 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:32:17.718 INFO:teuthology.orchestra.run.smithi160.stdout:Bootstrap complete. 2024-03-07T07:32:17.751 INFO:tasks.cephadm:Fetching config... 2024-03-07T07:32:17.751 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-03-07T07:32:17.751 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-07T07:32:17.766 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-07T07:32:17.766 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-03-07T07:32:17.766 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-07T07:32:17.821 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-07T07:32:17.822 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-03-07T07:32:17.822 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-07T07:32:17.889 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-07T07:32:17.889 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-03-07T07:32:17.889 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-07T07:32:17.943 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-07T07:32:17.944 DEBUG:teuthology.orchestra.run.smithi160:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDlldiONkKqY+THenf2nA7MYac60F5ox53Go88wJkC1j9uHHzwc7zD8GnFOIaj1Ztv/DgFLrUGhw0v8c0K6lzv57txMtfCMajOTJGmTiIjXejHm+kTP+mAfduB9Jy2m+wTN2NWhkaI3QLjKt0XYv+NZmeZLrwhQEaHiVwwzle5OQ+PAyaNeFllDJYgze+g09yJfOi6l0GBPdb40pZkdQIi1Ch1FGscB57fUxovgUypkwlzt1h8XrVJHVBYCzsArhxovd63RYV/DFprigczv3vJoSjNLnivyZfOeNLFj6beZq6svm3KbksdUS6vJVKE4OT7L9ppvu1mjHcWP1dCrNlvYjpwSI0rdRD3UVdXJ9KfNEVuL4IaWMKS2etuEImvQdiaf3t+KJuJ+INS7BuJjnRA3fSUmn/ImkRl3ccAiNW3mN0rV6d7bfKQfdal4oulh/HUwsvpOT+tkO8WTQdzoCJGDgStwUheNZvhIHcNtSzdyAT4SHsKMjpy2e90squcuYwM= ceph-826f6582-dc54-11ee-95c6-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-07T07:32:18.022 INFO:teuthology.orchestra.run.smithi160.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDlldiONkKqY+THenf2nA7MYac60F5ox53Go88wJkC1j9uHHzwc7zD8GnFOIaj1Ztv/DgFLrUGhw0v8c0K6lzv57txMtfCMajOTJGmTiIjXejHm+kTP+mAfduB9Jy2m+wTN2NWhkaI3QLjKt0XYv+NZmeZLrwhQEaHiVwwzle5OQ+PAyaNeFllDJYgze+g09yJfOi6l0GBPdb40pZkdQIi1Ch1FGscB57fUxovgUypkwlzt1h8XrVJHVBYCzsArhxovd63RYV/DFprigczv3vJoSjNLnivyZfOeNLFj6beZq6svm3KbksdUS6vJVKE4OT7L9ppvu1mjHcWP1dCrNlvYjpwSI0rdRD3UVdXJ9KfNEVuL4IaWMKS2etuEImvQdiaf3t+KJuJ+INS7BuJjnRA3fSUmn/ImkRl3ccAiNW3mN0rV6d7bfKQfdal4oulh/HUwsvpOT+tkO8WTQdzoCJGDgStwUheNZvhIHcNtSzdyAT4SHsKMjpy2e90squcuYwM= ceph-826f6582-dc54-11ee-95c6-87774f69a715 2024-03-07T07:32:18.038 DEBUG:teuthology.orchestra.run.smithi177:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDlldiONkKqY+THenf2nA7MYac60F5ox53Go88wJkC1j9uHHzwc7zD8GnFOIaj1Ztv/DgFLrUGhw0v8c0K6lzv57txMtfCMajOTJGmTiIjXejHm+kTP+mAfduB9Jy2m+wTN2NWhkaI3QLjKt0XYv+NZmeZLrwhQEaHiVwwzle5OQ+PAyaNeFllDJYgze+g09yJfOi6l0GBPdb40pZkdQIi1Ch1FGscB57fUxovgUypkwlzt1h8XrVJHVBYCzsArhxovd63RYV/DFprigczv3vJoSjNLnivyZfOeNLFj6beZq6svm3KbksdUS6vJVKE4OT7L9ppvu1mjHcWP1dCrNlvYjpwSI0rdRD3UVdXJ9KfNEVuL4IaWMKS2etuEImvQdiaf3t+KJuJ+INS7BuJjnRA3fSUmn/ImkRl3ccAiNW3mN0rV6d7bfKQfdal4oulh/HUwsvpOT+tkO8WTQdzoCJGDgStwUheNZvhIHcNtSzdyAT4SHsKMjpy2e90squcuYwM= ceph-826f6582-dc54-11ee-95c6-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-07T07:32:18.082 INFO:teuthology.orchestra.run.smithi177.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDlldiONkKqY+THenf2nA7MYac60F5ox53Go88wJkC1j9uHHzwc7zD8GnFOIaj1Ztv/DgFLrUGhw0v8c0K6lzv57txMtfCMajOTJGmTiIjXejHm+kTP+mAfduB9Jy2m+wTN2NWhkaI3QLjKt0XYv+NZmeZLrwhQEaHiVwwzle5OQ+PAyaNeFllDJYgze+g09yJfOi6l0GBPdb40pZkdQIi1Ch1FGscB57fUxovgUypkwlzt1h8XrVJHVBYCzsArhxovd63RYV/DFprigczv3vJoSjNLnivyZfOeNLFj6beZq6svm3KbksdUS6vJVKE4OT7L9ppvu1mjHcWP1dCrNlvYjpwSI0rdRD3UVdXJ9KfNEVuL4IaWMKS2etuEImvQdiaf3t+KJuJ+INS7BuJjnRA3fSUmn/ImkRl3ccAiNW3mN0rV6d7bfKQfdal4oulh/HUwsvpOT+tkO8WTQdzoCJGDgStwUheNZvhIHcNtSzdyAT4SHsKMjpy2e90squcuYwM= ceph-826f6582-dc54-11ee-95c6-87774f69a715 2024-03-07T07:32:18.097 DEBUG:teuthology.orchestra.run.smithi178:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDlldiONkKqY+THenf2nA7MYac60F5ox53Go88wJkC1j9uHHzwc7zD8GnFOIaj1Ztv/DgFLrUGhw0v8c0K6lzv57txMtfCMajOTJGmTiIjXejHm+kTP+mAfduB9Jy2m+wTN2NWhkaI3QLjKt0XYv+NZmeZLrwhQEaHiVwwzle5OQ+PAyaNeFllDJYgze+g09yJfOi6l0GBPdb40pZkdQIi1Ch1FGscB57fUxovgUypkwlzt1h8XrVJHVBYCzsArhxovd63RYV/DFprigczv3vJoSjNLnivyZfOeNLFj6beZq6svm3KbksdUS6vJVKE4OT7L9ppvu1mjHcWP1dCrNlvYjpwSI0rdRD3UVdXJ9KfNEVuL4IaWMKS2etuEImvQdiaf3t+KJuJ+INS7BuJjnRA3fSUmn/ImkRl3ccAiNW3mN0rV6d7bfKQfdal4oulh/HUwsvpOT+tkO8WTQdzoCJGDgStwUheNZvhIHcNtSzdyAT4SHsKMjpy2e90squcuYwM= ceph-826f6582-dc54-11ee-95c6-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-07T07:32:18.137 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:18 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/198146994' entity='client.admin' 2024-03-07T07:32:18.144 INFO:teuthology.orchestra.run.smithi178.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDlldiONkKqY+THenf2nA7MYac60F5ox53Go88wJkC1j9uHHzwc7zD8GnFOIaj1Ztv/DgFLrUGhw0v8c0K6lzv57txMtfCMajOTJGmTiIjXejHm+kTP+mAfduB9Jy2m+wTN2NWhkaI3QLjKt0XYv+NZmeZLrwhQEaHiVwwzle5OQ+PAyaNeFllDJYgze+g09yJfOi6l0GBPdb40pZkdQIi1Ch1FGscB57fUxovgUypkwlzt1h8XrVJHVBYCzsArhxovd63RYV/DFprigczv3vJoSjNLnivyZfOeNLFj6beZq6svm3KbksdUS6vJVKE4OT7L9ppvu1mjHcWP1dCrNlvYjpwSI0rdRD3UVdXJ9KfNEVuL4IaWMKS2etuEImvQdiaf3t+KJuJ+INS7BuJjnRA3fSUmn/ImkRl3ccAiNW3mN0rV6d7bfKQfdal4oulh/HUwsvpOT+tkO8WTQdzoCJGDgStwUheNZvhIHcNtSzdyAT4SHsKMjpy2e90squcuYwM= ceph-826f6582-dc54-11ee-95c6-87774f69a715 2024-03-07T07:32:18.160 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-07T07:32:18.407 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:32:20.909 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:20 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:20.909 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:20 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:20.909 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:20 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi160", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:32:20.910 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:20 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:20.910 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:20 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:32:20.910 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:20 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:20.933 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:20.931+0000 7f2969602700 1 -- 172.21.15.160:0/2300694349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2964076180 msgr2=0x7f29640745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:20.933 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:20.931+0000 7f2969602700 1 --2- 172.21.15.160:0/2300694349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2964076180 0x7f29640745e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2954009910 tx=0x7f2954009c20 comp rx=0 tx=0).stop 2024-03-07T07:32:20.933 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:20.931+0000 7f2969602700 1 -- 172.21.15.160:0/2300694349 shutdown_connections 2024-03-07T07:32:20.933 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:20.931+0000 7f2969602700 1 --2- 172.21.15.160:0/2300694349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2964076180 0x7f29640745e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:20.933 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:20.931+0000 7f2969602700 1 -- 172.21.15.160:0/2300694349 >> 172.21.15.160:0/2300694349 conn(0x7f29640fd180 msgr2=0x7f29640ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:32:20.933 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:20.931+0000 7f2969602700 1 -- 172.21.15.160:0/2300694349 shutdown_connections 2024-03-07T07:32:20.933 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:20.931+0000 7f2969602700 1 -- 172.21.15.160:0/2300694349 wait complete. 2024-03-07T07:32:20.933 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:20.932+0000 7f2969602700 1 Processor -- start 2024-03-07T07:32:20.933 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:20.932+0000 7f2969602700 1 -- start start 2024-03-07T07:32:20.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:20.932+0000 7f2969602700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2964076180 0x7f2964199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:32:20.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:20.932+0000 7f2969602700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2964109e50 con 0x7f2964076180 2024-03-07T07:32:20.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:20.932+0000 7f2962ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2964076180 0x7f2964199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:32:20.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:20.932+0000 7f2962ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2964076180 0x7f2964199490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59134/0 (socket says 172.21.15.160:59134) 2024-03-07T07:32:20.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:20.932+0000 7f2962ffd700 1 -- 172.21.15.160:0/3252057743 learned_addr learned my addr 172.21.15.160:0/3252057743 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:32:20.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:20.933+0000 7f2962ffd700 1 -- 172.21.15.160:0/3252057743 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29641999d0 con 0x7f2964076180 2024-03-07T07:32:20.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:20.933+0000 7f2962ffd700 1 --2- 172.21.15.160:0/3252057743 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2964076180 0x7f2964199490 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2954006010 tx=0x7f2954005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:32:20.935 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:20.933+0000 7f294bfff700 1 -- 172.21.15.160:0/3252057743 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2954027070 con 0x7f2964076180 2024-03-07T07:32:20.935 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:20.933+0000 7f2969602700 1 -- 172.21.15.160:0/3252057743 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2964199c60 con 0x7f2964076180 2024-03-07T07:32:20.935 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:20.933+0000 7f294bfff700 1 -- 172.21.15.160:0/3252057743 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f295400be40 con 0x7f2964076180 2024-03-07T07:32:20.935 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:20.933+0000 7f294bfff700 1 -- 172.21.15.160:0/3252057743 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f295400f460 con 0x7f2964076180 2024-03-07T07:32:20.935 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:20.933+0000 7f2969602700 1 -- 172.21.15.160:0/3252057743 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f296419c950 con 0x7f2964076180 2024-03-07T07:32:20.935 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:20.934+0000 7f294bfff700 1 -- 172.21.15.160:0/3252057743 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7f295402a070 con 0x7f2964076180 2024-03-07T07:32:20.935 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:20.934+0000 7f294bfff700 1 --2- 172.21.15.160:0/3252057743 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f294c038c40 0x7f294c03b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:32:20.936 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:20.934+0000 7f294bfff700 1 -- 172.21.15.160:0/3252057743 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f295401e030 con 0x7f2964076180 2024-03-07T07:32:20.936 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:20.934+0000 7f2969602700 1 -- 172.21.15.160:0/3252057743 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f296410a5d0 con 0x7f2964076180 2024-03-07T07:32:20.936 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:20.934+0000 7f29627fc700 1 --2- 172.21.15.160:0/3252057743 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f294c038c40 0x7f294c03b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:32:20.936 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:20.935+0000 7f29627fc700 1 --2- 172.21.15.160:0/3252057743 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f294c038c40 0x7f294c03b100 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2958006fd0 tx=0x7f2958006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:32:20.940 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:20.940+0000 7f294bfff700 1 -- 172.21.15.160:0/3252057743 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f2954013070 con 0x7f2964076180 2024-03-07T07:32:21.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:21.142+0000 7f2969602700 1 -- 172.21.15.160:0/3252057743 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f296402d9f0 con 0x7f2964076180 2024-03-07T07:32:21.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:21.145+0000 7f294bfff700 1 -- 172.21.15.160:0/3252057743 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f2954031020 con 0x7f2964076180 2024-03-07T07:32:21.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:21.147+0000 7f2969602700 1 -- 172.21.15.160:0/3252057743 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f294c038c40 msgr2=0x7f294c03b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:21.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:21.147+0000 7f2969602700 1 --2- 172.21.15.160:0/3252057743 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f294c038c40 0x7f294c03b100 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2958006fd0 tx=0x7f2958006e40 comp rx=0 tx=0).stop 2024-03-07T07:32:21.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:21.147+0000 7f2969602700 1 -- 172.21.15.160:0/3252057743 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2964076180 msgr2=0x7f2964199490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:21.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:21.147+0000 7f2969602700 1 --2- 172.21.15.160:0/3252057743 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2964076180 0x7f2964199490 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2954006010 tx=0x7f2954005050 comp rx=0 tx=0).stop 2024-03-07T07:32:21.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:21.147+0000 7f2969602700 1 -- 172.21.15.160:0/3252057743 shutdown_connections 2024-03-07T07:32:21.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:21.147+0000 7f2969602700 1 --2- 172.21.15.160:0/3252057743 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f294c038c40 0x7f294c03b100 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:21.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:21.147+0000 7f2969602700 1 --2- 172.21.15.160:0/3252057743 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2964076180 0x7f2964199490 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:21.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:21.147+0000 7f2969602700 1 -- 172.21.15.160:0/3252057743 >> 172.21.15.160:0/3252057743 conn(0x7f29640fd180 msgr2=0x7f29640fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:32:21.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:21.147+0000 7f2969602700 1 -- 172.21.15.160:0/3252057743 shutdown_connections 2024-03-07T07:32:21.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:21.147+0000 7f2969602700 1 -- 172.21.15.160:0/3252057743 wait complete. 2024-03-07T07:32:21.790 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-07T07:32:21.790 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-07T07:32:22.106 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:32:22.427 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:22 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/3252057743' entity='client.admin' 2024-03-07T07:32:22.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:22 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:32:22.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:22 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:23.925 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:23.923+0000 7f3d000b0700 1 -- 172.21.15.160:0/1062353875 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3cf8102600 msgr2=0x7f3cf81049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:23.925 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:23.923+0000 7f3d000b0700 1 --2- 172.21.15.160:0/1062353875 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3cf8102600 0x7f3cf81049f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8009910 tx=0x7f3ce8009c20 comp rx=0 tx=0).stop 2024-03-07T07:32:23.925 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:23.923+0000 7f3d000b0700 1 -- 172.21.15.160:0/1062353875 shutdown_connections 2024-03-07T07:32:23.925 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:23.923+0000 7f3d000b0700 1 --2- 172.21.15.160:0/1062353875 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3cf8102600 0x7f3cf81049f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:23.925 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:23.923+0000 7f3d000b0700 1 -- 172.21.15.160:0/1062353875 >> 172.21.15.160:0/1062353875 conn(0x7f3cf80fc050 msgr2=0x7f3cf80fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:32:23.925 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:23.923+0000 7f3d000b0700 1 -- 172.21.15.160:0/1062353875 shutdown_connections 2024-03-07T07:32:23.925 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:23.923+0000 7f3d000b0700 1 -- 172.21.15.160:0/1062353875 wait complete. 2024-03-07T07:32:23.925 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:23.924+0000 7f3d000b0700 1 Processor -- start 2024-03-07T07:32:23.926 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:23.924+0000 7f3d000b0700 1 -- start start 2024-03-07T07:32:23.926 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:23.924+0000 7f3d000b0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3cf8102600 0x7f3cf8101e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:32:23.926 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:23.924+0000 7f3d000b0700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf8107ba0 con 0x7f3cf8102600 2024-03-07T07:32:23.926 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:23.924+0000 7f3cfde4c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3cf8102600 0x7f3cf8101e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:32:23.926 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:23.924+0000 7f3cfde4c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3cf8102600 0x7f3cf8101e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59166/0 (socket says 172.21.15.160:59166) 2024-03-07T07:32:23.926 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:23.924+0000 7f3cfde4c700 1 -- 172.21.15.160:0/1059367455 learned_addr learned my addr 172.21.15.160:0/1059367455 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:32:23.926 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:23.924+0000 7f3cfde4c700 1 -- 172.21.15.160:0/1059367455 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf81023c0 con 0x7f3cf8102600 2024-03-07T07:32:23.926 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:23.924+0000 7f3cfde4c700 1 --2- 172.21.15.160:0/1059367455 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3cf8102600 0x7f3cf8101e80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8004fa0 tx=0x7f3ce8005080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:32:23.926 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:23.924+0000 7f3cf6ffd700 1 -- 172.21.15.160:0/1059367455 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3ce8027070 con 0x7f3cf8102600 2024-03-07T07:32:23.926 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:23.924+0000 7f3cf6ffd700 1 -- 172.21.15.160:0/1059367455 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3ce800be40 con 0x7f3cf8102600 2024-03-07T07:32:23.926 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:23.924+0000 7f3d000b0700 1 -- 172.21.15.160:0/1059367455 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cf81005b0 con 0x7f3cf8102600 2024-03-07T07:32:23.926 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:23.924+0000 7f3cf6ffd700 1 -- 172.21.15.160:0/1059367455 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3ce800f460 con 0x7f3cf8102600 2024-03-07T07:32:23.926 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:23.924+0000 7f3d000b0700 1 -- 172.21.15.160:0/1059367455 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cf8100af0 con 0x7f3cf8102600 2024-03-07T07:32:23.927 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:23.925+0000 7f3cf6ffd700 1 -- 172.21.15.160:0/1059367455 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7f3ce802a070 con 0x7f3cf8102600 2024-03-07T07:32:23.927 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:23.925+0000 7f3d000b0700 1 -- 172.21.15.160:0/1059367455 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cf8108320 con 0x7f3cf8102600 2024-03-07T07:32:23.927 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:23.925+0000 7f3cf6ffd700 1 --2- 172.21.15.160:0/1059367455 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f3ce0038c40 0x7f3ce003b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:32:23.928 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:23.926+0000 7f3cf6ffd700 1 -- 172.21.15.160:0/1059367455 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3ce801e030 con 0x7f3cf8102600 2024-03-07T07:32:23.928 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:23.926+0000 7f3cfd64b700 1 --2- 172.21.15.160:0/1059367455 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f3ce0038c40 0x7f3ce003b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:32:23.928 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:23.926+0000 7f3cfd64b700 1 --2- 172.21.15.160:0/1059367455 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f3ce0038c40 0x7f3ce003b100 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3cec006fd0 tx=0x7f3cec006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:32:23.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:23.930+0000 7f3cf6ffd700 1 -- 172.21.15.160:0/1059367455 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3ce8013070 con 0x7f3cf8102600 2024-03-07T07:32:24.131 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:24.130+0000 7f3d000b0700 1 -- 172.21.15.160:0/1059367455 --> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f3cf8190860 con 0x7f3ce0038c40 2024-03-07T07:32:24.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:24.134+0000 7f3cf6ffd700 1 -- 172.21.15.160:0/1059367455 <== mgr.14152 v2:172.21.15.160:6800/419630619 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3cf8190860 con 0x7f3ce0038c40 2024-03-07T07:32:24.136 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:24.136+0000 7f3d000b0700 1 -- 172.21.15.160:0/1059367455 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f3ce0038c40 msgr2=0x7f3ce003b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:24.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:24.136+0000 7f3d000b0700 1 --2- 172.21.15.160:0/1059367455 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f3ce0038c40 0x7f3ce003b100 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3cec006fd0 tx=0x7f3cec006e40 comp rx=0 tx=0).stop 2024-03-07T07:32:24.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:24.136+0000 7f3d000b0700 1 -- 172.21.15.160:0/1059367455 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3cf8102600 msgr2=0x7f3cf8101e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:24.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:24.136+0000 7f3d000b0700 1 --2- 172.21.15.160:0/1059367455 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3cf8102600 0x7f3cf8101e80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8004fa0 tx=0x7f3ce8005080 comp rx=0 tx=0).stop 2024-03-07T07:32:24.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:24.136+0000 7f3d000b0700 1 -- 172.21.15.160:0/1059367455 shutdown_connections 2024-03-07T07:32:24.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:24.136+0000 7f3d000b0700 1 --2- 172.21.15.160:0/1059367455 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f3ce0038c40 0x7f3ce003b100 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:24.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:24.136+0000 7f3d000b0700 1 --2- 172.21.15.160:0/1059367455 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3cf8102600 0x7f3cf8101e80 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:24.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:24.136+0000 7f3d000b0700 1 -- 172.21.15.160:0/1059367455 >> 172.21.15.160:0/1059367455 conn(0x7f3cf80fc050 msgr2=0x7f3cf80fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:32:24.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:24.136+0000 7f3d000b0700 1 -- 172.21.15.160:0/1059367455 shutdown_connections 2024-03-07T07:32:24.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:24.136+0000 7f3d000b0700 1 -- 172.21.15.160:0/1059367455 wait complete. 2024-03-07T07:32:24.844 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi177 2024-03-07T07:32:24.844 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-03-07T07:32:24.845 DEBUG:teuthology.orchestra.run.smithi177:> dd of=/etc/ceph/ceph.conf 2024-03-07T07:32:24.864 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-03-07T07:32:24.864 DEBUG:teuthology.orchestra.run.smithi177:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-07T07:32:24.921 INFO:tasks.cephadm:Adding host smithi177 to orchestrator... 2024-03-07T07:32:24.922 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph orch host add smithi177 2024-03-07T07:32:25.164 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:32:25.393 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:25 smithi160 ceph-mon[31963]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:32:25.393 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:25 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:25.394 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:25 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:32:25.394 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:25 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:32:25.394 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:25 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:32:25.394 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:25 smithi160 ceph-mon[31963]: Updating smithi160:/etc/ceph/ceph.conf 2024-03-07T07:32:25.394 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:25 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:25.394 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:25 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:25.394 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:25 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:26.389 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:26 smithi160 ceph-mon[31963]: Updating smithi160:/var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-07T07:32:26.389 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:26 smithi160 ceph-mon[31963]: Updating smithi160:/etc/ceph/ceph.client.admin.keyring 2024-03-07T07:32:26.389 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:26 smithi160 ceph-mon[31963]: Updating smithi160:/var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-07T07:32:26.696 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:26.695+0000 7f42a9bd0700 1 -- 172.21.15.160:0/1817689706 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42a4107be0 msgr2=0x7f42a4107fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:26.696 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:26.695+0000 7f42a9bd0700 1 --2- 172.21.15.160:0/1817689706 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42a4107be0 0x7f42a4107fc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4298009910 tx=0x7f4298009c20 comp rx=0 tx=0).stop 2024-03-07T07:32:26.696 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:26.695+0000 7f42a9bd0700 1 -- 172.21.15.160:0/1817689706 shutdown_connections 2024-03-07T07:32:26.696 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:26.695+0000 7f42a9bd0700 1 --2- 172.21.15.160:0/1817689706 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42a4107be0 0x7f42a4107fc0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:26.696 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:26.695+0000 7f42a9bd0700 1 -- 172.21.15.160:0/1817689706 >> 172.21.15.160:0/1817689706 conn(0x7f42a40fd620 msgr2=0x7f42a40ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:32:26.697 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:26.695+0000 7f42a9bd0700 1 -- 172.21.15.160:0/1817689706 shutdown_connections 2024-03-07T07:32:26.697 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:26.695+0000 7f42a9bd0700 1 -- 172.21.15.160:0/1817689706 wait complete. 2024-03-07T07:32:26.697 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:26.696+0000 7f42a9bd0700 1 Processor -- start 2024-03-07T07:32:26.697 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:26.696+0000 7f42a9bd0700 1 -- start start 2024-03-07T07:32:26.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:26.696+0000 7f42a9bd0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42a4107be0 0x7f42a4197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:32:26.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:26.696+0000 7f42a9bd0700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a410c1a0 con 0x7f42a4107be0 2024-03-07T07:32:26.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:26.696+0000 7f42a37fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42a4107be0 0x7f42a4197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:32:26.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:26.696+0000 7f42a37fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42a4107be0 0x7f42a4197620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59180/0 (socket says 172.21.15.160:59180) 2024-03-07T07:32:26.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:26.696+0000 7f42a37fe700 1 -- 172.21.15.160:0/3524163860 learned_addr learned my addr 172.21.15.160:0/3524163860 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:32:26.699 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:26.697+0000 7f42a37fe700 1 -- 172.21.15.160:0/3524163860 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a4197b60 con 0x7f42a4107be0 2024-03-07T07:32:26.699 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:26.697+0000 7f42a37fe700 1 --2- 172.21.15.160:0/3524163860 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42a4107be0 0x7f42a4197620 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4298005850 tx=0x7f4298005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:32:26.699 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:26.697+0000 7f42a0ff9700 1 -- 172.21.15.160:0/3524163860 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4298026070 con 0x7f42a4107be0 2024-03-07T07:32:26.699 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:26.697+0000 7f42a9bd0700 1 -- 172.21.15.160:0/3524163860 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42a4197df0 con 0x7f42a4107be0 2024-03-07T07:32:26.699 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:26.697+0000 7f42a0ff9700 1 -- 172.21.15.160:0/3524163860 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f429800bc30 con 0x7f42a4107be0 2024-03-07T07:32:26.699 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:26.697+0000 7f42a0ff9700 1 -- 172.21.15.160:0/3524163860 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4298003a20 con 0x7f42a4107be0 2024-03-07T07:32:26.699 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:26.697+0000 7f42a9bd0700 1 -- 172.21.15.160:0/3524163860 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42a4198250 con 0x7f42a4107be0 2024-03-07T07:32:26.699 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:26.698+0000 7f42a9bd0700 1 -- 172.21.15.160:0/3524163860 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42a41031e0 con 0x7f42a4107be0 2024-03-07T07:32:26.699 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:26.698+0000 7f42a0ff9700 1 -- 172.21.15.160:0/3524163860 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7f4298029070 con 0x7f42a4107be0 2024-03-07T07:32:26.700 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:26.698+0000 7f42a0ff9700 1 --2- 172.21.15.160:0/3524163860 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f428c038c40 0x7f428c03b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:32:26.700 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:26.698+0000 7f42a0ff9700 1 -- 172.21.15.160:0/3524163860 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f429801e070 con 0x7f42a4107be0 2024-03-07T07:32:26.700 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:26.698+0000 7f42a2ffd700 1 --2- 172.21.15.160:0/3524163860 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f428c038c40 0x7f428c03b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:32:26.700 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:26.699+0000 7f42a2ffd700 1 --2- 172.21.15.160:0/3524163860 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f428c038c40 0x7f428c03b100 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4294006fd0 tx=0x7f4294006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:32:26.703 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:26.703+0000 7f42a0ff9700 1 -- 172.21.15.160:0/3524163860 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4298013070 con 0x7f42a4107be0 2024-03-07T07:32:26.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:26.902+0000 7f42a9bd0700 1 -- 172.21.15.160:0/3524163860 --> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi177", "target": ["mon-mgr", ""]}) v1 -- 0x7f42a41989f0 con 0x7f428c038c40 2024-03-07T07:32:27.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:27 smithi160 ceph-mon[31963]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi177", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:32:28.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:28 smithi160 ceph-mon[31963]: Deploying cephadm binary to smithi177 2024-03-07T07:32:28.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:28 smithi160 ceph-mon[31963]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:32:31.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:31.012+0000 7f42a0ff9700 1 -- 172.21.15.160:0/3524163860 <== mgr.14152 v2:172.21.15.160:6800/419630619 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f42a41989f0 con 0x7f428c038c40 2024-03-07T07:32:31.013 INFO:teuthology.orchestra.run.smithi160.stdout:Added host 'smithi177' with addr '172.21.15.177' 2024-03-07T07:32:31.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:31.014+0000 7f42a9bd0700 1 -- 172.21.15.160:0/3524163860 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f428c038c40 msgr2=0x7f428c03b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:31.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:31.014+0000 7f42a9bd0700 1 --2- 172.21.15.160:0/3524163860 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f428c038c40 0x7f428c03b100 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4294006fd0 tx=0x7f4294006e40 comp rx=0 tx=0).stop 2024-03-07T07:32:31.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:31.014+0000 7f42a9bd0700 1 -- 172.21.15.160:0/3524163860 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42a4107be0 msgr2=0x7f42a4197620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:31.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:31.014+0000 7f42a9bd0700 1 --2- 172.21.15.160:0/3524163860 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42a4107be0 0x7f42a4197620 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4298005850 tx=0x7f4298005030 comp rx=0 tx=0).stop 2024-03-07T07:32:31.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:31.014+0000 7f42a9bd0700 1 -- 172.21.15.160:0/3524163860 shutdown_connections 2024-03-07T07:32:31.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:31.014+0000 7f42a9bd0700 1 --2- 172.21.15.160:0/3524163860 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f428c038c40 0x7f428c03b100 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:31.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:31.014+0000 7f42a9bd0700 1 --2- 172.21.15.160:0/3524163860 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42a4107be0 0x7f42a4197620 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:31.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:31.014+0000 7f42a9bd0700 1 -- 172.21.15.160:0/3524163860 >> 172.21.15.160:0/3524163860 conn(0x7f42a40fd620 msgr2=0x7f42a40fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:32:31.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:31.014+0000 7f42a9bd0700 1 -- 172.21.15.160:0/3524163860 shutdown_connections 2024-03-07T07:32:31.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:31.014+0000 7f42a9bd0700 1 -- 172.21.15.160:0/3524163860 wait complete. 2024-03-07T07:32:31.427 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:31 smithi160 ceph-mon[31963]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:32:31.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:31 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:31.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:31 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:32:31.548 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph orch host ls --format=json 2024-03-07T07:32:31.784 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:32:32.618 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:32 smithi160 ceph-mon[31963]: Added host smithi177 2024-03-07T07:32:32.618 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:32 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:32.618 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:32 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:33.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:33.433+0000 7f4dc1bf9700 1 -- 172.21.15.160:0/2911629439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4dbc107bf0 msgr2=0x7f4dbc107fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:33.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:33.433+0000 7f4dc1bf9700 1 --2- 172.21.15.160:0/2911629439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4dbc107bf0 0x7f4dbc107fd0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4dac009910 tx=0x7f4dac009c20 comp rx=0 tx=0).stop 2024-03-07T07:32:33.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:33.433+0000 7f4dc1bf9700 1 -- 172.21.15.160:0/2911629439 shutdown_connections 2024-03-07T07:32:33.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:33.433+0000 7f4dc1bf9700 1 --2- 172.21.15.160:0/2911629439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4dbc107bf0 0x7f4dbc107fd0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:33.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:33.433+0000 7f4dc1bf9700 1 -- 172.21.15.160:0/2911629439 >> 172.21.15.160:0/2911629439 conn(0x7f4dbc0fd650 msgr2=0x7f4dbc0ffa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:32:33.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:33.433+0000 7f4dc1bf9700 1 -- 172.21.15.160:0/2911629439 shutdown_connections 2024-03-07T07:32:33.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:33.433+0000 7f4dc1bf9700 1 -- 172.21.15.160:0/2911629439 wait complete. 2024-03-07T07:32:33.435 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:33 smithi160 ceph-mon[31963]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:32:33.437 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:33.433+0000 7f4dc1bf9700 1 Processor -- start 2024-03-07T07:32:33.437 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:33.434+0000 7f4dc1bf9700 1 -- start start 2024-03-07T07:32:33.437 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:33.434+0000 7f4dc1bf9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4dbc107bf0 0x7f4dbc1975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:32:33.437 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:33.434+0000 7f4dc1bf9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dbc10c1b0 con 0x7f4dbc107bf0 2024-03-07T07:32:33.437 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:33.434+0000 7f4dbb7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4dbc107bf0 0x7f4dbc1975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:32:33.437 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:33.434+0000 7f4dbb7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4dbc107bf0 0x7f4dbc1975f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:38072/0 (socket says 172.21.15.160:38072) 2024-03-07T07:32:33.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:33.434+0000 7f4dbb7fe700 1 -- 172.21.15.160:0/3120090015 learned_addr learned my addr 172.21.15.160:0/3120090015 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:32:33.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:33.434+0000 7f4dbb7fe700 1 -- 172.21.15.160:0/3120090015 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dbc197b30 con 0x7f4dbc107bf0 2024-03-07T07:32:33.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:33.435+0000 7f4dbb7fe700 1 --2- 172.21.15.160:0/3120090015 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4dbc107bf0 0x7f4dbc1975f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4dac005850 tx=0x7f4dac005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:32:33.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:33.435+0000 7f4db8ff9700 1 -- 172.21.15.160:0/3120090015 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4dac026070 con 0x7f4dbc107bf0 2024-03-07T07:32:33.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:33.435+0000 7f4dc1bf9700 1 -- 172.21.15.160:0/3120090015 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dbc197dc0 con 0x7f4dbc107bf0 2024-03-07T07:32:33.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:33.435+0000 7f4db8ff9700 1 -- 172.21.15.160:0/3120090015 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4dac00bc30 con 0x7f4dbc107bf0 2024-03-07T07:32:33.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:33.435+0000 7f4db8ff9700 1 -- 172.21.15.160:0/3120090015 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4dac003a20 con 0x7f4dbc107bf0 2024-03-07T07:32:33.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:33.435+0000 7f4dc1bf9700 1 -- 172.21.15.160:0/3120090015 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dbc19bec0 con 0x7f4dbc107bf0 2024-03-07T07:32:33.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:33.436+0000 7f4db8ff9700 1 -- 172.21.15.160:0/3120090015 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7f4dac029070 con 0x7f4dbc107bf0 2024-03-07T07:32:33.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:33.436+0000 7f4dc1bf9700 1 -- 172.21.15.160:0/3120090015 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dbc1031f0 con 0x7f4dbc107bf0 2024-03-07T07:32:33.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:33.436+0000 7f4db8ff9700 1 --2- 172.21.15.160:0/3120090015 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f4da4038c40 0x7f4da403b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:32:33.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:33.436+0000 7f4db8ff9700 1 -- 172.21.15.160:0/3120090015 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4dac01e070 con 0x7f4dbc107bf0 2024-03-07T07:32:33.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:33.436+0000 7f4dbaffd700 1 --2- 172.21.15.160:0/3120090015 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f4da4038c40 0x7f4da403b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:32:33.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:33.436+0000 7f4dbaffd700 1 --2- 172.21.15.160:0/3120090015 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f4da4038c40 0x7f4da403b100 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4db0006fd0 tx=0x7f4db0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:32:33.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:33.441+0000 7f4db8ff9700 1 -- 172.21.15.160:0/3120090015 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4dac013070 con 0x7f4dbc107bf0 2024-03-07T07:32:33.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:33.653+0000 7f4dc1bf9700 1 -- 172.21.15.160:0/3120090015 --> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4dbc198950 con 0x7f4da4038c40 2024-03-07T07:32:33.655 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:33.655+0000 7f4db8ff9700 1 -- 172.21.15.160:0/3120090015 <== mgr.14152 v2:172.21.15.160:6800/419630619 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+161 (secure 0 0 0) 0x7f4dbc198950 con 0x7f4da4038c40 2024-03-07T07:32:33.655 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:32:33.655 INFO:teuthology.orchestra.run.smithi160.stdout:[{"addr": "172.21.15.160", "hostname": "smithi160", "labels": [], "status": ""}, {"addr": "172.21.15.177", "hostname": "smithi177", "labels": [], "status": ""}] 2024-03-07T07:32:33.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:33.657+0000 7f4dc1bf9700 1 -- 172.21.15.160:0/3120090015 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f4da4038c40 msgr2=0x7f4da403b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:33.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:33.657+0000 7f4dc1bf9700 1 --2- 172.21.15.160:0/3120090015 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f4da4038c40 0x7f4da403b100 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4db0006fd0 tx=0x7f4db0006e40 comp rx=0 tx=0).stop 2024-03-07T07:32:33.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:33.657+0000 7f4dc1bf9700 1 -- 172.21.15.160:0/3120090015 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4dbc107bf0 msgr2=0x7f4dbc1975f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:33.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:33.657+0000 7f4dc1bf9700 1 --2- 172.21.15.160:0/3120090015 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4dbc107bf0 0x7f4dbc1975f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4dac005850 tx=0x7f4dac005030 comp rx=0 tx=0).stop 2024-03-07T07:32:33.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:33.657+0000 7f4dc1bf9700 1 -- 172.21.15.160:0/3120090015 shutdown_connections 2024-03-07T07:32:33.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:33.657+0000 7f4dc1bf9700 1 --2- 172.21.15.160:0/3120090015 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f4da4038c40 0x7f4da403b100 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:33.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:33.657+0000 7f4dc1bf9700 1 --2- 172.21.15.160:0/3120090015 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4dbc107bf0 0x7f4dbc1975f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:33.658 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:33.657+0000 7f4dc1bf9700 1 -- 172.21.15.160:0/3120090015 >> 172.21.15.160:0/3120090015 conn(0x7f4dbc0fd650 msgr2=0x7f4dbc0fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:32:33.658 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:33.657+0000 7f4dc1bf9700 1 -- 172.21.15.160:0/3120090015 shutdown_connections 2024-03-07T07:32:33.658 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:33.657+0000 7f4dc1bf9700 1 -- 172.21.15.160:0/3120090015 wait complete. 2024-03-07T07:32:34.239 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi178 2024-03-07T07:32:34.239 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-03-07T07:32:34.239 DEBUG:teuthology.orchestra.run.smithi178:> dd of=/etc/ceph/ceph.conf 2024-03-07T07:32:34.257 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-03-07T07:32:34.257 DEBUG:teuthology.orchestra.run.smithi178:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-07T07:32:34.314 INFO:tasks.cephadm:Adding host smithi178 to orchestrator... 2024-03-07T07:32:34.314 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph orch host add smithi178 2024-03-07T07:32:34.549 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:32:34.835 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:34 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:34.835 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:34 smithi160 ceph-mon[31963]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-07T07:32:34.835 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:34 smithi160 ceph-mon[31963]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:32:36.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:36.050+0000 7f12f6839700 1 -- 172.21.15.160:0/3277928719 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12f0106a30 msgr2=0x7f12f0106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:36.052 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:36.050+0000 7f12f6839700 1 --2- 172.21.15.160:0/3277928719 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12f0106a30 0x7f12f0106e10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f12e4009910 tx=0x7f12e4009c20 comp rx=0 tx=0).stop 2024-03-07T07:32:36.052 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:36.050+0000 7f12f6839700 1 -- 172.21.15.160:0/3277928719 shutdown_connections 2024-03-07T07:32:36.052 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:36.050+0000 7f12f6839700 1 --2- 172.21.15.160:0/3277928719 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12f0106a30 0x7f12f0106e10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:36.052 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:36.050+0000 7f12f6839700 1 -- 172.21.15.160:0/3277928719 >> 172.21.15.160:0/3277928719 conn(0x7f12f00fc580 msgr2=0x7f12f00fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:32:36.052 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:36.050+0000 7f12f6839700 1 -- 172.21.15.160:0/3277928719 shutdown_connections 2024-03-07T07:32:36.052 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:36.050+0000 7f12f6839700 1 -- 172.21.15.160:0/3277928719 wait complete. 2024-03-07T07:32:36.052 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:36.051+0000 7f12f6839700 1 Processor -- start 2024-03-07T07:32:36.052 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:36.051+0000 7f12f6839700 1 -- start start 2024-03-07T07:32:36.052 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:36.051+0000 7f12f6839700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12f0106a30 0x7f12f0197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:32:36.052 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:36.051+0000 7f12f6839700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12f010b070 con 0x7f12f0106a30 2024-03-07T07:32:36.052 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:36.051+0000 7f12effff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12f0106a30 0x7f12f0197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:32:36.052 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:36.051+0000 7f12effff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12f0106a30 0x7f12f0197640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:38100/0 (socket says 172.21.15.160:38100) 2024-03-07T07:32:36.053 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:36.051+0000 7f12effff700 1 -- 172.21.15.160:0/3260455077 learned_addr learned my addr 172.21.15.160:0/3260455077 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:32:36.053 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:36.051+0000 7f12effff700 1 -- 172.21.15.160:0/3260455077 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f0197b80 con 0x7f12f0106a30 2024-03-07T07:32:36.053 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:36.051+0000 7f12effff700 1 --2- 172.21.15.160:0/3260455077 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12f0106a30 0x7f12f0197640 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f12e4005850 tx=0x7f12e4005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:32:36.054 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:36.052+0000 7f12ed7fa700 1 -- 172.21.15.160:0/3260455077 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f12e4026070 con 0x7f12f0106a30 2024-03-07T07:32:36.054 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:36.052+0000 7f12ed7fa700 1 -- 172.21.15.160:0/3260455077 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f12e400bbd0 con 0x7f12f0106a30 2024-03-07T07:32:36.054 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:36.052+0000 7f12f6839700 1 -- 172.21.15.160:0/3260455077 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12f0197e70 con 0x7f12f0106a30 2024-03-07T07:32:36.054 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:36.052+0000 7f12ed7fa700 1 -- 172.21.15.160:0/3260455077 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f12e4003970 con 0x7f12f0106a30 2024-03-07T07:32:36.055 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:36.052+0000 7f12f6839700 1 -- 172.21.15.160:0/3260455077 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12f019bf10 con 0x7f12f0106a30 2024-03-07T07:32:36.055 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:36.052+0000 7f12ed7fa700 1 -- 172.21.15.160:0/3260455077 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7f12e4029070 con 0x7f12f0106a30 2024-03-07T07:32:36.055 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:36.053+0000 7f12ed7fa700 1 --2- 172.21.15.160:0/3260455077 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f12d8038ca0 0x7f12d803b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:32:36.055 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:36.053+0000 7f12ed7fa700 1 -- 172.21.15.160:0/3260455077 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f12e401e070 con 0x7f12f0106a30 2024-03-07T07:32:36.055 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:36.053+0000 7f12ef7fe700 1 --2- 172.21.15.160:0/3260455077 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f12d8038ca0 0x7f12d803b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:32:36.055 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:36.053+0000 7f12f6839700 1 -- 172.21.15.160:0/3260455077 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12f0102170 con 0x7f12f0106a30 2024-03-07T07:32:36.055 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:36.053+0000 7f12ef7fe700 1 --2- 172.21.15.160:0/3260455077 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f12d8038ca0 0x7f12d803b160 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f12e0006fd0 tx=0x7f12e0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:32:36.058 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:36.058+0000 7f12ed7fa700 1 -- 172.21.15.160:0/3260455077 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f12e4030070 con 0x7f12f0106a30 2024-03-07T07:32:36.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:36.256+0000 7f12f6839700 1 -- 172.21.15.160:0/3260455077 --> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi178", "target": ["mon-mgr", ""]}) v1 -- 0x7f12f0063780 con 0x7f12d8038ca0 2024-03-07T07:32:37.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:37 smithi160 ceph-mon[31963]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:32:37.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:37 smithi160 ceph-mon[31963]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi178", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:32:37.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:37 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:37.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:37 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:37.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:37 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:37.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:37 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:37.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:37 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi177", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:32:37.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:37 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:32:37.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:37 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:32:38.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:38 smithi160 ceph-mon[31963]: Updating smithi177:/etc/ceph/ceph.conf 2024-03-07T07:32:38.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:38 smithi160 ceph-mon[31963]: Deploying cephadm binary to smithi178 2024-03-07T07:32:38.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:38 smithi160 ceph-mon[31963]: Updating smithi177:/var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-07T07:32:38.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:38 smithi160 ceph-mon[31963]: Updating smithi177:/etc/ceph/ceph.client.admin.keyring 2024-03-07T07:32:39.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:39 smithi160 ceph-mon[31963]: Updating smithi177:/var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-07T07:32:39.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:39 smithi160 ceph-mon[31963]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:32:40.362 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:40.361+0000 7f12ed7fa700 1 -- 172.21.15.160:0/3260455077 <== mgr.14152 v2:172.21.15.160:6800/419630619 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f12f0063780 con 0x7f12d8038ca0 2024-03-07T07:32:40.362 INFO:teuthology.orchestra.run.smithi160.stdout:Added host 'smithi178' with addr '172.21.15.178' 2024-03-07T07:32:40.364 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:40.364+0000 7f12f6839700 1 -- 172.21.15.160:0/3260455077 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f12d8038ca0 msgr2=0x7f12d803b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:40.364 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:40.364+0000 7f12f6839700 1 --2- 172.21.15.160:0/3260455077 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f12d8038ca0 0x7f12d803b160 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f12e0006fd0 tx=0x7f12e0006e40 comp rx=0 tx=0).stop 2024-03-07T07:32:40.364 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:40.364+0000 7f12f6839700 1 -- 172.21.15.160:0/3260455077 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12f0106a30 msgr2=0x7f12f0197640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:40.364 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:40.364+0000 7f12f6839700 1 --2- 172.21.15.160:0/3260455077 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12f0106a30 0x7f12f0197640 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f12e4005850 tx=0x7f12e4005030 comp rx=0 tx=0).stop 2024-03-07T07:32:40.365 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:40.364+0000 7f12f6839700 1 -- 172.21.15.160:0/3260455077 shutdown_connections 2024-03-07T07:32:40.365 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:40.364+0000 7f12f6839700 1 --2- 172.21.15.160:0/3260455077 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f12d8038ca0 0x7f12d803b160 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:40.365 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:40.364+0000 7f12f6839700 1 --2- 172.21.15.160:0/3260455077 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12f0106a30 0x7f12f0197640 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:40.365 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:40.364+0000 7f12f6839700 1 -- 172.21.15.160:0/3260455077 >> 172.21.15.160:0/3260455077 conn(0x7f12f00fc580 msgr2=0x7f12f0104540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:32:40.365 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:40.364+0000 7f12f6839700 1 -- 172.21.15.160:0/3260455077 shutdown_connections 2024-03-07T07:32:40.365 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:40.364+0000 7f12f6839700 1 -- 172.21.15.160:0/3260455077 wait complete. 2024-03-07T07:32:40.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:40 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:40.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:40 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:40.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:40 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:40.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:40 smithi160 ceph-mon[31963]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:32:40.929 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:40 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:40.929 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:40 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:32:40.975 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph orch host ls --format=json 2024-03-07T07:32:41.212 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:32:42.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:41 smithi160 ceph-mon[31963]: Added host smithi178 2024-03-07T07:32:42.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:41 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:42.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:41 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:42.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:42.631+0000 7f5dd628f700 1 -- 172.21.15.160:0/3386281596 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5dd0106a00 msgr2=0x7f5dd0106de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:42.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:42.631+0000 7f5dd628f700 1 --2- 172.21.15.160:0/3386281596 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5dd0106a00 0x7f5dd0106de0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4009910 tx=0x7f5dc4009c20 comp rx=0 tx=0).stop 2024-03-07T07:32:42.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:42.631+0000 7f5dd628f700 1 -- 172.21.15.160:0/3386281596 shutdown_connections 2024-03-07T07:32:42.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:42.631+0000 7f5dd628f700 1 --2- 172.21.15.160:0/3386281596 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5dd0106a00 0x7f5dd0106de0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:42.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:42.631+0000 7f5dd628f700 1 -- 172.21.15.160:0/3386281596 >> 172.21.15.160:0/3386281596 conn(0x7f5dd00777f0 msgr2=0x7f5dd00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:32:42.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:42.631+0000 7f5dd628f700 1 -- 172.21.15.160:0/3386281596 shutdown_connections 2024-03-07T07:32:42.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:42.631+0000 7f5dd628f700 1 -- 172.21.15.160:0/3386281596 wait complete. 2024-03-07T07:32:42.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:42.632+0000 7f5dd628f700 1 Processor -- start 2024-03-07T07:32:42.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:42.632+0000 7f5dd628f700 1 -- start start 2024-03-07T07:32:42.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:42.632+0000 7f5dd628f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5dd0106a00 0x7f5dd0196500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:32:42.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:42.632+0000 7f5dd628f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd010b040 con 0x7f5dd0106a00 2024-03-07T07:32:42.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:42.632+0000 7f5dcffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5dd0106a00 0x7f5dd0196500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:32:42.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:42.632+0000 7f5dcffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5dd0106a00 0x7f5dd0196500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:43436/0 (socket says 172.21.15.160:43436) 2024-03-07T07:32:42.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:42.632+0000 7f5dcffff700 1 -- 172.21.15.160:0/261725381 learned_addr learned my addr 172.21.15.160:0/261725381 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:32:42.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:42.632+0000 7f5dcffff700 1 -- 172.21.15.160:0/261725381 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd0196a40 con 0x7f5dd0106a00 2024-03-07T07:32:42.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:42.633+0000 7f5dcffff700 1 --2- 172.21.15.160:0/261725381 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5dd0106a00 0x7f5dd0196500 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4006010 tx=0x7f5dc4005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:32:42.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:42.633+0000 7f5dcd7fa700 1 -- 172.21.15.160:0/261725381 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5dc4026070 con 0x7f5dd0106a00 2024-03-07T07:32:42.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:42.633+0000 7f5dd628f700 1 -- 172.21.15.160:0/261725381 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dd0196d30 con 0x7f5dd0106a00 2024-03-07T07:32:42.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:42.633+0000 7f5dcd7fa700 1 -- 172.21.15.160:0/261725381 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5dc400bbd0 con 0x7f5dd0106a00 2024-03-07T07:32:42.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:42.633+0000 7f5dcd7fa700 1 -- 172.21.15.160:0/261725381 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5dc4003910 con 0x7f5dd0106a00 2024-03-07T07:32:42.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:42.633+0000 7f5dd628f700 1 -- 172.21.15.160:0/261725381 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dd019add0 con 0x7f5dd0106a00 2024-03-07T07:32:42.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:42.634+0000 7f5dcd7fa700 1 -- 172.21.15.160:0/261725381 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7f5dc4029070 con 0x7f5dd0106a00 2024-03-07T07:32:42.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:42.634+0000 7f5dd628f700 1 -- 172.21.15.160:0/261725381 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dd0102140 con 0x7f5dd0106a00 2024-03-07T07:32:42.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:42.634+0000 7f5dcd7fa700 1 --2- 172.21.15.160:0/261725381 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f5db8038c40 0x7f5db803b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:32:42.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:42.634+0000 7f5dcd7fa700 1 -- 172.21.15.160:0/261725381 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5dc401e070 con 0x7f5dd0106a00 2024-03-07T07:32:42.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:42.634+0000 7f5dcf7fe700 1 --2- 172.21.15.160:0/261725381 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f5db8038c40 0x7f5db803b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:32:42.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:42.634+0000 7f5dcf7fe700 1 --2- 172.21.15.160:0/261725381 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f5db8038c40 0x7f5db803b100 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0006fd0 tx=0x7f5dc0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:32:42.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:42.639+0000 7f5dcd7fa700 1 -- 172.21.15.160:0/261725381 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5dc4013070 con 0x7f5dd0106a00 2024-03-07T07:32:42.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:42.845+0000 7f5dd628f700 1 -- 172.21.15.160:0/261725381 --> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5dd0197860 con 0x7f5db8038c40 2024-03-07T07:32:42.847 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:32:42.847 INFO:teuthology.orchestra.run.smithi160.stdout:[{"addr": "172.21.15.160", "hostname": "smithi160", "labels": [], "status": ""}, {"addr": "172.21.15.177", "hostname": "smithi177", "labels": [], "status": ""}, {"addr": "172.21.15.178", "hostname": "smithi178", "labels": [], "status": ""}] 2024-03-07T07:32:42.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:42.846+0000 7f5dcd7fa700 1 -- 172.21.15.160:0/261725381 <== mgr.14152 v2:172.21.15.160:6800/419630619 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+241 (secure 0 0 0) 0x7f5dd0197860 con 0x7f5db8038c40 2024-03-07T07:32:42.849 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:42.848+0000 7f5dd628f700 1 -- 172.21.15.160:0/261725381 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f5db8038c40 msgr2=0x7f5db803b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:42.849 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:42.848+0000 7f5dd628f700 1 --2- 172.21.15.160:0/261725381 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f5db8038c40 0x7f5db803b100 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0006fd0 tx=0x7f5dc0006e40 comp rx=0 tx=0).stop 2024-03-07T07:32:42.849 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:42.848+0000 7f5dd628f700 1 -- 172.21.15.160:0/261725381 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5dd0106a00 msgr2=0x7f5dd0196500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:42.849 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:42.848+0000 7f5dd628f700 1 --2- 172.21.15.160:0/261725381 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5dd0106a00 0x7f5dd0196500 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4006010 tx=0x7f5dc4005030 comp rx=0 tx=0).stop 2024-03-07T07:32:42.849 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:42.849+0000 7f5dd628f700 1 -- 172.21.15.160:0/261725381 shutdown_connections 2024-03-07T07:32:42.849 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:42.849+0000 7f5dd628f700 1 --2- 172.21.15.160:0/261725381 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f5db8038c40 0x7f5db803b100 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:42.849 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:42.849+0000 7f5dd628f700 1 --2- 172.21.15.160:0/261725381 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5dd0106a00 0x7f5dd0196500 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:42.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:42.849+0000 7f5dd628f700 1 -- 172.21.15.160:0/261725381 >> 172.21.15.160:0/261725381 conn(0x7f5dd00777f0 msgr2=0x7f5dd0102500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:32:42.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:42.849+0000 7f5dd628f700 1 -- 172.21.15.160:0/261725381 shutdown_connections 2024-03-07T07:32:42.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:42.849+0000 7f5dd628f700 1 -- 172.21.15.160:0/261725381 wait complete. 2024-03-07T07:32:43.177 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:42 smithi160 ceph-mon[31963]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:32:43.494 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-07T07:32:43.494 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph osd crush tunables default 2024-03-07T07:32:43.727 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:32:44.143 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:43 smithi160 ceph-mon[31963]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-07T07:32:44.144 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:43 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:45.000 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:44 smithi160 ceph-mon[31963]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:32:45.216 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:45.211+0000 7efed3b84700 1 -- 172.21.15.160:0/3882705261 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efecc105a20 msgr2=0x7efecc105e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:45.217 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:45.211+0000 7efed3b84700 1 --2- 172.21.15.160:0/3882705261 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efecc105a20 0x7efecc105e00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7efec8009910 tx=0x7efec8009c20 comp rx=0 tx=0).stop 2024-03-07T07:32:45.217 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:45.211+0000 7efed3b84700 1 -- 172.21.15.160:0/3882705261 shutdown_connections 2024-03-07T07:32:45.217 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:45.211+0000 7efed3b84700 1 --2- 172.21.15.160:0/3882705261 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efecc105a20 0x7efecc105e00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:45.217 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:45.211+0000 7efed3b84700 1 -- 172.21.15.160:0/3882705261 >> 172.21.15.160:0/3882705261 conn(0x7efecc0fbcc0 msgr2=0x7efecc0fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:32:45.217 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:45.211+0000 7efed3b84700 1 -- 172.21.15.160:0/3882705261 shutdown_connections 2024-03-07T07:32:45.217 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:45.211+0000 7efed3b84700 1 -- 172.21.15.160:0/3882705261 wait complete. 2024-03-07T07:32:45.217 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:45.212+0000 7efed3b84700 1 Processor -- start 2024-03-07T07:32:45.217 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:45.212+0000 7efed3b84700 1 -- start start 2024-03-07T07:32:45.217 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:45.212+0000 7efed3b84700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efecc105a20 0x7efecc198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:32:45.217 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:45.212+0000 7efed3b84700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efecc10b070 con 0x7efecc105a20 2024-03-07T07:32:45.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:45.212+0000 7efed1920700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efecc105a20 0x7efecc198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:32:45.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:45.212+0000 7efed1920700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efecc105a20 0x7efecc198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:43458/0 (socket says 172.21.15.160:43458) 2024-03-07T07:32:45.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:45.212+0000 7efed1920700 1 -- 172.21.15.160:0/890340355 learned_addr learned my addr 172.21.15.160:0/890340355 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:32:45.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:45.212+0000 7efed1920700 1 -- 172.21.15.160:0/890340355 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efecc1989c0 con 0x7efecc105a20 2024-03-07T07:32:45.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:45.212+0000 7efed1920700 1 --2- 172.21.15.160:0/890340355 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efecc105a20 0x7efecc198480 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efec8005850 tx=0x7efec8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:32:45.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:45.212+0000 7efec2ffd700 1 -- 172.21.15.160:0/890340355 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efec8026070 con 0x7efecc105a20 2024-03-07T07:32:45.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:45.213+0000 7efec2ffd700 1 -- 172.21.15.160:0/890340355 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efec800bc30 con 0x7efecc105a20 2024-03-07T07:32:45.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:45.213+0000 7efed3b84700 1 -- 172.21.15.160:0/890340355 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efecc198c50 con 0x7efecc105a20 2024-03-07T07:32:45.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:45.213+0000 7efec2ffd700 1 -- 172.21.15.160:0/890340355 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efec8003a20 con 0x7efecc105a20 2024-03-07T07:32:45.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:45.213+0000 7efed3b84700 1 -- 172.21.15.160:0/890340355 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efecc1990b0 con 0x7efecc105a20 2024-03-07T07:32:45.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:45.213+0000 7efec2ffd700 1 -- 172.21.15.160:0/890340355 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7efec8029070 con 0x7efecc105a20 2024-03-07T07:32:45.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:45.214+0000 7efed3b84700 1 -- 172.21.15.160:0/890340355 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efecc102160 con 0x7efecc105a20 2024-03-07T07:32:45.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:45.214+0000 7efec2ffd700 1 --2- 172.21.15.160:0/890340355 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7efeb8038ca0 0x7efeb803b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:32:45.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:45.214+0000 7efec2ffd700 1 -- 172.21.15.160:0/890340355 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7efec801e070 con 0x7efecc105a20 2024-03-07T07:32:45.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:45.214+0000 7efed111f700 1 --2- 172.21.15.160:0/890340355 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7efeb8038ca0 0x7efeb803b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:32:45.219 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:45.214+0000 7efed111f700 1 --2- 172.21.15.160:0/890340355 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7efeb8038ca0 0x7efeb803b160 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7efebc006fd0 tx=0x7efebc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:32:45.219 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:45.219+0000 7efec2ffd700 1 -- 172.21.15.160:0/890340355 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7efec8013070 con 0x7efecc105a20 2024-03-07T07:32:45.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:45.418+0000 7efed3b84700 1 -- 172.21.15.160:0/890340355 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7efecc063780 con 0x7efecc105a20 2024-03-07T07:32:45.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:45.947+0000 7efec2ffd700 1 -- 172.21.15.160:0/890340355 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7efec800bde0 con 0x7efecc105a20 2024-03-07T07:32:45.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:45.950+0000 7efed3b84700 1 -- 172.21.15.160:0/890340355 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7efeb8038ca0 msgr2=0x7efeb803b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:45.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:45.950+0000 7efed3b84700 1 --2- 172.21.15.160:0/890340355 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7efeb8038ca0 0x7efeb803b160 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7efebc006fd0 tx=0x7efebc006e40 comp rx=0 tx=0).stop 2024-03-07T07:32:45.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:45.950+0000 7efed3b84700 1 -- 172.21.15.160:0/890340355 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efecc105a20 msgr2=0x7efecc198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:45.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:45.950+0000 7efed3b84700 1 --2- 172.21.15.160:0/890340355 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efecc105a20 0x7efecc198480 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efec8005850 tx=0x7efec8005030 comp rx=0 tx=0).stop 2024-03-07T07:32:45.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:45.950+0000 7efed3b84700 1 -- 172.21.15.160:0/890340355 shutdown_connections 2024-03-07T07:32:45.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:45.950+0000 7efed3b84700 1 --2- 172.21.15.160:0/890340355 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7efeb8038ca0 0x7efeb803b160 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:45.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:45.950+0000 7efed3b84700 1 --2- 172.21.15.160:0/890340355 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efecc105a20 0x7efecc198480 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:45.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:45.950+0000 7efed3b84700 1 -- 172.21.15.160:0/890340355 >> 172.21.15.160:0/890340355 conn(0x7efecc0fbcc0 msgr2=0x7efecc0fc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:32:45.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:45.950+0000 7efed3b84700 1 -- 172.21.15.160:0/890340355 shutdown_connections 2024-03-07T07:32:45.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:32:45.950+0000 7efed3b84700 1 -- 172.21.15.160:0/890340355 wait complete. 2024-03-07T07:32:45.951 INFO:teuthology.orchestra.run.smithi160.stderr:adjusted tunables profile to default 2024-03-07T07:32:46.278 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:45 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/890340355' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-07T07:32:46.606 INFO:tasks.cephadm:Adding mon.a on smithi160 2024-03-07T07:32:46.606 INFO:tasks.cephadm:Adding mon.b on smithi177 2024-03-07T07:32:46.606 INFO:tasks.cephadm:Adding mon.c on smithi178 2024-03-07T07:32:46.606 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph orch apply mon '3;smithi160:172.21.15.160=a;smithi177:172.21.15.177=b;smithi178:172.21.15.178=c' 2024-03-07T07:32:46.839 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-07T07:32:47.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:46 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/890340355' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-07T07:32:47.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:46 smithi160 ceph-mon[31963]: osdmap e4: 0 total, 0 up, 0 in 2024-03-07T07:32:47.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:46 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:47.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:46 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:47.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:46 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:47.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:46 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:47.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:46 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi178", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:32:47.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:46 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:32:47.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:46 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:32:47.429 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:46 smithi160 ceph-mon[31963]: Updating smithi178:/etc/ceph/ceph.conf 2024-03-07T07:32:47.429 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:46 smithi160 ceph-mon[31963]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:32:47.429 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:46 smithi160 ceph-mon[31963]: Updating smithi178:/var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-07T07:32:47.429 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:46 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:47.429 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:46 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:47.429 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:46 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:48.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:48.298+0000 7f454881d700 1 -- 172.21.15.178:0/286199690 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4544106a30 msgr2=0x7f4544106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:48.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:48.298+0000 7f454881d700 1 --2- 172.21.15.178:0/286199690 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4544106a30 0x7f4544106e10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4538009910 tx=0x7f4538009c20 comp rx=0 tx=0).stop 2024-03-07T07:32:48.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:48.299+0000 7f454881d700 1 -- 172.21.15.178:0/286199690 shutdown_connections 2024-03-07T07:32:48.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:48.299+0000 7f454881d700 1 --2- 172.21.15.178:0/286199690 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4544106a30 0x7f4544106e10 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:48.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:48.299+0000 7f454881d700 1 -- 172.21.15.178:0/286199690 >> 172.21.15.178:0/286199690 conn(0x7f45440fc560 msgr2=0x7f45440fe980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:32:48.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:48.299+0000 7f454881d700 1 -- 172.21.15.178:0/286199690 shutdown_connections 2024-03-07T07:32:48.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:48.299+0000 7f454881d700 1 -- 172.21.15.178:0/286199690 wait complete. 2024-03-07T07:32:48.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:48.299+0000 7f454881d700 1 Processor -- start 2024-03-07T07:32:48.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:48.299+0000 7f454881d700 1 -- start start 2024-03-07T07:32:48.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:48.300+0000 7f454881d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4544106a30 0x7f45441965e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:32:48.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:48.300+0000 7f454881d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454410b070 con 0x7f4544106a30 2024-03-07T07:32:48.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:48.300+0000 7f454259c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4544106a30 0x7f45441965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:32:48.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:48.300+0000 7f454259c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4544106a30 0x7f45441965e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.178:41794/0 (socket says 172.21.15.178:41794) 2024-03-07T07:32:48.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:48.300+0000 7f454259c700 1 -- 172.21.15.178:0/3716817260 learned_addr learned my addr 172.21.15.178:0/3716817260 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-03-07T07:32:48.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:48.300+0000 7f454259c700 1 -- 172.21.15.178:0/3716817260 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4544196b20 con 0x7f4544106a30 2024-03-07T07:32:48.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:48.300+0000 7f454259c700 1 --2- 172.21.15.178:0/3716817260 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4544106a30 0x7f45441965e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4538005850 tx=0x7f4538005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:32:48.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:48.300+0000 7f45337fe700 1 -- 172.21.15.178:0/3716817260 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4538026070 con 0x7f4544106a30 2024-03-07T07:32:48.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:48.300+0000 7f45337fe700 1 -- 172.21.15.178:0/3716817260 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f453800bbd0 con 0x7f4544106a30 2024-03-07T07:32:48.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:48.300+0000 7f454881d700 1 -- 172.21.15.178:0/3716817260 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4544196e10 con 0x7f4544106a30 2024-03-07T07:32:48.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:48.300+0000 7f45337fe700 1 -- 172.21.15.178:0/3716817260 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4538003970 con 0x7f4544106a30 2024-03-07T07:32:48.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:48.301+0000 7f454881d700 1 -- 172.21.15.178:0/3716817260 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f454419aeb0 con 0x7f4544106a30 2024-03-07T07:32:48.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:48.301+0000 7f45337fe700 1 -- 172.21.15.178:0/3716817260 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7f4538029070 con 0x7f4544106a30 2024-03-07T07:32:48.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:48.301+0000 7f454881d700 1 -- 172.21.15.178:0/3716817260 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4544102170 con 0x7f4544106a30 2024-03-07T07:32:48.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:48.302+0000 7f45337fe700 1 --2- 172.21.15.178:0/3716817260 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f452c038c40 0x7f452c03b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:32:48.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:48.302+0000 7f45337fe700 1 -- 172.21.15.178:0/3716817260 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f453801e070 con 0x7f4544106a30 2024-03-07T07:32:48.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:48.302+0000 7f4541d9b700 1 --2- 172.21.15.178:0/3716817260 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f452c038c40 0x7f452c03b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:32:48.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:48.302+0000 7f4541d9b700 1 --2- 172.21.15.178:0/3716817260 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f452c038c40 0x7f452c03b100 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4534006fd0 tx=0x7f4534006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:32:48.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:48.307+0000 7f45337fe700 1 -- 172.21.15.178:0/3716817260 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4538013070 con 0x7f4544106a30 2024-03-07T07:32:48.427 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:48 smithi160 ceph-mon[31963]: Updating smithi178:/etc/ceph/ceph.client.admin.keyring 2024-03-07T07:32:48.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:48 smithi160 ceph-mon[31963]: Updating smithi178:/var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-07T07:32:48.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:48.509+0000 7f454881d700 1 -- 172.21.15.178:0/3716817260 --> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi160:172.21.15.160=a;smithi177:172.21.15.177=b;smithi178:172.21.15.178=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f4544197940 con 0x7f452c038c40 2024-03-07T07:32:48.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:48.516+0000 7f45337fe700 1 -- 172.21.15.178:0/3716817260 <== mgr.14152 v2:172.21.15.160:6800/419630619 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f4544197940 con 0x7f452c038c40 2024-03-07T07:32:48.516 INFO:teuthology.orchestra.run.smithi178.stdout:Scheduled mon update... 2024-03-07T07:32:48.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:48.518+0000 7f454881d700 1 -- 172.21.15.178:0/3716817260 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f452c038c40 msgr2=0x7f452c03b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:48.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:48.518+0000 7f454881d700 1 --2- 172.21.15.178:0/3716817260 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f452c038c40 0x7f452c03b100 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4534006fd0 tx=0x7f4534006e40 comp rx=0 tx=0).stop 2024-03-07T07:32:48.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:48.518+0000 7f454881d700 1 -- 172.21.15.178:0/3716817260 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4544106a30 msgr2=0x7f45441965e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:48.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:48.518+0000 7f454881d700 1 --2- 172.21.15.178:0/3716817260 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4544106a30 0x7f45441965e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4538005850 tx=0x7f4538005030 comp rx=0 tx=0).stop 2024-03-07T07:32:48.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:48.518+0000 7f454881d700 1 -- 172.21.15.178:0/3716817260 shutdown_connections 2024-03-07T07:32:48.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:48.518+0000 7f454881d700 1 --2- 172.21.15.178:0/3716817260 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f452c038c40 0x7f452c03b100 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:48.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:48.518+0000 7f454881d700 1 --2- 172.21.15.178:0/3716817260 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4544106a30 0x7f45441965e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:48.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:48.518+0000 7f454881d700 1 -- 172.21.15.178:0/3716817260 >> 172.21.15.178:0/3716817260 conn(0x7f45440fc560 msgr2=0x7f45441085e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:32:48.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:48.518+0000 7f454881d700 1 -- 172.21.15.178:0/3716817260 shutdown_connections 2024-03-07T07:32:48.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:48.518+0000 7f454881d700 1 -- 172.21.15.178:0/3716817260 wait complete. 2024-03-07T07:32:49.095 DEBUG:teuthology.orchestra.run.smithi177:mon.b> sudo journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@mon.b.service 2024-03-07T07:32:49.098 DEBUG:teuthology.orchestra.run.smithi178:mon.c> sudo journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@mon.c.service 2024-03-07T07:32:49.100 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-07T07:32:49.100 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph mon dump -f json 2024-03-07T07:32:49.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:49 smithi160 ceph-mon[31963]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:32:49.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:49 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:49.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:49 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:32:49.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:49 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:32:49.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:49 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:32:49.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:49 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:49.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:49 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-07T07:32:49.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:49 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:32:49.757 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-07T07:32:50.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:50 smithi160 ceph-mon[31963]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi160:172.21.15.160=a;smithi177:172.21.15.177=b;smithi178:172.21.15.178=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:32:50.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:50 smithi160 ceph-mon[31963]: Saving service mon spec with placement smithi160:172.21.15.160=a;smithi177:172.21.15.177=b;smithi178:172.21.15.178=c;count:3 2024-03-07T07:32:50.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:50 smithi160 ceph-mon[31963]: Deploying daemon mon.c on smithi178 2024-03-07T07:32:51.427 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:51 smithi160 ceph-mon[31963]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:32:52.816 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:52.811+0000 7f2db9327700 1 -- 172.21.15.178:0/1732234820 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2db4106a00 msgr2=0x7f2db4106de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:52.816 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:52.811+0000 7f2db9327700 1 --2- 172.21.15.178:0/1732234820 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2db4106a00 0x7f2db4106de0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2da4009910 tx=0x7f2da4009c20 comp rx=0 tx=0).stop 2024-03-07T07:32:52.816 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:52.811+0000 7f2db9327700 1 -- 172.21.15.178:0/1732234820 shutdown_connections 2024-03-07T07:32:52.816 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:52.811+0000 7f2db9327700 1 --2- 172.21.15.178:0/1732234820 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2db4106a00 0x7f2db4106de0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:52.816 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:52.811+0000 7f2db9327700 1 -- 172.21.15.178:0/1732234820 >> 172.21.15.178:0/1732234820 conn(0x7f2db40777f0 msgr2=0x7f2db40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:32:52.816 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:52.811+0000 7f2db9327700 1 -- 172.21.15.178:0/1732234820 shutdown_connections 2024-03-07T07:32:52.816 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:52.811+0000 7f2db9327700 1 -- 172.21.15.178:0/1732234820 wait complete. 2024-03-07T07:32:52.816 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:52.812+0000 7f2db9327700 1 Processor -- start 2024-03-07T07:32:52.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:52.812+0000 7f2db9327700 1 -- start start 2024-03-07T07:32:52.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:52.812+0000 7f2db9327700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2db4106a00 0x7f2db4198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:32:52.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:52.812+0000 7f2db9327700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db410b040 con 0x7f2db4106a00 2024-03-07T07:32:52.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:52.812+0000 7f2db2ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2db4106a00 0x7f2db4198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:32:52.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:52.812+0000 7f2db2ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2db4106a00 0x7f2db4198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.178:41820/0 (socket says 172.21.15.178:41820) 2024-03-07T07:32:52.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:52.813+0000 7f2db2ffd700 1 -- 172.21.15.178:0/35947633 learned_addr learned my addr 172.21.15.178:0/35947633 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-03-07T07:32:52.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:52.813+0000 7f2db2ffd700 1 -- 172.21.15.178:0/35947633 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db4198950 con 0x7f2db4106a00 2024-03-07T07:32:52.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:52.813+0000 7f2db2ffd700 1 --2- 172.21.15.178:0/35947633 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2db4106a00 0x7f2db4198410 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2da4005850 tx=0x7f2da4005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:32:52.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:52.813+0000 7f2d9bfff700 1 -- 172.21.15.178:0/35947633 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2da4026070 con 0x7f2db4106a00 2024-03-07T07:32:52.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:52.813+0000 7f2d9bfff700 1 -- 172.21.15.178:0/35947633 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2da400bbd0 con 0x7f2db4106a00 2024-03-07T07:32:52.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:52.813+0000 7f2d9bfff700 1 -- 172.21.15.178:0/35947633 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2da4003970 con 0x7f2db4106a00 2024-03-07T07:32:52.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:52.813+0000 7f2db9327700 1 -- 172.21.15.178:0/35947633 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2db4198c40 con 0x7f2db4106a00 2024-03-07T07:32:52.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:52.813+0000 7f2db9327700 1 -- 172.21.15.178:0/35947633 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2db4199100 con 0x7f2db4106a00 2024-03-07T07:32:52.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:52.814+0000 7f2d9bfff700 1 -- 172.21.15.178:0/35947633 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7f2da4029070 con 0x7f2db4106a00 2024-03-07T07:32:52.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:52.814+0000 7f2d9bfff700 1 --2- 172.21.15.178:0/35947633 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2d9c038c40 0x7f2d9c03b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:32:52.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:52.815+0000 7f2d9bfff700 1 -- 172.21.15.178:0/35947633 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2da401e070 con 0x7f2db4106a00 2024-03-07T07:32:52.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:52.815+0000 7f2db9327700 1 -- 172.21.15.178:0/35947633 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2db4102140 con 0x7f2db4106a00 2024-03-07T07:32:52.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:52.815+0000 7f2db27fc700 1 --2- 172.21.15.178:0/35947633 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2d9c038c40 0x7f2d9c03b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:32:52.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:52.815+0000 7f2db27fc700 1 --2- 172.21.15.178:0/35947633 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2d9c038c40 0x7f2d9c03b100 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2da8006fd0 tx=0x7f2da8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:32:52.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:52.820+0000 7f2d9bfff700 1 -- 172.21.15.178:0/35947633 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f2da4030070 con 0x7f2db4106a00 2024-03-07T07:32:53.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:52 smithi178 podman[36617]: 2024-03-07 07:32:52.951775783 +0000 UTC m=+0.116035311 container create 3f6302e362f68b96182490ae1b07cd441c759ef8692f689b4acb6c18e7bb4ca0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mon-c, maintainer=Guillaume Abrioux , GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-8b896a9, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222) 2024-03-07T07:32:53.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:53.112+0000 7f2db9327700 1 -- 172.21.15.178:0/35947633 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f2db4063780 con 0x7f2db4106a00 2024-03-07T07:32:53.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:53.112+0000 7f2d9bfff700 1 -- 172.21.15.178:0/35947633 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+749 (secure 0 0 0) 0x7f2da4013070 con 0x7f2db4106a00 2024-03-07T07:32:53.113 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-03-07T07:32:53.114 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":1,"fsid":"826f6582-dc54-11ee-95c6-87774f69a715","modified":"2024-03-07T07:31:08.991282Z","created":"2024-03-07T07:31:08.991282Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-07T07:32:53.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:53.114+0000 7f2db9327700 1 -- 172.21.15.178:0/35947633 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2d9c038c40 msgr2=0x7f2d9c03b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:53.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:53.114+0000 7f2db9327700 1 --2- 172.21.15.178:0/35947633 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2d9c038c40 0x7f2d9c03b100 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2da8006fd0 tx=0x7f2da8006e40 comp rx=0 tx=0).stop 2024-03-07T07:32:53.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:53.114+0000 7f2db9327700 1 -- 172.21.15.178:0/35947633 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2db4106a00 msgr2=0x7f2db4198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:53.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:53.114+0000 7f2db9327700 1 --2- 172.21.15.178:0/35947633 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2db4106a00 0x7f2db4198410 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2da4005850 tx=0x7f2da4005030 comp rx=0 tx=0).stop 2024-03-07T07:32:53.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:53.114+0000 7f2db9327700 1 -- 172.21.15.178:0/35947633 shutdown_connections 2024-03-07T07:32:53.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:53.114+0000 7f2db9327700 1 --2- 172.21.15.178:0/35947633 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2d9c038c40 0x7f2d9c03b100 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:53.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:53.114+0000 7f2db9327700 1 --2- 172.21.15.178:0/35947633 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2db4106a00 0x7f2db4198410 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:53.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:53.114+0000 7f2db9327700 1 -- 172.21.15.178:0/35947633 >> 172.21.15.178:0/35947633 conn(0x7f2db40777f0 msgr2=0x7f2db4102500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:32:53.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:53.115+0000 7f2db9327700 1 -- 172.21.15.178:0/35947633 shutdown_connections 2024-03-07T07:32:53.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:53.115+0000 7f2db9327700 1 -- 172.21.15.178:0/35947633 wait complete. 2024-03-07T07:32:53.116 INFO:teuthology.orchestra.run.smithi178.stderr:dumped monmap epoch 1 2024-03-07T07:32:53.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 podman[36617]: 2024-03-07 07:32:53.171502932 +0000 UTC m=+0.335762459 container init 3f6302e362f68b96182490ae1b07cd441c759ef8692f689b4acb6c18e7bb4ca0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-8b896a9, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True) 2024-03-07T07:32:53.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 podman[36617]: 2024-03-07 07:32:53.174690243 +0000 UTC m=+0.338949771 container start 3f6302e362f68b96182490ae1b07cd441c759ef8692f689b4acb6c18e7bb4ca0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mon-c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.build-date=20240222, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-07T07:32:53.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: set uid:gid to 167:167 (ceph:ceph) 2024-03-07T07:32:53.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: ceph version 19.0.0-1848-g8b896a9e (8b896a9e145796119f0451201dc4d53ddac97db0) squid (dev), process ceph-mon, pid 2 2024-03-07T07:32:53.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: pidfile_write: ignore empty --pid-file 2024-03-07T07:32:53.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: load: jerasure load: lrc 2024-03-07T07:32:53.558 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: RocksDB version: 7.9.2 2024-03-07T07:32:53.558 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Git sha 0 2024-03-07T07:32:53.559 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Compile date 2024-03-06 21:30:51 2024-03-07T07:32:53.559 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: DB SUMMARY 2024-03-07T07:32:53.559 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: DB Session ID: WL3HYORZCJKB54BW8N1Q 2024-03-07T07:32:53.559 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: CURRENT file: CURRENT 2024-03-07T07:32:53.559 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: IDENTITY file: IDENTITY 2024-03-07T07:32:53.559 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-07T07:32:53.559 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-03-07T07:32:53.560 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-03-07T07:32:53.560 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.error_if_exists: 0 2024-03-07T07:32:53.560 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.create_if_missing: 0 2024-03-07T07:32:53.560 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.paranoid_checks: 1 2024-03-07T07:32:53.560 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-07T07:32:53.560 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-07T07:32:53.560 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-07T07:32:53.560 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.env: 0x5564434ddb20 2024-03-07T07:32:53.560 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.fs: PosixFileSystem 2024-03-07T07:32:53.560 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.info_log: 0x5564442a7920 2024-03-07T07:32:53.560 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.max_file_opening_threads: 16 2024-03-07T07:32:53.560 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.statistics: (nil) 2024-03-07T07:32:53.560 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.use_fsync: 0 2024-03-07T07:32:53.561 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.max_log_file_size: 0 2024-03-07T07:32:53.561 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-07T07:32:53.561 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-07T07:32:53.561 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.keep_log_file_num: 1000 2024-03-07T07:32:53.561 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.recycle_log_file_num: 0 2024-03-07T07:32:53.561 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.allow_fallocate: 1 2024-03-07T07:32:53.561 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.allow_mmap_reads: 0 2024-03-07T07:32:53.561 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.allow_mmap_writes: 0 2024-03-07T07:32:53.561 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.use_direct_reads: 0 2024-03-07T07:32:53.561 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-07T07:32:53.561 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.create_missing_column_families: 0 2024-03-07T07:32:53.561 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.db_log_dir: 2024-03-07T07:32:53.561 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.wal_dir: 2024-03-07T07:32:53.561 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-07T07:32:53.562 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-07T07:32:53.562 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-07T07:32:53.562 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-07T07:32:53.562 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-07T07:32:53.562 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-07T07:32:53.562 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.advise_random_on_open: 1 2024-03-07T07:32:53.562 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.db_write_buffer_size: 0 2024-03-07T07:32:53.562 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.write_buffer_manager: 0x55644427b400 2024-03-07T07:32:53.562 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-07T07:32:53.562 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-07T07:32:53.562 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-07T07:32:53.562 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.rate_limiter: (nil) 2024-03-07T07:32:53.562 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-07T07:32:53.563 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.wal_recovery_mode: 2 2024-03-07T07:32:53.563 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.enable_thread_tracking: 0 2024-03-07T07:32:53.563 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.enable_pipelined_write: 0 2024-03-07T07:32:53.563 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.unordered_write: 0 2024-03-07T07:32:53.563 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-07T07:32:53.563 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-07T07:32:53.563 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-07T07:32:53.563 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-07T07:32:53.563 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.row_cache: None 2024-03-07T07:32:53.563 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.wal_filter: None 2024-03-07T07:32:53.565 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-07T07:32:53.565 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.allow_ingest_behind: 0 2024-03-07T07:32:53.565 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.two_write_queues: 0 2024-03-07T07:32:53.565 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.manual_wal_flush: 0 2024-03-07T07:32:53.565 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.wal_compression: 0 2024-03-07T07:32:53.565 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.atomic_flush: 0 2024-03-07T07:32:53.565 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-07T07:32:53.566 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-07T07:32:53.566 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-07T07:32:53.566 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.log_readahead_size: 0 2024-03-07T07:32:53.566 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-07T07:32:53.566 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.best_efforts_recovery: 0 2024-03-07T07:32:53.566 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-07T07:32:53.566 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-07T07:32:53.566 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.allow_data_in_errors: 0 2024-03-07T07:32:53.566 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.db_host_id: __hostname__ 2024-03-07T07:32:53.566 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-07T07:32:53.566 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.max_background_jobs: 2 2024-03-07T07:32:53.566 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.max_background_compactions: -1 2024-03-07T07:32:53.566 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.max_subcompactions: 1 2024-03-07T07:32:53.567 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-07T07:32:53.567 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-07T07:32:53.567 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-07T07:32:53.567 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.max_total_wal_size: 0 2024-03-07T07:32:53.567 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-07T07:32:53.567 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-07T07:32:53.567 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-07T07:32:53.567 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-07T07:32:53.567 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.max_open_files: -1 2024-03-07T07:32:53.567 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.bytes_per_sync: 0 2024-03-07T07:32:53.567 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-07T07:32:53.567 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-07T07:32:53.567 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.compaction_readahead_size: 0 2024-03-07T07:32:53.567 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.max_background_flushes: -1 2024-03-07T07:32:53.568 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Compression algorithms supported: 2024-03-07T07:32:53.568 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-07T07:32:53.568 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: kZSTD supported: 0 2024-03-07T07:32:53.568 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: kXpressCompression supported: 0 2024-03-07T07:32:53.568 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: kLZ4HCCompression supported: 1 2024-03-07T07:32:53.568 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: kZlibCompression supported: 1 2024-03-07T07:32:53.568 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: kSnappyCompression supported: 1 2024-03-07T07:32:53.568 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: kLZ4Compression supported: 1 2024-03-07T07:32:53.568 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: kBZip2Compression supported: 0 2024-03-07T07:32:53.568 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-07T07:32:53.568 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-07T07:32:53.568 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-03-07T07:32:53.568 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-07T07:32:53.568 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-07T07:32:53.569 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.merge_operator: 2024-03-07T07:32:53.569 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.compaction_filter: None 2024-03-07T07:32:53.569 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.compaction_filter_factory: None 2024-03-07T07:32:53.569 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.sst_partitioner_factory: None 2024-03-07T07:32:53.569 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-07T07:32:53.569 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-07T07:32:53.569 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5564442a7a20) 2024-03-07T07:32:53.569 INFO:journalctl@ceph.mon.c.smithi178.stdout: cache_index_and_filter_blocks: 1 2024-03-07T07:32:53.569 INFO:journalctl@ceph.mon.c.smithi178.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-07T07:32:53.569 INFO:journalctl@ceph.mon.c.smithi178.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-07T07:32:53.569 INFO:journalctl@ceph.mon.c.smithi178.stdout: pin_top_level_index_and_filter: 1 2024-03-07T07:32:53.569 INFO:journalctl@ceph.mon.c.smithi178.stdout: index_type: 0 2024-03-07T07:32:53.569 INFO:journalctl@ceph.mon.c.smithi178.stdout: data_block_index_type: 0 2024-03-07T07:32:53.569 INFO:journalctl@ceph.mon.c.smithi178.stdout: index_shortening: 1 2024-03-07T07:32:53.570 INFO:journalctl@ceph.mon.c.smithi178.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-07T07:32:53.570 INFO:journalctl@ceph.mon.c.smithi178.stdout: checksum: 4 2024-03-07T07:32:53.570 INFO:journalctl@ceph.mon.c.smithi178.stdout: no_block_cache: 0 2024-03-07T07:32:53.570 INFO:journalctl@ceph.mon.c.smithi178.stdout: block_cache: 0x5564442ad090 2024-03-07T07:32:53.570 INFO:journalctl@ceph.mon.c.smithi178.stdout: block_cache_name: BinnedLRUCache 2024-03-07T07:32:53.570 INFO:journalctl@ceph.mon.c.smithi178.stdout: block_cache_options: 2024-03-07T07:32:53.570 INFO:journalctl@ceph.mon.c.smithi178.stdout: capacity : 536870912 2024-03-07T07:32:53.570 INFO:journalctl@ceph.mon.c.smithi178.stdout: num_shard_bits : 4 2024-03-07T07:32:53.570 INFO:journalctl@ceph.mon.c.smithi178.stdout: strict_capacity_limit : 0 2024-03-07T07:32:53.570 INFO:journalctl@ceph.mon.c.smithi178.stdout: high_pri_pool_ratio: 0.000 2024-03-07T07:32:53.570 INFO:journalctl@ceph.mon.c.smithi178.stdout: block_cache_compressed: (nil) 2024-03-07T07:32:53.570 INFO:journalctl@ceph.mon.c.smithi178.stdout: persistent_cache: (nil) 2024-03-07T07:32:53.570 INFO:journalctl@ceph.mon.c.smithi178.stdout: block_size: 4096 2024-03-07T07:32:53.570 INFO:journalctl@ceph.mon.c.smithi178.stdout: block_size_deviation: 10 2024-03-07T07:32:53.571 INFO:journalctl@ceph.mon.c.smithi178.stdout: block_restart_interval: 16 2024-03-07T07:32:53.571 INFO:journalctl@ceph.mon.c.smithi178.stdout: index_block_restart_interval: 1 2024-03-07T07:32:53.571 INFO:journalctl@ceph.mon.c.smithi178.stdout: metadata_block_size: 4096 2024-03-07T07:32:53.571 INFO:journalctl@ceph.mon.c.smithi178.stdout: partition_filters: 0 2024-03-07T07:32:53.571 INFO:journalctl@ceph.mon.c.smithi178.stdout: use_delta_encoding: 1 2024-03-07T07:32:53.571 INFO:journalctl@ceph.mon.c.smithi178.stdout: filter_policy: bloomfilter 2024-03-07T07:32:53.571 INFO:journalctl@ceph.mon.c.smithi178.stdout: whole_key_filtering: 1 2024-03-07T07:32:53.571 INFO:journalctl@ceph.mon.c.smithi178.stdout: verify_compression: 0 2024-03-07T07:32:53.571 INFO:journalctl@ceph.mon.c.smithi178.stdout: read_amp_bytes_per_bit: 0 2024-03-07T07:32:53.571 INFO:journalctl@ceph.mon.c.smithi178.stdout: format_version: 5 2024-03-07T07:32:53.571 INFO:journalctl@ceph.mon.c.smithi178.stdout: enable_index_compression: 1 2024-03-07T07:32:53.571 INFO:journalctl@ceph.mon.c.smithi178.stdout: block_align: 0 2024-03-07T07:32:53.571 INFO:journalctl@ceph.mon.c.smithi178.stdout: max_auto_readahead_size: 262144 2024-03-07T07:32:53.571 INFO:journalctl@ceph.mon.c.smithi178.stdout: prepopulate_block_cache: 0 2024-03-07T07:32:53.571 INFO:journalctl@ceph.mon.c.smithi178.stdout: initial_auto_readahead_size: 8192 2024-03-07T07:32:53.572 INFO:journalctl@ceph.mon.c.smithi178.stdout: num_file_reads_for_auto_readahead: 2 2024-03-07T07:32:53.572 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.write_buffer_size: 33554432 2024-03-07T07:32:53.572 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.max_write_buffer_number: 2 2024-03-07T07:32:53.572 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.compression: NoCompression 2024-03-07T07:32:53.572 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.bottommost_compression: Disabled 2024-03-07T07:32:53.572 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.prefix_extractor: nullptr 2024-03-07T07:32:53.572 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-07T07:32:53.572 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.num_levels: 7 2024-03-07T07:32:53.572 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-07T07:32:53.572 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-07T07:32:53.572 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-07T07:32:53.572 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-07T07:32:53.573 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-07T07:32:53.573 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-07T07:32:53.573 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-07T07:32:53.573 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-07T07:32:53.573 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-07T07:32:53.573 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-07T07:32:53.573 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-07T07:32:53.573 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-07T07:32:53.573 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-07T07:32:53.573 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.compression_opts.level: 32767 2024-03-07T07:32:53.573 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.compression_opts.strategy: 0 2024-03-07T07:32:53.573 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-07T07:32:53.573 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-07T07:32:53.573 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-07T07:32:53.574 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-07T07:32:53.574 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.compression_opts.enabled: false 2024-03-07T07:32:53.574 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-07T07:32:53.574 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-07T07:32:53.574 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-07T07:32:53.574 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-07T07:32:53.574 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.target_file_size_base: 67108864 2024-03-07T07:32:53.574 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-07T07:32:53.574 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-07T07:32:53.574 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-07T07:32:53.574 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-07T07:32:53.574 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-07T07:32:53.574 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-07T07:32:53.574 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-07T07:32:53.575 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-07T07:32:53.575 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-07T07:32:53.575 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-07T07:32:53.575 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-07T07:32:53.575 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-07T07:32:53.575 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-07T07:32:53.575 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-07T07:32:53.575 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.arena_block_size: 1048576 2024-03-07T07:32:53.575 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-07T07:32:53.575 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-07T07:32:53.575 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.disable_auto_compactions: 0 2024-03-07T07:32:53.575 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-07T07:32:53.575 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-07T07:32:53.576 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-07T07:32:53.576 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-07T07:32:53.576 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-07T07:32:53.576 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-07T07:32:53.576 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-07T07:32:53.576 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-07T07:32:53.576 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-07T07:32:53.576 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-07T07:32:53.576 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.table_properties_collectors: 2024-03-07T07:32:53.576 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.inplace_update_support: 0 2024-03-07T07:32:53.576 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-07T07:32:53.576 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-07T07:32:53.576 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-07T07:32:53.577 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-07T07:32:53.577 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.bloom_locality: 0 2024-03-07T07:32:53.577 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.max_successive_merges: 0 2024-03-07T07:32:53.577 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-07T07:32:53.577 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.paranoid_file_checks: 0 2024-03-07T07:32:53.577 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.force_consistency_checks: 1 2024-03-07T07:32:53.577 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.report_bg_io_stats: 0 2024-03-07T07:32:53.577 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.ttl: 2592000 2024-03-07T07:32:53.577 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-07T07:32:53.577 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-07T07:32:53.577 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-07T07:32:53.577 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.enable_blob_files: false 2024-03-07T07:32:53.577 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.min_blob_size: 0 2024-03-07T07:32:53.578 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.blob_file_size: 268435456 2024-03-07T07:32:53.578 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-07T07:32:53.578 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-07T07:32:53.578 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-07T07:32:53.578 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-07T07:32:53.578 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-07T07:32:53.578 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.blob_file_starting_level: 0 2024-03-07T07:32:53.578 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-07T07:32:53.578 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-07T07:32:53.578 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-07T07:32:53.578 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 1cce565e-2212-434a-9244-326f2219fde2 2024-03-07T07:32:53.578 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709796773204488, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-07T07:32:53.579 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-07T07:32:53.579 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709796773204901, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1709796773, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "1cce565e-2212-434a-9244-326f2219fde2", "db_session_id": "WL3HYORZCJKB54BW8N1Q", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-07T07:32:53.579 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709796773204979, "job": 1, "event": "recovery_finished"} 2024-03-07T07:32:53.579 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-07T07:32:53.579 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-07T07:32:53.579 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5564443a0000 2024-03-07T07:32:53.579 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: DB pointer 0x55644438a000 2024-03-07T07:32:53.579 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-07T07:32:53.579 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-07T07:32:53.579 INFO:journalctl@ceph.mon.c.smithi178.stdout: ** DB Stats ** 2024-03-07T07:32:53.579 INFO:journalctl@ceph.mon.c.smithi178.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-07T07:32:53.579 INFO:journalctl@ceph.mon.c.smithi178.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-07T07:32:53.580 INFO:journalctl@ceph.mon.c.smithi178.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-07T07:32:53.580 INFO:journalctl@ceph.mon.c.smithi178.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-07T07:32:53.580 INFO:journalctl@ceph.mon.c.smithi178.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-07T07:32:53.580 INFO:journalctl@ceph.mon.c.smithi178.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-07T07:32:53.580 INFO:journalctl@ceph.mon.c.smithi178.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-07T07:32:53.580 INFO:journalctl@ceph.mon.c.smithi178.stdout: 2024-03-07T07:32:53.580 INFO:journalctl@ceph.mon.c.smithi178.stdout: ** Compaction Stats [default] ** 2024-03-07T07:32:53.580 INFO:journalctl@ceph.mon.c.smithi178.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-07T07:32:53.580 INFO:journalctl@ceph.mon.c.smithi178.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-07T07:32:53.580 INFO:journalctl@ceph.mon.c.smithi178.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-07T07:32:53.580 INFO:journalctl@ceph.mon.c.smithi178.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-07T07:32:53.580 INFO:journalctl@ceph.mon.c.smithi178.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-07T07:32:53.581 INFO:journalctl@ceph.mon.c.smithi178.stdout: 2024-03-07T07:32:53.581 INFO:journalctl@ceph.mon.c.smithi178.stdout: ** Compaction Stats [default] ** 2024-03-07T07:32:53.581 INFO:journalctl@ceph.mon.c.smithi178.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-07T07:32:53.581 INFO:journalctl@ceph.mon.c.smithi178.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-07T07:32:53.581 INFO:journalctl@ceph.mon.c.smithi178.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-07T07:32:53.581 INFO:journalctl@ceph.mon.c.smithi178.stdout: 2024-03-07T07:32:53.581 INFO:journalctl@ceph.mon.c.smithi178.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-07T07:32:53.581 INFO:journalctl@ceph.mon.c.smithi178.stdout: 2024-03-07T07:32:53.581 INFO:journalctl@ceph.mon.c.smithi178.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-07T07:32:53.581 INFO:journalctl@ceph.mon.c.smithi178.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-07T07:32:53.581 INFO:journalctl@ceph.mon.c.smithi178.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-07T07:32:53.581 INFO:journalctl@ceph.mon.c.smithi178.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-07T07:32:53.581 INFO:journalctl@ceph.mon.c.smithi178.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-07T07:32:53.582 INFO:journalctl@ceph.mon.c.smithi178.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-07T07:32:53.582 INFO:journalctl@ceph.mon.c.smithi178.stdout: Cumulative compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-07T07:32:53.582 INFO:journalctl@ceph.mon.c.smithi178.stdout: Interval compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-07T07:32:53.582 INFO:journalctl@ceph.mon.c.smithi178.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-07T07:32:53.582 INFO:journalctl@ceph.mon.c.smithi178.stdout: Block cache BinnedLRUCache@0x5564442ad090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-03-07T07:32:53.582 INFO:journalctl@ceph.mon.c.smithi178.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-07T07:32:53.582 INFO:journalctl@ceph.mon.c.smithi178.stdout: 2024-03-07T07:32:53.582 INFO:journalctl@ceph.mon.c.smithi178.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-07T07:32:53.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-03-07T07:32:53.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: using public_addr v2:172.21.15.178:0/0 -> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-03-07T07:32:53.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: starting mon.c rank -1 at public addrs [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] at bind addrs [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 826f6582-dc54-11ee-95c6-87774f69a715 2024-03-07T07:32:53.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: mon.c@-1(???) e0 preinit fsid 826f6582-dc54-11ee-95c6-87774f69a715 2024-03-07T07:32:53.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: mon.c@-1(synchronizing).mds e1 new map 2024-03-07T07:32:53.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: mon.c@-1(synchronizing).mds e1 print_map 2024-03-07T07:32:53.583 INFO:journalctl@ceph.mon.c.smithi178.stdout: e1 2024-03-07T07:32:53.583 INFO:journalctl@ceph.mon.c.smithi178.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-07T07:32:53.583 INFO:journalctl@ceph.mon.c.smithi178.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-07T07:32:53.583 INFO:journalctl@ceph.mon.c.smithi178.stdout: legacy client fscid: -1 2024-03-07T07:32:53.583 INFO:journalctl@ceph.mon.c.smithi178.stdout: 2024-03-07T07:32:53.583 INFO:journalctl@ceph.mon.c.smithi178.stdout: No filesystems configured 2024-03-07T07:32:53.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-07T07:32:53.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-07T07:32:53.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-07T07:32:53.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-07T07:32:53.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-07T07:32:53.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-07T07:32:53.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-07T07:32:53.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-07T07:32:53.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-07T07:32:53.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-07T07:32:53.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: Added host smithi177 2024-03-07T07:32:53.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:53.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:53.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:32:53.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:53.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-07T07:32:53.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:32:53.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:32:53.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi178", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:32:53.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:53.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:53.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:53.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:53.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi177", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:32:53.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:32:53.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:32:53.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: Updating smithi177:/etc/ceph/ceph.conf 2024-03-07T07:32:53.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: Deploying cephadm binary to smithi178 2024-03-07T07:32:53.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: Updating smithi177:/var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-07T07:32:53.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: Updating smithi177:/etc/ceph/ceph.client.admin.keyring 2024-03-07T07:32:53.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: Updating smithi177:/var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-07T07:32:53.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:32:53.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:53.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:53.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:53.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:32:53.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:53.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:32:53.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: Added host smithi178 2024-03-07T07:32:53.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:53.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:53.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:32:53.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-07T07:32:53.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:53.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:32:53.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/890340355' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-07T07:32:53.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/890340355' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-07T07:32:53.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: osdmap e4: 0 total, 0 up, 0 in 2024-03-07T07:32:53.587 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:53.587 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:53.587 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:53.587 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:53.587 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi178", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:32:53.587 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:32:53.587 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:32:53.587 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: Updating smithi178:/etc/ceph/ceph.conf 2024-03-07T07:32:53.587 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:32:53.587 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: Updating smithi178:/var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-07T07:32:53.587 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:53.587 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:53.587 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:53.587 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: Updating smithi178:/etc/ceph/ceph.client.admin.keyring 2024-03-07T07:32:53.588 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: Updating smithi178:/var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-07T07:32:53.588 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:32:53.588 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:53.588 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:32:53.588 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:32:53.588 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:32:53.588 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:53.588 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-07T07:32:53.588 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:32:53.588 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi160:172.21.15.160=a;smithi177:172.21.15.177=b;smithi178:172.21.15.178=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:32:53.588 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: Saving service mon spec with placement smithi160:172.21.15.160=a;smithi177:172.21.15.177=b;smithi178:172.21.15.178=c;count:3 2024-03-07T07:32:53.588 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: Deploying daemon mon.c on smithi178 2024-03-07T07:32:53.588 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:32:53.588 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 ceph-mon[36650]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-07T07:32:53.589 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 bash[36617]: 3f6302e362f68b96182490ae1b07cd441c759ef8692f689b4acb6c18e7bb4ca0 2024-03-07T07:32:53.589 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:53 smithi178 systemd[1]: Started Ceph mon.c for 826f6582-dc54-11ee-95c6-87774f69a715. 2024-03-07T07:32:54.920 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-07T07:32:54.921 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph mon dump -f json 2024-03-07T07:32:55.167 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.c/config 2024-03-07T07:32:58.634 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:58 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-07T07:32:58.634 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:58 smithi160 ceph-mon[31963]: mon.a calling monitor election 2024-03-07T07:32:58.634 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:58 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-07T07:32:58.634 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:58 smithi160 ceph-mon[31963]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:32:58.634 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:58 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-07T07:32:58.634 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:58 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-07T07:32:58.635 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:58 smithi160 ceph-mon[31963]: mon.c calling monitor election 2024-03-07T07:32:58.635 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:58 smithi160 ceph-mon[31963]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:32:58.635 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:58 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-07T07:32:58.635 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:58 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-07T07:32:58.635 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:58 smithi160 ceph-mon[31963]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:32:58.635 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:58 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-07T07:32:58.635 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:58 smithi160 ceph-mon[31963]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-07T07:32:58.635 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:58 smithi160 ceph-mon[31963]: monmap e2: 2 mons at {a=[v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0],c=[v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-07T07:32:58.635 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:58 smithi160 ceph-mon[31963]: fsmap 2024-03-07T07:32:58.635 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:58 smithi160 ceph-mon[31963]: osdmap e4: 0 total, 0 up, 0 in 2024-03-07T07:32:58.635 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:58 smithi160 ceph-mon[31963]: mgrmap e13: x(active, since 50s) 2024-03-07T07:32:58.635 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:58 smithi160 ceph-mon[31963]: overall HEALTH_OK 2024-03-07T07:32:58.635 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:58 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:58.635 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:58 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:58.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:58 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-07T07:32:58.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:58 smithi178 ceph-mon[36650]: mon.a calling monitor election 2024-03-07T07:32:58.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:58 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-07T07:32:58.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:58 smithi178 ceph-mon[36650]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:32:58.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:58 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-07T07:32:58.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:58 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-07T07:32:58.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:58 smithi178 ceph-mon[36650]: mon.c calling monitor election 2024-03-07T07:32:58.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:58 smithi178 ceph-mon[36650]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:32:58.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:58 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-07T07:32:58.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:58 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-07T07:32:58.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:58 smithi178 ceph-mon[36650]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:32:58.808 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:58 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-07T07:32:58.808 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:58 smithi178 ceph-mon[36650]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-07T07:32:58.808 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:58 smithi178 ceph-mon[36650]: monmap e2: 2 mons at {a=[v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0],c=[v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-07T07:32:58.808 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:58 smithi178 ceph-mon[36650]: fsmap 2024-03-07T07:32:58.808 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:58 smithi178 ceph-mon[36650]: osdmap e4: 0 total, 0 up, 0 in 2024-03-07T07:32:58.808 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:58 smithi178 ceph-mon[36650]: mgrmap e13: x(active, since 50s) 2024-03-07T07:32:58.808 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:58 smithi178 ceph-mon[36650]: overall HEALTH_OK 2024-03-07T07:32:58.808 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:58 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:58.808 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:58 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:59.660 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:59 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:59.661 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:59 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-07T07:32:59.661 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:59 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:32:59.661 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:59 smithi178 ceph-mon[36650]: Deploying daemon mon.b on smithi177 2024-03-07T07:32:59.661 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:32:59 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-07T07:32:59.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.659+0000 7f4589595700 1 -- 172.21.15.178:0/2807870436 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4584102e20 msgr2=0x7f4570005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:59.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.659+0000 7f4589595700 1 --2- 172.21.15.178:0/2807870436 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4584102e20 0x7f4570005610 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4574009bf0 tx=0x7f4574009f00 comp rx=0 tx=0).stop 2024-03-07T07:32:59.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.659+0000 7f4589595700 1 -- 172.21.15.178:0/2807870436 shutdown_connections 2024-03-07T07:32:59.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.659+0000 7f4589595700 1 --2- 172.21.15.178:0/2807870436 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4584102e20 0x7f4570005610 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:59.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.659+0000 7f4589595700 1 -- 172.21.15.178:0/2807870436 >> 172.21.15.178:0/2807870436 conn(0x7f45840fc670 msgr2=0x7f45840fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:32:59.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.659+0000 7f4589595700 1 -- 172.21.15.178:0/2807870436 shutdown_connections 2024-03-07T07:32:59.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.659+0000 7f4589595700 1 -- 172.21.15.178:0/2807870436 wait complete. 2024-03-07T07:32:59.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.660+0000 7f4589595700 1 Processor -- start 2024-03-07T07:32:59.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.660+0000 7f4589595700 1 -- start start 2024-03-07T07:32:59.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.660+0000 7f4589595700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4584102e20 0x7f4584197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:32:59.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.661+0000 7f4589595700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4584197c70 0x7f458419c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:32:59.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.661+0000 7f4589595700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4584109d00 con 0x7f4584102e20 2024-03-07T07:32:59.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.661+0000 7f4589595700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4584109b80 con 0x7f4584197c70 2024-03-07T07:32:59.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.661+0000 7f4582ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4584102e20 0x7f4584197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:32:59.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.661+0000 7f45827fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4584197c70 0x7f458419c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:32:59.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.661+0000 7f4582ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4584102e20 0x7f4584197730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.178:38178/0 (socket says 172.21.15.178:38178) 2024-03-07T07:32:59.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.661+0000 7f4582ffd700 1 -- 172.21.15.178:0/1974901566 learned_addr learned my addr 172.21.15.178:0/1974901566 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-03-07T07:32:59.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.661+0000 7f45827fc700 1 -- 172.21.15.178:0/1974901566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4584197c70 msgr2=0x7f458419c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-07T07:32:59.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.661+0000 7f45827fc700 1 -- 172.21.15.178:0/1974901566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4584197c70 msgr2=0x7f458419c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-07T07:32:59.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.661+0000 7f45827fc700 1 --2- 172.21.15.178:0/1974901566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4584197c70 0x7f458419c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-07T07:32:59.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.661+0000 7f45827fc700 1 --2- 172.21.15.178:0/1974901566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4584197c70 0x7f458419c000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-07T07:32:59.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.661+0000 7f4582ffd700 1 -- 172.21.15.178:0/1974901566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4584197c70 msgr2=0x7f458419c000 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-07T07:32:59.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.661+0000 7f4582ffd700 1 --2- 172.21.15.178:0/1974901566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4584197c70 0x7f458419c000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:59.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.661+0000 7f4582ffd700 1 -- 172.21.15.178:0/1974901566 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45841982b0 con 0x7f4584102e20 2024-03-07T07:32:59.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.661+0000 7f4582ffd700 1 --2- 172.21.15.178:0/1974901566 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4584102e20 0x7f4584197730 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4574009610 tx=0x7f4574007ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:32:59.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.661+0000 7f457bfff700 1 -- 172.21.15.178:0/1974901566 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f4574018bf0 con 0x7f4584102e20 2024-03-07T07:32:59.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.661+0000 7f457bfff700 1 -- 172.21.15.178:0/1974901566 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4574018d90 con 0x7f4584102e20 2024-03-07T07:32:59.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.661+0000 7f4589595700 1 -- 172.21.15.178:0/1974901566 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f458419c540 con 0x7f4584102e20 2024-03-07T07:32:59.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.662+0000 7f457bfff700 1 -- 172.21.15.178:0/1974901566 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f4574005220 con 0x7f4584102e20 2024-03-07T07:32:59.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.662+0000 7f4589595700 1 -- 172.21.15.178:0/1974901566 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f458419caa0 con 0x7f4584102e20 2024-03-07T07:32:59.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.662+0000 7f457bfff700 1 -- 172.21.15.178:0/1974901566 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7f457401e070 con 0x7f4584102e20 2024-03-07T07:32:59.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.663+0000 7f457bfff700 1 --2- 172.21.15.178:0/1974901566 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f4564038a30 0x7f456403aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:32:59.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.663+0000 7f457bfff700 1 -- 172.21.15.178:0/1974901566 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f457401c070 con 0x7f4584102e20 2024-03-07T07:32:59.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.663+0000 7f45827fc700 1 --2- 172.21.15.178:0/1974901566 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f4564038a30 0x7f456403aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:32:59.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.663+0000 7f4589595700 1 -- 172.21.15.178:0/1974901566 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f458406ac10 con 0x7f4584102e20 2024-03-07T07:32:59.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.663+0000 7f45827fc700 1 --2- 172.21.15.178:0/1974901566 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f4564038a30 0x7f456403aef0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f456c009850 tx=0x7f456c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:32:59.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.668+0000 7f457bfff700 1 -- 172.21.15.178:0/1974901566 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4574016630 con 0x7f4584102e20 2024-03-07T07:32:59.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:59 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:32:59.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:59 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-07T07:32:59.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:59 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:32:59.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:59 smithi160 ceph-mon[31963]: Deploying daemon mon.b on smithi177 2024-03-07T07:32:59.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:32:59 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-07T07:32:59.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.969+0000 7f4589595700 1 -- 172.21.15.178:0/1974901566 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f458419cd50 con 0x7f4584102e20 2024-03-07T07:32:59.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.969+0000 7f457bfff700 1 -- 172.21.15.178:0/1974901566 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1019 (secure 0 0 0) 0x7f4574036020 con 0x7f4584102e20 2024-03-07T07:32:59.971 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-03-07T07:32:59.971 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":2,"fsid":"826f6582-dc54-11ee-95c6-87774f69a715","modified":"2024-03-07T07:32:53.226486Z","created":"2024-03-07T07:31:08.991282Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-07T07:32:59.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.972+0000 7f4589595700 1 -- 172.21.15.178:0/1974901566 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f4564038a30 msgr2=0x7f456403aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:59.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.972+0000 7f4589595700 1 --2- 172.21.15.178:0/1974901566 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f4564038a30 0x7f456403aef0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f456c009850 tx=0x7f456c008040 comp rx=0 tx=0).stop 2024-03-07T07:32:59.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.972+0000 7f4589595700 1 -- 172.21.15.178:0/1974901566 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4584102e20 msgr2=0x7f4584197730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:32:59.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.972+0000 7f4589595700 1 --2- 172.21.15.178:0/1974901566 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4584102e20 0x7f4584197730 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4574009610 tx=0x7f4574007ed0 comp rx=0 tx=0).stop 2024-03-07T07:32:59.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.972+0000 7f4589595700 1 -- 172.21.15.178:0/1974901566 shutdown_connections 2024-03-07T07:32:59.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.972+0000 7f4589595700 1 --2- 172.21.15.178:0/1974901566 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4584102e20 0x7f4584197730 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:59.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.972+0000 7f4589595700 1 --2- 172.21.15.178:0/1974901566 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f4564038a30 0x7f456403aef0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:59.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.972+0000 7f4589595700 1 --2- 172.21.15.178:0/1974901566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4584197c70 0x7f458419c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:32:59.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.972+0000 7f4589595700 1 -- 172.21.15.178:0/1974901566 >> 172.21.15.178:0/1974901566 conn(0x7f45840fc670 msgr2=0x7f4584103bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:32:59.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.972+0000 7f4589595700 1 -- 172.21.15.178:0/1974901566 shutdown_connections 2024-03-07T07:32:59.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:32:59.972+0000 7f4589595700 1 -- 172.21.15.178:0/1974901566 wait complete. 2024-03-07T07:32:59.973 INFO:teuthology.orchestra.run.smithi178.stderr:dumped monmap epoch 2 2024-03-07T07:33:00.546 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:00 smithi178 ceph-mon[36650]: from='client.? 172.21.15.178:0/1974901566' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-07T07:33:00.547 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:00 smithi178 ceph-mon[36650]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:00.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:00 smithi160 ceph-mon[31963]: from='client.? 172.21.15.178:0/1974901566' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-07T07:33:00.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:00 smithi160 ceph-mon[31963]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:01.573 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-07T07:33:01.573 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph mon dump -f json 2024-03-07T07:33:01.818 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.c/config 2024-03-07T07:33:02.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:02 smithi177 systemd[1]: Started Ceph mon.b for 826f6582-dc54-11ee-95c6-87774f69a715. 2024-03-07T07:33:07.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:07 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-07T07:33:07.429 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:07 smithi160 ceph-mon[31963]: mon.c calling monitor election 2024-03-07T07:33:07.429 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:07 smithi160 ceph-mon[31963]: mon.a calling monitor election 2024-03-07T07:33:07.429 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:07 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-07T07:33:07.429 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:07 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-07T07:33:07.429 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:07 smithi160 ceph-mon[31963]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:07.429 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:07 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-07T07:33:07.429 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:07 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-07T07:33:07.429 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:07 smithi160 ceph-mon[31963]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:07.429 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:07 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-07T07:33:07.429 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:07 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-07T07:33:07.430 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:07 smithi160 ceph-mon[31963]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:07.430 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:07 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-07T07:33:07.430 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:07 smithi160 ceph-mon[31963]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-07T07:33:07.430 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:07 smithi160 ceph-mon[31963]: monmap e3: 3 mons at {a=[v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0],b=[v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0],c=[v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-07T07:33:07.430 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:07 smithi160 ceph-mon[31963]: fsmap 2024-03-07T07:33:07.430 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:07 smithi160 ceph-mon[31963]: osdmap e4: 0 total, 0 up, 0 in 2024-03-07T07:33:07.430 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:07 smithi160 ceph-mon[31963]: mgrmap e13: x(active, since 58s) 2024-03-07T07:33:07.430 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:07 smithi160 ceph-mon[31963]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-07T07:33:07.430 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:07 smithi160 ceph-mon[31963]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-07T07:33:07.430 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:07 smithi160 ceph-mon[31963]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-07T07:33:07.430 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:07 smithi160 ceph-mon[31963]: mon.b (rank 2) addr [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] is down (out of quorum) 2024-03-07T07:33:07.430 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:07 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:07.430 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:07 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:07.431 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:07 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:07.431 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:07 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:07.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:07 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-07T07:33:07.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:07 smithi178 ceph-mon[36650]: mon.c calling monitor election 2024-03-07T07:33:07.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:07 smithi178 ceph-mon[36650]: mon.a calling monitor election 2024-03-07T07:33:07.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:07 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-07T07:33:07.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:07 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-07T07:33:07.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:07 smithi178 ceph-mon[36650]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:07.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:07 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-07T07:33:07.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:07 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-07T07:33:07.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:07 smithi178 ceph-mon[36650]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:07.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:07 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-07T07:33:07.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:07 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-07T07:33:07.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:07 smithi178 ceph-mon[36650]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:07.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:07 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-07T07:33:07.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:07 smithi178 ceph-mon[36650]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-07T07:33:07.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:07 smithi178 ceph-mon[36650]: monmap e3: 3 mons at {a=[v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0],b=[v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0],c=[v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-07T07:33:07.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:07 smithi178 ceph-mon[36650]: fsmap 2024-03-07T07:33:07.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:07 smithi178 ceph-mon[36650]: osdmap e4: 0 total, 0 up, 0 in 2024-03-07T07:33:07.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:07 smithi178 ceph-mon[36650]: mgrmap e13: x(active, since 58s) 2024-03-07T07:33:07.558 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:07 smithi178 ceph-mon[36650]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-07T07:33:07.558 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:07 smithi178 ceph-mon[36650]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-07T07:33:07.558 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:07 smithi178 ceph-mon[36650]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-07T07:33:07.558 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:07 smithi178 ceph-mon[36650]: mon.b (rank 2) addr [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] is down (out of quorum) 2024-03-07T07:33:07.558 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:07 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:07.558 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:07 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:07.558 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:07 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:07.558 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:07 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:07.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.675+0000 7fa656e18700 1 -- 172.21.15.178:0/4070422487 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6400035c0 msgr2=0x7fa640005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:33:07.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.675+0000 7fa656e18700 1 --2- 172.21.15.178:0/4070422487 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6400035c0 0x7fa640005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa650100a50 tx=0x7fa64400fe60 comp rx=0 tx=0).stop 2024-03-07T07:33:07.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.675+0000 7fa656e18700 1 -- 172.21.15.178:0/4070422487 shutdown_connections 2024-03-07T07:33:07.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.675+0000 7fa656e18700 1 --2- 172.21.15.178:0/4070422487 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6400035c0 0x7fa640005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:33:07.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.675+0000 7fa656e18700 1 -- 172.21.15.178:0/4070422487 >> 172.21.15.178:0/4070422487 conn(0x7fa6500fc5f0 msgr2=0x7fa6500fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:33:07.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.675+0000 7fa656e18700 1 -- 172.21.15.178:0/4070422487 shutdown_connections 2024-03-07T07:33:07.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.675+0000 7fa656e18700 1 -- 172.21.15.178:0/4070422487 wait complete. 2024-03-07T07:33:07.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.675+0000 7fa656e18700 1 Processor -- start 2024-03-07T07:33:07.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.676+0000 7fa656e18700 1 -- start start 2024-03-07T07:33:07.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.676+0000 7fa656e18700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6400035c0 0x7fa6501027b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:33:07.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.676+0000 7fa656e18700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa650102cf0 0x7fa650103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:33:07.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.676+0000 7fa656e18700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6501a3b00 0x7fa6501a5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:33:07.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.676+0000 7fa656e18700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa650108b70 con 0x7fa650102cf0 2024-03-07T07:33:07.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.676+0000 7fa656e18700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6501089f0 con 0x7fa6400035c0 2024-03-07T07:33:07.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.676+0000 7fa656e18700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa650108cf0 con 0x7fa6501a3b00 2024-03-07T07:33:07.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.676+0000 7fa6553b5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6501a3b00 0x7fa6501a5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:33:07.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.676+0000 7fa6553b5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6501a3b00 0x7fa6501a5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:40202/0 (socket says 172.21.15.178:40202) 2024-03-07T07:33:07.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.676+0000 7fa6553b5700 1 -- 172.21.15.178:0/3205607863 learned_addr learned my addr 172.21.15.178:0/3205607863 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-03-07T07:33:07.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.676+0000 7fa6553b5700 1 -- 172.21.15.178:0/3205607863 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6501a3b00 msgr2=0x7fa6501a5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-03-07T07:33:07.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.676+0000 7fa654bb4700 1 --2- 172.21.15.178:0/3205607863 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6400035c0 0x7fa6501027b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:33:07.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.676+0000 7fa6553b5700 1 -- 172.21.15.178:0/3205607863 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6501a3b00 msgr2=0x7fa6501a5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-07T07:33:07.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.676+0000 7fa64ffff700 1 --2- 172.21.15.178:0/3205607863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa650102cf0 0x7fa650103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:33:07.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.676+0000 7fa6553b5700 1 --2- 172.21.15.178:0/3205607863 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6501a3b00 0x7fa6501a5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-07T07:33:07.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.676+0000 7fa6553b5700 1 --2- 172.21.15.178:0/3205607863 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6501a3b00 0x7fa6501a5ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-07T07:33:07.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.676+0000 7fa654bb4700 1 -- 172.21.15.178:0/3205607863 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6400035c0 msgr2=0x7fa6501027b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-07T07:33:07.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.676+0000 7fa654bb4700 1 -- 172.21.15.178:0/3205607863 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6400035c0 msgr2=0x7fa6501027b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-07T07:33:07.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.676+0000 7fa654bb4700 1 --2- 172.21.15.178:0/3205607863 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6400035c0 0x7fa6501027b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-07T07:33:07.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.676+0000 7fa654bb4700 1 --2- 172.21.15.178:0/3205607863 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6400035c0 0x7fa6501027b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-07T07:33:07.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.676+0000 7fa64ffff700 1 -- 172.21.15.178:0/3205607863 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6501a3b00 msgr2=0x7fa6501a5ef0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-07T07:33:07.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.676+0000 7fa64ffff700 1 --2- 172.21.15.178:0/3205607863 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6501a3b00 0x7fa6501a5ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:33:07.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.676+0000 7fa64ffff700 1 -- 172.21.15.178:0/3205607863 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6400035c0 msgr2=0x7fa6501027b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-07T07:33:07.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.676+0000 7fa64ffff700 1 --2- 172.21.15.178:0/3205607863 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6400035c0 0x7fa6501027b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:33:07.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.676+0000 7fa64ffff700 1 -- 172.21.15.178:0/3205607863 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6501a6620 con 0x7fa650102cf0 2024-03-07T07:33:07.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.677+0000 7fa64ffff700 1 --2- 172.21.15.178:0/3205607863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa650102cf0 0x7fa650103170 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa63c00b550 tx=0x7fa63c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:33:07.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.677+0000 7fa64dffb700 1 -- 172.21.15.178:0/3205607863 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63c004230 con 0x7fa650102cf0 2024-03-07T07:33:07.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.677+0000 7fa64dffb700 1 -- 172.21.15.178:0/3205607863 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa63c0043d0 con 0x7fa650102cf0 2024-03-07T07:33:07.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.677+0000 7fa656e18700 1 -- 172.21.15.178:0/3205607863 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6501a6910 con 0x7fa650102cf0 2024-03-07T07:33:07.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.677+0000 7fa64dffb700 1 -- 172.21.15.178:0/3205607863 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63c011c20 con 0x7fa650102cf0 2024-03-07T07:33:07.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.677+0000 7fa656e18700 1 -- 172.21.15.178:0/3205607863 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6501a6e50 con 0x7fa650102cf0 2024-03-07T07:33:07.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.678+0000 7fa64dffb700 1 -- 172.21.15.178:0/3205607863 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7fa63c01a400 con 0x7fa650102cf0 2024-03-07T07:33:07.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.678+0000 7fa64dffb700 1 --2- 172.21.15.178:0/3205607863 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa6340390d0 0x7fa63403b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:33:07.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.678+0000 7fa656e18700 1 -- 172.21.15.178:0/3205607863 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa650063780 con 0x7fa650102cf0 2024-03-07T07:33:07.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.678+0000 7fa64dffb700 1 -- 172.21.15.178:0/3205607863 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa63c04b9f0 con 0x7fa650102cf0 2024-03-07T07:33:07.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.678+0000 7fa654bb4700 1 --2- 172.21.15.178:0/3205607863 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa6340390d0 0x7fa63403b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:33:07.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.679+0000 7fa654bb4700 1 --2- 172.21.15.178:0/3205607863 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa6340390d0 0x7fa63403b590 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa644020040 tx=0x7fa644001e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:33:07.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.683+0000 7fa64dffb700 1 -- 172.21.15.178:0/3205607863 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa63c01fe00 con 0x7fa650102cf0 2024-03-07T07:33:07.989 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.988+0000 7fa656e18700 1 -- 172.21.15.178:0/3205607863 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa6501a71e0 con 0x7fa650102cf0 2024-03-07T07:33:07.989 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.989+0000 7fa64dffb700 1 -- 172.21.15.178:0/3205607863 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1287 (secure 0 0 0) 0x7fa63c01fa80 con 0x7fa650102cf0 2024-03-07T07:33:07.990 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-03-07T07:33:07.990 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":3,"fsid":"826f6582-dc54-11ee-95c6-87774f69a715","modified":"2024-03-07T07:33:02.054802Z","created":"2024-03-07T07:31:08.991282Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:3300","nonce":0},{"type":"v1","addr":"172.21.15.177:6789","nonce":0}]},"addr":"172.21.15.177:6789/0","public_addr":"172.21.15.177:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-07T07:33:07.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.991+0000 7fa656e18700 1 -- 172.21.15.178:0/3205607863 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa6340390d0 msgr2=0x7fa63403b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:33:07.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.991+0000 7fa656e18700 1 --2- 172.21.15.178:0/3205607863 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa6340390d0 0x7fa63403b590 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa644020040 tx=0x7fa644001e70 comp rx=0 tx=0).stop 2024-03-07T07:33:07.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.991+0000 7fa656e18700 1 -- 172.21.15.178:0/3205607863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa650102cf0 msgr2=0x7fa650103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:33:07.992 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.991+0000 7fa656e18700 1 --2- 172.21.15.178:0/3205607863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa650102cf0 0x7fa650103170 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa63c00b550 tx=0x7fa63c00b910 comp rx=0 tx=0).stop 2024-03-07T07:33:07.992 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.991+0000 7fa656e18700 1 -- 172.21.15.178:0/3205607863 shutdown_connections 2024-03-07T07:33:07.992 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.991+0000 7fa656e18700 1 --2- 172.21.15.178:0/3205607863 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa6340390d0 0x7fa63403b590 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:33:07.992 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.991+0000 7fa656e18700 1 --2- 172.21.15.178:0/3205607863 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6501a3b00 0x7fa6501a5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:33:07.992 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.991+0000 7fa656e18700 1 --2- 172.21.15.178:0/3205607863 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6400035c0 0x7fa6501027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:33:07.992 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.991+0000 7fa656e18700 1 --2- 172.21.15.178:0/3205607863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa650102cf0 0x7fa650103170 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:33:07.992 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.991+0000 7fa656e18700 1 -- 172.21.15.178:0/3205607863 >> 172.21.15.178:0/3205607863 conn(0x7fa6500fc5f0 msgr2=0x7fa650105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:33:07.992 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.991+0000 7fa656e18700 1 -- 172.21.15.178:0/3205607863 shutdown_connections 2024-03-07T07:33:07.992 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:07.991+0000 7fa656e18700 1 -- 172.21.15.178:0/3205607863 wait complete. 2024-03-07T07:33:07.993 INFO:teuthology.orchestra.run.smithi178.stderr:dumped monmap epoch 3 2024-03-07T07:33:08.303 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:08 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:33:08.303 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:08 smithi178 ceph-mon[36650]: from='client.? 172.21.15.178:0/3205607863' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-07T07:33:08.304 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:08 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-07T07:33:08.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:08 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:33:08.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:08 smithi160 ceph-mon[31963]: from='client.? 172.21.15.178:0/3205607863' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-07T07:33:08.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:08 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-07T07:33:09.231 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-07T07:33:09.231 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph config generate-minimal-conf 2024-03-07T07:33:09.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-07T07:33:09.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: mon.c calling monitor election 2024-03-07T07:33:09.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: mon.a calling monitor election 2024-03-07T07:33:09.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-07T07:33:09.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-07T07:33:09.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:09.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-07T07:33:09.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-07T07:33:09.358 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:09.358 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-07T07:33:09.358 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-07T07:33:09.358 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:09.358 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-07T07:33:09.358 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-07T07:33:09.358 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: monmap e3: 3 mons at {a=[v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0],b=[v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0],c=[v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-07T07:33:09.358 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: fsmap 2024-03-07T07:33:09.358 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: osdmap e4: 0 total, 0 up, 0 in 2024-03-07T07:33:09.358 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: mgrmap e13: x(active, since 58s) 2024-03-07T07:33:09.358 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-07T07:33:09.358 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-07T07:33:09.358 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-07T07:33:09.358 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: mon.b (rank 2) addr [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] is down (out of quorum) 2024-03-07T07:33:09.359 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:09.359 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:09.359 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:09.359 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:09.359 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:33:09.359 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: from='client.? 172.21.15.178:0/3205607863' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-07T07:33:09.359 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-07T07:33:09.484 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:33:09.855 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:09 smithi160 ceph-mon[31963]: mon.b calling monitor election 2024-03-07T07:33:09.855 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:09 smithi160 ceph-mon[31963]: Updating smithi160:/etc/ceph/ceph.conf 2024-03-07T07:33:09.855 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:09 smithi160 ceph-mon[31963]: Updating smithi177:/etc/ceph/ceph.conf 2024-03-07T07:33:09.855 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:09 smithi160 ceph-mon[31963]: Updating smithi178:/etc/ceph/ceph.conf 2024-03-07T07:33:09.855 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:09 smithi160 ceph-mon[31963]: Updating smithi160:/var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-07T07:33:09.855 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:09 smithi160 ceph-mon[31963]: Updating smithi177:/var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-07T07:33:09.855 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:09 smithi160 ceph-mon[31963]: Updating smithi178:/var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-07T07:33:09.855 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:09 smithi160 ceph-mon[31963]: mon.b calling monitor election 2024-03-07T07:33:09.855 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:09 smithi160 ceph-mon[31963]: mon.c calling monitor election 2024-03-07T07:33:09.856 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:09 smithi160 ceph-mon[31963]: mon.a calling monitor election 2024-03-07T07:33:09.856 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:09 smithi160 ceph-mon[31963]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-07T07:33:09.856 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:09 smithi160 ceph-mon[31963]: monmap e3: 3 mons at {a=[v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0],b=[v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0],c=[v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-07T07:33:09.856 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:09 smithi160 ceph-mon[31963]: fsmap 2024-03-07T07:33:09.856 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:09 smithi160 ceph-mon[31963]: osdmap e4: 0 total, 0 up, 0 in 2024-03-07T07:33:09.856 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:09 smithi160 ceph-mon[31963]: mgrmap e13: x(active, since 60s) 2024-03-07T07:33:09.856 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:09 smithi160 ceph-mon[31963]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-07T07:33:09.856 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:09 smithi160 ceph-mon[31963]: Cluster is now healthy 2024-03-07T07:33:09.856 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:09 smithi160 ceph-mon[31963]: overall HEALTH_OK 2024-03-07T07:33:09.856 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:09 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:09.856 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:09 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:09.856 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:09 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:09.856 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:09 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:09.856 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:09 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:09.857 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:09 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:09.857 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:09 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:09.857 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:09 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:09.857 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:09 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:09.857 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:09 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:09.857 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:09 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:09.857 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:09 smithi160 ceph-mon[31963]: Reconfiguring mon.a (unknown last config time)... 2024-03-07T07:33:09.857 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:09 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-07T07:33:09.857 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:09 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-07T07:33:09.857 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:09 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:33:09.857 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:09 smithi160 ceph-mon[31963]: Reconfiguring daemon mon.a on smithi160 2024-03-07T07:33:10.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:09 smithi178 ceph-mon[36650]: mon.b calling monitor election 2024-03-07T07:33:10.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:09 smithi178 ceph-mon[36650]: Updating smithi160:/etc/ceph/ceph.conf 2024-03-07T07:33:10.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:09 smithi178 ceph-mon[36650]: Updating smithi177:/etc/ceph/ceph.conf 2024-03-07T07:33:10.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:09 smithi178 ceph-mon[36650]: Updating smithi178:/etc/ceph/ceph.conf 2024-03-07T07:33:10.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:09 smithi178 ceph-mon[36650]: Updating smithi160:/var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-07T07:33:10.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:09 smithi178 ceph-mon[36650]: Updating smithi177:/var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-07T07:33:10.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:09 smithi178 ceph-mon[36650]: Updating smithi178:/var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-07T07:33:10.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:09 smithi178 ceph-mon[36650]: mon.b calling monitor election 2024-03-07T07:33:10.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:09 smithi178 ceph-mon[36650]: mon.c calling monitor election 2024-03-07T07:33:10.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:09 smithi178 ceph-mon[36650]: mon.a calling monitor election 2024-03-07T07:33:10.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:09 smithi178 ceph-mon[36650]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-07T07:33:10.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:09 smithi178 ceph-mon[36650]: monmap e3: 3 mons at {a=[v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0],b=[v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0],c=[v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-07T07:33:10.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:09 smithi178 ceph-mon[36650]: fsmap 2024-03-07T07:33:10.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:09 smithi178 ceph-mon[36650]: osdmap e4: 0 total, 0 up, 0 in 2024-03-07T07:33:10.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:09 smithi178 ceph-mon[36650]: mgrmap e13: x(active, since 60s) 2024-03-07T07:33:10.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:09 smithi178 ceph-mon[36650]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-07T07:33:10.058 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:09 smithi178 ceph-mon[36650]: Cluster is now healthy 2024-03-07T07:33:10.058 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:09 smithi178 ceph-mon[36650]: overall HEALTH_OK 2024-03-07T07:33:10.058 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:09 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:10.058 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:09 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:10.058 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:09 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:10.058 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:09 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:10.058 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:09 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:10.058 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:09 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:10.058 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:09 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:10.058 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:09 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:10.058 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:09 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:10.058 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:09 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:10.058 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:09 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:10.058 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:09 smithi178 ceph-mon[36650]: Reconfiguring mon.a (unknown last config time)... 2024-03-07T07:33:10.059 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:09 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-07T07:33:10.059 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:09 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-07T07:33:10.059 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:09 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:33:10.059 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:09 smithi178 ceph-mon[36650]: Reconfiguring daemon mon.a on smithi160 2024-03-07T07:33:10.108 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: mon.b calling monitor election 2024-03-07T07:33:10.108 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: Updating smithi160:/etc/ceph/ceph.conf 2024-03-07T07:33:10.108 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: Updating smithi177:/etc/ceph/ceph.conf 2024-03-07T07:33:10.108 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: Updating smithi178:/etc/ceph/ceph.conf 2024-03-07T07:33:10.108 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: Updating smithi160:/var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-07T07:33:10.108 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: Updating smithi177:/var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-07T07:33:10.108 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: Updating smithi178:/var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-07T07:33:10.109 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: mon.b calling monitor election 2024-03-07T07:33:10.109 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: mon.c calling monitor election 2024-03-07T07:33:10.109 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: mon.a calling monitor election 2024-03-07T07:33:10.109 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-07T07:33:10.109 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: monmap e3: 3 mons at {a=[v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0],b=[v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0],c=[v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-07T07:33:10.109 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: fsmap 2024-03-07T07:33:10.109 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: osdmap e4: 0 total, 0 up, 0 in 2024-03-07T07:33:10.110 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: mgrmap e13: x(active, since 60s) 2024-03-07T07:33:10.110 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-07T07:33:10.110 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: Cluster is now healthy 2024-03-07T07:33:10.110 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: overall HEALTH_OK 2024-03-07T07:33:10.110 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:10.110 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:10.110 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:10.110 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:10.111 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:10.111 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:10.111 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:10.111 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:10.111 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:10.111 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:10.111 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:10.111 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: Reconfiguring mon.a (unknown last config time)... 2024-03-07T07:33:10.111 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-07T07:33:10.112 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-07T07:33:10.112 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:33:10.112 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:09 smithi177 ceph-mon[36578]: Reconfiguring daemon mon.a on smithi160 2024-03-07T07:33:10.904 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:10 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-07T07:33:10.904 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:10 smithi160 ceph-mon[31963]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:11.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:10 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-07T07:33:11.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:10 smithi178 ceph-mon[36650]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:11.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:10 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-07T07:33:11.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:10 smithi177 ceph-mon[36578]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:11.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:11.970+0000 7f268aa89700 1 -- 172.21.15.160:0/4067770499 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2684101ba0 msgr2=0x7f2684102020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:33:11.975 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:11.970+0000 7f268aa89700 1 --2- 172.21.15.160:0/4067770499 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2684101ba0 0x7f2684102020 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2670009990 tx=0x7f2670009ca0 comp rx=0 tx=0).stop 2024-03-07T07:33:11.975 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:11.970+0000 7f268aa89700 1 -- 172.21.15.160:0/4067770499 shutdown_connections 2024-03-07T07:33:11.975 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:11.970+0000 7f268aa89700 1 --2- 172.21.15.160:0/4067770499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2684102560 0x7f268410df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:33:11.975 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:11.970+0000 7f268aa89700 1 --2- 172.21.15.160:0/4067770499 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2684106090 0x7f2684106470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:33:11.975 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:11.970+0000 7f268aa89700 1 --2- 172.21.15.160:0/4067770499 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2684101ba0 0x7f2684102020 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:33:11.975 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:11.970+0000 7f268aa89700 1 -- 172.21.15.160:0/4067770499 >> 172.21.15.160:0/4067770499 conn(0x7f26840763b0 msgr2=0x7f26840767c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:33:11.975 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:11.970+0000 7f268aa89700 1 -- 172.21.15.160:0/4067770499 shutdown_connections 2024-03-07T07:33:11.975 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:11.970+0000 7f268aa89700 1 -- 172.21.15.160:0/4067770499 wait complete. 2024-03-07T07:33:11.976 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:11.971+0000 7f268aa89700 1 Processor -- start 2024-03-07T07:33:11.976 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:11.971+0000 7f268aa89700 1 -- start start 2024-03-07T07:33:11.976 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:11.971+0000 7f268aa89700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2684101ba0 0x7f2684102d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:33:11.976 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:11.971+0000 7f268aa89700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2684102560 0x7f26841032d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:33:11.976 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:11.971+0000 7f268aa89700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2684106090 0x7f2684103810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:33:11.976 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:11.971+0000 7f268aa89700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2684111390 con 0x7f2684101ba0 2024-03-07T07:33:11.976 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:11.971+0000 7f268aa89700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2684111210 con 0x7f2684106090 2024-03-07T07:33:11.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:11.971+0000 7f268aa89700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2684111510 con 0x7f2684102560 2024-03-07T07:33:11.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:11.971+0000 7f2688825700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2684101ba0 0x7f2684102d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:33:11.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:11.971+0000 7f2688825700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2684101ba0 0x7f2684102d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:60754/0 (socket says 172.21.15.160:60754) 2024-03-07T07:33:11.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:11.971+0000 7f2688825700 1 -- 172.21.15.160:0/955806791 learned_addr learned my addr 172.21.15.160:0/955806791 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:33:11.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:11.971+0000 7f2689026700 1 --2- 172.21.15.160:0/955806791 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2684106090 0x7f2684103810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:33:11.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:11.971+0000 7f2683fff700 1 --2- 172.21.15.160:0/955806791 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2684102560 0x7f26841032d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:33:11.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:11.972+0000 7f2688825700 1 -- 172.21.15.160:0/955806791 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2684102560 msgr2=0x7f26841032d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:33:11.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:11.972+0000 7f2688825700 1 --2- 172.21.15.160:0/955806791 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2684102560 0x7f26841032d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:33:11.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:11.972+0000 7f2688825700 1 -- 172.21.15.160:0/955806791 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2684106090 msgr2=0x7f2684103810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:33:11.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:11.972+0000 7f2688825700 1 --2- 172.21.15.160:0/955806791 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2684106090 0x7f2684103810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:33:11.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:11.972+0000 7f2688825700 1 -- 172.21.15.160:0/955806791 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26841a6da0 con 0x7f2684101ba0 2024-03-07T07:33:11.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:11.972+0000 7f2688825700 1 --2- 172.21.15.160:0/955806791 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2684101ba0 0x7f2684102d90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f267800e9a0 tx=0x7f267800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:33:11.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:11.972+0000 7f2681ffb700 1 -- 172.21.15.160:0/955806791 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267800cbd0 con 0x7f2684101ba0 2024-03-07T07:33:11.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:11.972+0000 7f268aa89700 1 -- 172.21.15.160:0/955806791 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26841a7090 con 0x7f2684101ba0 2024-03-07T07:33:11.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:11.972+0000 7f2681ffb700 1 -- 172.21.15.160:0/955806791 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f267800cd70 con 0x7f2684101ba0 2024-03-07T07:33:11.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:11.972+0000 7f2681ffb700 1 -- 172.21.15.160:0/955806791 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2678018750 con 0x7f2684101ba0 2024-03-07T07:33:11.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:11.972+0000 7f268aa89700 1 -- 172.21.15.160:0/955806791 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26841a75d0 con 0x7f2684101ba0 2024-03-07T07:33:11.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:11.973+0000 7f2681ffb700 1 -- 172.21.15.160:0/955806791 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7f2678010430 con 0x7f2684101ba0 2024-03-07T07:33:11.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:11.973+0000 7f268aa89700 1 -- 172.21.15.160:0/955806791 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2684068e10 con 0x7f2684101ba0 2024-03-07T07:33:11.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:11.973+0000 7f2681ffb700 1 --2- 172.21.15.160:0/955806791 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2674038e80 0x7f267403b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:33:11.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:11.973+0000 7f2681ffb700 1 -- 172.21.15.160:0/955806791 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f267804c050 con 0x7f2684101ba0 2024-03-07T07:33:11.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:11.973+0000 7f2683fff700 1 --2- 172.21.15.160:0/955806791 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2674038e80 0x7f267403b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:33:11.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:11.974+0000 7f2683fff700 1 --2- 172.21.15.160:0/955806791 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2674038e80 0x7f267403b340 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f267000b2e0 tx=0x7f2670005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:33:11.980 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:11.978+0000 7f2681ffb700 1 -- 172.21.15.160:0/955806791 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f2678020360 con 0x7f2684101ba0 2024-03-07T07:33:12.175 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:12.174+0000 7f268aa89700 1 -- 172.21.15.160:0/955806791 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f26841a7830 con 0x7f2684101ba0 2024-03-07T07:33:12.175 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:12.174+0000 7f2681ffb700 1 -- 172.21.15.160:0/955806791 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+277 (secure 0 0 0) 0x7f2678010720 con 0x7f2684101ba0 2024-03-07T07:33:12.175 INFO:teuthology.orchestra.run.smithi160.stdout:# minimal ceph.conf for 826f6582-dc54-11ee-95c6-87774f69a715 2024-03-07T07:33:12.175 INFO:teuthology.orchestra.run.smithi160.stdout:[global] 2024-03-07T07:33:12.175 INFO:teuthology.orchestra.run.smithi160.stdout: fsid = 826f6582-dc54-11ee-95c6-87774f69a715 2024-03-07T07:33:12.175 INFO:teuthology.orchestra.run.smithi160.stdout: mon_host = [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-03-07T07:33:12.178 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:12.177+0000 7f268aa89700 1 -- 172.21.15.160:0/955806791 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2674038e80 msgr2=0x7f267403b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:33:12.178 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:12.177+0000 7f268aa89700 1 --2- 172.21.15.160:0/955806791 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2674038e80 0x7f267403b340 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f267000b2e0 tx=0x7f2670005fb0 comp rx=0 tx=0).stop 2024-03-07T07:33:12.178 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:12.177+0000 7f268aa89700 1 -- 172.21.15.160:0/955806791 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2684101ba0 msgr2=0x7f2684102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:33:12.178 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:12.177+0000 7f268aa89700 1 --2- 172.21.15.160:0/955806791 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2684101ba0 0x7f2684102d90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f267800e9a0 tx=0x7f267800ecb0 comp rx=0 tx=0).stop 2024-03-07T07:33:12.178 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:12.177+0000 7f268aa89700 1 -- 172.21.15.160:0/955806791 shutdown_connections 2024-03-07T07:33:12.178 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:12.177+0000 7f268aa89700 1 --2- 172.21.15.160:0/955806791 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2684106090 0x7f2684103810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:33:12.178 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:12.177+0000 7f268aa89700 1 --2- 172.21.15.160:0/955806791 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2684101ba0 0x7f2684102d90 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:33:12.178 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:12.177+0000 7f268aa89700 1 --2- 172.21.15.160:0/955806791 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2674038e80 0x7f267403b340 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:33:12.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:12.177+0000 7f268aa89700 1 --2- 172.21.15.160:0/955806791 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2684102560 0x7f26841032d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:33:12.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:12.177+0000 7f268aa89700 1 -- 172.21.15.160:0/955806791 >> 172.21.15.160:0/955806791 conn(0x7f26840763b0 msgr2=0x7f2684100b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:33:12.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:12.178+0000 7f268aa89700 1 -- 172.21.15.160:0/955806791 shutdown_connections 2024-03-07T07:33:12.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:12.178+0000 7f268aa89700 1 -- 172.21.15.160:0/955806791 wait complete. 2024-03-07T07:33:12.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:12 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:12.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:12 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:12.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:12 smithi178 ceph-mon[36650]: Reconfiguring mon.b (monmap changed)... 2024-03-07T07:33:12.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:12 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-07T07:33:12.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:12 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-07T07:33:12.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:12 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:33:12.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:12 smithi178 ceph-mon[36650]: Reconfiguring daemon mon.b on smithi177 2024-03-07T07:33:12.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:12 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/955806791' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:33:12.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:12 smithi178 ceph-mon[36650]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:12.814 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:12 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:12.814 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:12 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:12.814 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:12 smithi160 ceph-mon[31963]: Reconfiguring mon.b (monmap changed)... 2024-03-07T07:33:12.815 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:12 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-07T07:33:12.815 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:12 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-07T07:33:12.815 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:12 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:33:12.815 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:12 smithi160 ceph-mon[31963]: Reconfiguring daemon mon.b on smithi177 2024-03-07T07:33:12.815 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:12 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/955806791' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:33:12.815 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:12 smithi160 ceph-mon[31963]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:12.841 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-07T07:33:12.842 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-03-07T07:33:12.842 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/ceph/ceph.conf 2024-03-07T07:33:12.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:12 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:12.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:12 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:12.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:12 smithi177 ceph-mon[36578]: Reconfiguring mon.b (monmap changed)... 2024-03-07T07:33:12.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:12 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-07T07:33:12.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:12 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-07T07:33:12.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:12 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:33:12.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:12 smithi177 ceph-mon[36578]: Reconfiguring daemon mon.b on smithi177 2024-03-07T07:33:12.858 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:12 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/955806791' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:33:12.858 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:12 smithi177 ceph-mon[36578]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:12.872 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-03-07T07:33:12.873 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-07T07:33:12.941 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-03-07T07:33:12.941 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/ceph/ceph.conf 2024-03-07T07:33:12.971 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-03-07T07:33:12.971 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-07T07:33:13.038 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-03-07T07:33:13.038 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/ceph/ceph.conf 2024-03-07T07:33:13.073 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-03-07T07:33:13.073 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-07T07:33:13.144 INFO:tasks.cephadm:Adding mgr.x on smithi160 2024-03-07T07:33:13.144 INFO:tasks.cephadm:Adding mgr.y on smithi177 2024-03-07T07:33:13.144 INFO:tasks.cephadm:Adding mgr.z on smithi178 2024-03-07T07:33:13.144 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph orch apply mgr '3;smithi160=x;smithi177=y;smithi178=z' 2024-03-07T07:33:13.429 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.c/config 2024-03-07T07:33:14.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:14 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:14.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:14 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:14.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:14 smithi178 ceph-mon[36650]: Reconfiguring mon.c (monmap changed)... 2024-03-07T07:33:14.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:14 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-07T07:33:14.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:14 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-07T07:33:14.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:14 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:33:14.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:14 smithi178 ceph-mon[36650]: Reconfiguring daemon mon.c on smithi178 2024-03-07T07:33:14.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:14 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:14.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:14 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:14.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:14 smithi177 ceph-mon[36578]: Reconfiguring mon.c (monmap changed)... 2024-03-07T07:33:14.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:14 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-07T07:33:14.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:14 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-07T07:33:14.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:14 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:33:14.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:14 smithi177 ceph-mon[36578]: Reconfiguring daemon mon.c on smithi178 2024-03-07T07:33:14.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:14 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:14.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:14 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:14.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:14 smithi160 ceph-mon[31963]: Reconfiguring mon.c (monmap changed)... 2024-03-07T07:33:14.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:14 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-07T07:33:14.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:14 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-07T07:33:14.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:14 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:33:14.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:14 smithi160 ceph-mon[31963]: Reconfiguring daemon mon.c on smithi178 2024-03-07T07:33:15.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:15 smithi178 ceph-mon[36650]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:15.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:15 smithi177 ceph-mon[36578]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:15.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:15 smithi160 ceph-mon[31963]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:15.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.723+0000 7f9e6b7e8700 1 -- 172.21.15.178:0/3500949272 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e64102180 msgr2=0x7f9e641025a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:33:15.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.723+0000 7f9e6b7e8700 1 --2- 172.21.15.178:0/3500949272 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e64102180 0x7f9e641025a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9e54009990 tx=0x7f9e54009ca0 comp rx=0 tx=0).stop 2024-03-07T07:33:15.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.723+0000 7f9e6b7e8700 1 -- 172.21.15.178:0/3500949272 shutdown_connections 2024-03-07T07:33:15.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.723+0000 7f9e6b7e8700 1 --2- 172.21.15.178:0/3500949272 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e64103d20 0x7f9e6410a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:33:15.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.723+0000 7f9e6b7e8700 1 --2- 172.21.15.178:0/3500949272 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e64102180 0x7f9e641025a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:33:15.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.723+0000 7f9e6b7e8700 1 --2- 172.21.15.178:0/3500949272 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9e64103380 0x7f9e641037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:33:15.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.723+0000 7f9e6b7e8700 1 -- 172.21.15.178:0/3500949272 >> 172.21.15.178:0/3500949272 conn(0x7f9e640fd760 msgr2=0x7f9e640ffb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:33:15.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.723+0000 7f9e6b7e8700 1 -- 172.21.15.178:0/3500949272 shutdown_connections 2024-03-07T07:33:15.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.723+0000 7f9e6b7e8700 1 -- 172.21.15.178:0/3500949272 wait complete. 2024-03-07T07:33:15.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.724+0000 7f9e6b7e8700 1 Processor -- start 2024-03-07T07:33:15.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.724+0000 7f9e6b7e8700 1 -- start start 2024-03-07T07:33:15.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.724+0000 7f9e6b7e8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e64102180 0x7f9e641983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:33:15.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.724+0000 7f9e6b7e8700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9e64103380 0x7f9e64198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:33:15.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.724+0000 7f9e6b7e8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e64103d20 0x7f9e6419f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:33:15.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.724+0000 7f9e6b7e8700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6410dd60 con 0x7f9e64102180 2024-03-07T07:33:15.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.724+0000 7f9e6b7e8700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6410dbe0 con 0x7f9e64103380 2024-03-07T07:33:15.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.724+0000 7f9e6b7e8700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6410dee0 con 0x7f9e64103d20 2024-03-07T07:33:15.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.724+0000 7f9e69d85700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e64103d20 0x7f9e6419f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:33:15.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.724+0000 7f9e69d85700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e64103d20 0x7f9e6419f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:49348/0 (socket says 172.21.15.178:49348) 2024-03-07T07:33:15.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.724+0000 7f9e69d85700 1 -- 172.21.15.178:0/2462769420 learned_addr learned my addr 172.21.15.178:0/2462769420 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-03-07T07:33:15.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.724+0000 7f9e69584700 1 --2- 172.21.15.178:0/2462769420 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e64102180 0x7f9e641983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:33:15.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.724+0000 7f9e68d83700 1 --2- 172.21.15.178:0/2462769420 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9e64103380 0x7f9e64198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:33:15.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.725+0000 7f9e69584700 1 -- 172.21.15.178:0/2462769420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e64103d20 msgr2=0x7f9e6419f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:33:15.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.725+0000 7f9e69584700 1 --2- 172.21.15.178:0/2462769420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e64103d20 0x7f9e6419f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:33:15.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.725+0000 7f9e69584700 1 -- 172.21.15.178:0/2462769420 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9e64103380 msgr2=0x7f9e64198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:33:15.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.725+0000 7f9e69584700 1 --2- 172.21.15.178:0/2462769420 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9e64103380 0x7f9e64198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:33:15.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.725+0000 7f9e69584700 1 -- 172.21.15.178:0/2462769420 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e641a0030 con 0x7f9e64102180 2024-03-07T07:33:15.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.725+0000 7f9e69d85700 1 --2- 172.21.15.178:0/2462769420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e64103d20 0x7f9e6419f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:33:15.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.725+0000 7f9e69584700 1 --2- 172.21.15.178:0/2462769420 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e64102180 0x7f9e641983d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9e540098e0 tx=0x7f9e5400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:33:15.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.725+0000 7f9e5a7fc700 1 -- 172.21.15.178:0/2462769420 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e54026070 con 0x7f9e64102180 2024-03-07T07:33:15.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.725+0000 7f9e5a7fc700 1 -- 172.21.15.178:0/2462769420 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9e5400fe20 con 0x7f9e64102180 2024-03-07T07:33:15.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.725+0000 7f9e5a7fc700 1 -- 172.21.15.178:0/2462769420 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e54017650 con 0x7f9e64102180 2024-03-07T07:33:15.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.725+0000 7f9e6b7e8700 1 -- 172.21.15.178:0/2462769420 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e64074380 con 0x7f9e64102180 2024-03-07T07:33:15.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.725+0000 7f9e6b7e8700 1 -- 172.21.15.178:0/2462769420 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e64074880 con 0x7f9e64102180 2024-03-07T07:33:15.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.727+0000 7f9e5a7fc700 1 -- 172.21.15.178:0/2462769420 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7f9e54029070 con 0x7f9e64102180 2024-03-07T07:33:15.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.727+0000 7f9e6b7e8700 1 -- 172.21.15.178:0/2462769420 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e64068e10 con 0x7f9e64102180 2024-03-07T07:33:15.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.727+0000 7f9e5a7fc700 1 --2- 172.21.15.178:0/2462769420 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f9e50038e80 0x7f9e5003b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:33:15.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.727+0000 7f9e5a7fc700 1 -- 172.21.15.178:0/2462769420 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9e5404cdf0 con 0x7f9e64102180 2024-03-07T07:33:15.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.727+0000 7f9e68d83700 1 --2- 172.21.15.178:0/2462769420 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f9e50038e80 0x7f9e5003b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:33:15.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.727+0000 7f9e68d83700 1 --2- 172.21.15.178:0/2462769420 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f9e50038e80 0x7f9e5003b340 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9e60009bf0 tx=0x7f9e60009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:33:15.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.732+0000 7f9e5a7fc700 1 -- 172.21.15.178:0/2462769420 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9e54013070 con 0x7f9e64102180 2024-03-07T07:33:15.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.937+0000 7f9e6b7e8700 1 -- 172.21.15.178:0/2462769420 --> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi160=x;smithi177=y;smithi178=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f9e64062590 con 0x7f9e50038e80 2024-03-07T07:33:15.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.947+0000 7f9e5a7fc700 1 -- 172.21.15.178:0/2462769420 <== mgr.14152 v2:172.21.15.160:6800/419630619 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f9e64062590 con 0x7f9e50038e80 2024-03-07T07:33:15.946 INFO:teuthology.orchestra.run.smithi178.stdout:Scheduled mgr update... 2024-03-07T07:33:15.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.949+0000 7f9e6b7e8700 1 -- 172.21.15.178:0/2462769420 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f9e50038e80 msgr2=0x7f9e5003b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:33:15.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.949+0000 7f9e6b7e8700 1 --2- 172.21.15.178:0/2462769420 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f9e50038e80 0x7f9e5003b340 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9e60009bf0 tx=0x7f9e60009260 comp rx=0 tx=0).stop 2024-03-07T07:33:15.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.949+0000 7f9e6b7e8700 1 -- 172.21.15.178:0/2462769420 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e64102180 msgr2=0x7f9e641983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:33:15.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.949+0000 7f9e6b7e8700 1 --2- 172.21.15.178:0/2462769420 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e64102180 0x7f9e641983d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9e540098e0 tx=0x7f9e5400f690 comp rx=0 tx=0).stop 2024-03-07T07:33:15.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.949+0000 7f9e6b7e8700 1 -- 172.21.15.178:0/2462769420 shutdown_connections 2024-03-07T07:33:15.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.949+0000 7f9e6b7e8700 1 --2- 172.21.15.178:0/2462769420 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f9e50038e80 0x7f9e5003b340 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:33:15.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.949+0000 7f9e6b7e8700 1 --2- 172.21.15.178:0/2462769420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e64103d20 0x7f9e6419f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:33:15.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.949+0000 7f9e6b7e8700 1 --2- 172.21.15.178:0/2462769420 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e64102180 0x7f9e641983d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:33:15.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.949+0000 7f9e6b7e8700 1 --2- 172.21.15.178:0/2462769420 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9e64103380 0x7f9e64198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:33:15.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.949+0000 7f9e6b7e8700 1 -- 172.21.15.178:0/2462769420 >> 172.21.15.178:0/2462769420 conn(0x7f9e640fd760 msgr2=0x7f9e64106750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:33:15.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.949+0000 7f9e6b7e8700 1 -- 172.21.15.178:0/2462769420 shutdown_connections 2024-03-07T07:33:15.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:33:15.949+0000 7f9e6b7e8700 1 -- 172.21.15.178:0/2462769420 wait complete. 2024-03-07T07:33:16.518 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:16 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:16.518 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:16 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:16.518 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:16 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:33:16.518 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:16 smithi178 ceph-mon[36650]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi160=x;smithi177=y;smithi178=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:33:16.518 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:16 smithi178 ceph-mon[36650]: Saving service mgr spec with placement smithi160=x;smithi177=y;smithi178=z;count:3 2024-03-07T07:33:16.518 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:16 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:16.518 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:16 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:33:16.518 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:16 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:33:16.518 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:16 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:16.518 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:16 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-07T07:33:16.518 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:16 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-07T07:33:16.519 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:16 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-07T07:33:16.519 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:16 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:33:16.519 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:16 smithi178 ceph-mon[36650]: Deploying daemon mgr.z on smithi178 2024-03-07T07:33:16.519 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:16 smithi178 ceph-mon[36650]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:16.542 DEBUG:teuthology.orchestra.run.smithi177:mgr.y> sudo journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@mgr.y.service 2024-03-07T07:33:16.545 DEBUG:teuthology.orchestra.run.smithi178:mgr.z> sudo journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@mgr.z.service 2024-03-07T07:33:16.547 INFO:tasks.cephadm:Deploying OSDs... 2024-03-07T07:33:16.547 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-03-07T07:33:16.548 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/scratch_devs of=/dev/stdout 2024-03-07T07:33:16.565 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-07T07:33:16.565 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_1 2024-03-07T07:33:16.621 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-07T07:33:16.621 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-07T07:33:16.621 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-03-07T07:33:16.621 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-07T07:33:16.621 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:device_t:s0 2024-03-07T07:33:16.621 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2024-03-07 07:32:21.260596403 +0000 2024-03-07T07:33:16.621 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2024-03-07 07:26:57.481041486 +0000 2024-03-07T07:33:16.621 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2024-03-07 07:26:57.481041486 +0000 2024-03-07T07:33:16.622 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: 2024-03-07 07:26:57.481041486 +0000 2024-03-07T07:33:16.622 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-07T07:33:16.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:16 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:16.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:16 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:16.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:16 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:33:16.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:16 smithi160 ceph-mon[31963]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi160=x;smithi177=y;smithi178=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:33:16.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:16 smithi160 ceph-mon[31963]: Saving service mgr spec with placement smithi160=x;smithi177=y;smithi178=z;count:3 2024-03-07T07:33:16.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:16 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:16.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:16 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:33:16.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:16 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:33:16.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:16 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:16.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:16 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-07T07:33:16.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:16 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-07T07:33:16.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:16 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-07T07:33:16.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:16 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:33:16.679 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:16 smithi160 ceph-mon[31963]: Deploying daemon mgr.z on smithi178 2024-03-07T07:33:16.679 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:16 smithi160 ceph-mon[31963]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:16.688 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2024-03-07T07:33:16.688 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2024-03-07T07:33:16.688 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.000217472 s, 2.4 MB/s 2024-03-07T07:33:16.689 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-07T07:33:16.746 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_2 2024-03-07T07:33:16.802 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-07T07:33:16.802 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-07T07:33:16.802 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-03-07T07:33:16.802 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-07T07:33:16.802 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:device_t:s0 2024-03-07T07:33:16.802 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2024-03-07 07:32:21.290595974 +0000 2024-03-07T07:33:16.803 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2024-03-07 07:26:57.849036286 +0000 2024-03-07T07:33:16.803 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2024-03-07 07:26:57.849036286 +0000 2024-03-07T07:33:16.803 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: 2024-03-07 07:26:57.849036286 +0000 2024-03-07T07:33:16.803 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-07T07:33:16.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:16 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:16.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:16 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:16.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:16 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:33:16.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:16 smithi177 ceph-mon[36578]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi160=x;smithi177=y;smithi178=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:33:16.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:16 smithi177 ceph-mon[36578]: Saving service mgr spec with placement smithi160=x;smithi177=y;smithi178=z;count:3 2024-03-07T07:33:16.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:16 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:16.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:16 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:33:16.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:16 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:33:16.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:16 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:16.858 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:16 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-07T07:33:16.858 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:16 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-07T07:33:16.858 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:16 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-07T07:33:16.858 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:16 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:33:16.858 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:16 smithi177 ceph-mon[36578]: Deploying daemon mgr.z on smithi178 2024-03-07T07:33:16.858 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:16 smithi177 ceph-mon[36578]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:16.868 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2024-03-07T07:33:16.868 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2024-03-07T07:33:16.868 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.000205846 s, 2.5 MB/s 2024-03-07T07:33:16.869 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-07T07:33:16.927 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_3 2024-03-07T07:33:16.982 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-07T07:33:16.982 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-07T07:33:16.982 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-03-07T07:33:16.982 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-07T07:33:16.982 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:device_t:s0 2024-03-07T07:33:16.982 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2024-03-07 07:32:21.317595588 +0000 2024-03-07T07:33:16.982 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2024-03-07 07:26:58.188031495 +0000 2024-03-07T07:33:16.982 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2024-03-07 07:26:58.188031495 +0000 2024-03-07T07:33:16.983 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: 2024-03-07 07:26:58.188031495 +0000 2024-03-07T07:33:16.983 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-07T07:33:17.049 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2024-03-07T07:33:17.049 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2024-03-07T07:33:17.049 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.000197255 s, 2.6 MB/s 2024-03-07T07:33:17.050 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-07T07:33:17.105 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_4 2024-03-07T07:33:17.161 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-07T07:33:17.161 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-07T07:33:17.161 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-03-07T07:33:17.161 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-07T07:33:17.162 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:device_t:s0 2024-03-07T07:33:17.162 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2024-03-07 07:32:21.345595187 +0000 2024-03-07T07:33:17.162 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2024-03-07 07:26:58.539026535 +0000 2024-03-07T07:33:17.162 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2024-03-07 07:26:58.539026535 +0000 2024-03-07T07:33:17.162 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: 2024-03-07 07:26:58.539026535 +0000 2024-03-07T07:33:17.162 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-07T07:33:17.228 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2024-03-07T07:33:17.228 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2024-03-07T07:33:17.228 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.000194193 s, 2.6 MB/s 2024-03-07T07:33:17.229 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-07T07:33:17.285 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-03-07T07:33:17.285 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/scratch_devs of=/dev/stdout 2024-03-07T07:33:17.303 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-07T07:33:17.304 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_1 2024-03-07T07:33:17.362 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-07T07:33:17.362 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-07T07:33:17.362 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 5h/5d Inode: 661 Links: 1 2024-03-07T07:33:17.362 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-07T07:33:17.363 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2024-03-07T07:33:17.363 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2024-03-07 07:33:01.500748120 +0000 2024-03-07T07:33:17.363 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2024-03-07 07:27:19.334450682 +0000 2024-03-07T07:33:17.363 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2024-03-07 07:27:19.334450682 +0000 2024-03-07T07:33:17.363 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: 2024-03-07 07:27:19.334450682 +0000 2024-03-07T07:33:17.363 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-07T07:33:17.431 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2024-03-07T07:33:17.432 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2024-03-07T07:33:17.432 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000288304 s, 1.8 MB/s 2024-03-07T07:33:17.433 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-07T07:33:17.491 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_2 2024-03-07T07:33:17.547 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-07T07:33:17.548 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-07T07:33:17.548 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 5h/5d Inode: 681 Links: 1 2024-03-07T07:33:17.548 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-07T07:33:17.548 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2024-03-07T07:33:17.548 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2024-03-07 07:33:01.500748120 +0000 2024-03-07T07:33:17.548 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2024-03-07 07:27:19.698440568 +0000 2024-03-07T07:33:17.548 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2024-03-07 07:27:19.698440568 +0000 2024-03-07T07:33:17.548 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: 2024-03-07 07:27:19.698440568 +0000 2024-03-07T07:33:17.548 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-07T07:33:17.616 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2024-03-07T07:33:17.616 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2024-03-07T07:33:17.616 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000189604 s, 2.7 MB/s 2024-03-07T07:33:17.617 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-07T07:33:17.674 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_3 2024-03-07T07:33:17.730 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-07T07:33:17.731 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-07T07:33:17.731 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-03-07T07:33:17.731 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-07T07:33:17.731 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2024-03-07T07:33:17.731 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2024-03-07 07:33:01.501748094 +0000 2024-03-07T07:33:17.731 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2024-03-07 07:27:20.041431037 +0000 2024-03-07T07:33:17.731 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2024-03-07 07:27:20.041431037 +0000 2024-03-07T07:33:17.731 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: 2024-03-07 07:27:20.041431037 +0000 2024-03-07T07:33:17.732 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-07T07:33:17.799 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2024-03-07T07:33:17.799 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2024-03-07T07:33:17.799 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000207424 s, 2.5 MB/s 2024-03-07T07:33:17.801 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-07T07:33:17.858 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_4 2024-03-07T07:33:17.915 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-07T07:33:17.915 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-07T07:33:17.915 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 5h/5d Inode: 719 Links: 1 2024-03-07T07:33:17.915 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-07T07:33:17.915 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2024-03-07T07:33:17.915 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2024-03-07 07:33:01.501748094 +0000 2024-03-07T07:33:17.915 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2024-03-07 07:27:20.381421590 +0000 2024-03-07T07:33:17.915 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2024-03-07 07:27:20.381421590 +0000 2024-03-07T07:33:17.915 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: 2024-03-07 07:27:20.381421590 +0000 2024-03-07T07:33:17.916 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-07T07:33:17.982 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2024-03-07T07:33:17.982 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2024-03-07T07:33:17.982 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.00022744 s, 2.3 MB/s 2024-03-07T07:33:17.983 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-07T07:33:18.040 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-03-07T07:33:18.040 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/scratch_devs of=/dev/stdout 2024-03-07T07:33:18.057 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Mar 07 07:33:17 smithi178 systemd[1]: Starting Ceph mgr.z for 826f6582-dc54-11ee-95c6-87774f69a715... 2024-03-07T07:33:18.058 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-07T07:33:18.058 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_1 2024-03-07T07:33:18.113 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-07T07:33:18.113 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-07T07:33:18.113 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 661 Links: 1 2024-03-07T07:33:18.113 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-07T07:33:18.113 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-03-07T07:33:18.113 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-03-07 07:33:17.885997368 +0000 2024-03-07T07:33:18.113 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-03-07 07:27:25.724302595 +0000 2024-03-07T07:33:18.113 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-03-07 07:27:25.724302595 +0000 2024-03-07T07:33:18.113 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-03-07 07:27:25.724302595 +0000 2024-03-07T07:33:18.113 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-07T07:33:18.180 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-03-07T07:33:18.180 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-03-07T07:33:18.181 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000112231 s, 4.6 MB/s 2024-03-07T07:33:18.182 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-07T07:33:18.237 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_2 2024-03-07T07:33:18.293 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-07T07:33:18.293 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-07T07:33:18.293 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-03-07T07:33:18.293 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-07T07:33:18.294 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-03-07T07:33:18.294 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-03-07 07:33:17.886997351 +0000 2024-03-07T07:33:18.294 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-03-07 07:27:26.074296100 +0000 2024-03-07T07:33:18.294 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-03-07 07:27:26.074296100 +0000 2024-03-07T07:33:18.294 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-03-07 07:27:26.074296100 +0000 2024-03-07T07:33:18.294 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-07T07:33:18.358 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Mar 07 07:33:18 smithi178 podman[38263]: 2024-03-07 07:33:18.072116723 +0000 UTC m=+0.017836306 image pull e362cd7cebfee51077fb90b7f01895811140c9a4272edd83ad9910b05e3a46f6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2 2024-03-07T07:33:18.358 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Mar 07 07:33:18 smithi178 podman[38263]: 2024-03-07 07:33:18.184933852 +0000 UTC m=+0.130653420 container create e1f247c6e191f52a6ac942cbc8dc93251793412c717309519ed6424207bc58ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-z, org.label-schema.build-date=20240222, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b) 2024-03-07T07:33:18.360 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-03-07T07:33:18.360 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-03-07T07:33:18.360 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 8.5565e-05 s, 6.0 MB/s 2024-03-07T07:33:18.361 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-07T07:33:18.416 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_3 2024-03-07T07:33:18.471 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-07T07:33:18.471 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-07T07:33:18.471 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 700 Links: 1 2024-03-07T07:33:18.471 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-07T07:33:18.471 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-03-07T07:33:18.471 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-03-07 07:33:17.886997351 +0000 2024-03-07T07:33:18.471 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-03-07 07:27:26.405289958 +0000 2024-03-07T07:33:18.471 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-03-07 07:27:26.405289958 +0000 2024-03-07T07:33:18.471 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-03-07 07:27:26.405289958 +0000 2024-03-07T07:33:18.472 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-07T07:33:18.537 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-03-07T07:33:18.538 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-03-07T07:33:18.538 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000113503 s, 4.5 MB/s 2024-03-07T07:33:18.539 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-07T07:33:18.595 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_4 2024-03-07T07:33:18.650 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-07T07:33:18.650 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-07T07:33:18.650 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-03-07T07:33:18.650 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-07T07:33:18.650 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-03-07T07:33:18.650 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-03-07 07:33:17.887997333 +0000 2024-03-07T07:33:18.650 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-03-07 07:27:26.744283667 +0000 2024-03-07T07:33:18.651 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-03-07 07:27:26.744283667 +0000 2024-03-07T07:33:18.651 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-03-07 07:27:26.744283667 +0000 2024-03-07T07:33:18.651 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-07T07:33:18.714 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Mar 07 07:33:18 smithi178 podman[38263]: 2024-03-07 07:33:18.380466222 +0000 UTC m=+0.326185794 container init e1f247c6e191f52a6ac942cbc8dc93251793412c717309519ed6424207bc58ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-z, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD) 2024-03-07T07:33:18.714 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Mar 07 07:33:18 smithi178 podman[38263]: 2024-03-07 07:33:18.383110765 +0000 UTC m=+0.328830334 container start e1f247c6e191f52a6ac942cbc8dc93251793412c717309519ed6424207bc58ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-z, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, ceph=True, RELEASE=wip-lusov-qdb-8b896a9, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-07T07:33:18.714 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Mar 07 07:33:18 smithi178 ceph-mgr[38379]: -- 172.21.15.178:0/177584941 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f2d9d625a0 con 0x55f2d9d36c00 2024-03-07T07:33:18.714 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Mar 07 07:33:18 smithi178 bash[38263]: e1f247c6e191f52a6ac942cbc8dc93251793412c717309519ed6424207bc58ad 2024-03-07T07:33:18.714 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Mar 07 07:33:18 smithi178 systemd[1]: Started Ceph mgr.z for 826f6582-dc54-11ee-95c6-87774f69a715. 2024-03-07T07:33:18.716 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-03-07T07:33:18.716 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-03-07T07:33:18.716 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 8.9507e-05 s, 5.7 MB/s 2024-03-07T07:33:18.717 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-07T07:33:18.772 INFO:tasks.cephadm:Deploying osd.0 on smithi160 with /dev/vg_nvme/lv_4... 2024-03-07T07:33:18.772 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-07T07:33:19.016 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:33:19.022 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Mar 07 07:33:18 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-z[38341]: 2024-03-07T07:33:18.930+0000 7f71827fa200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-07T07:33:19.306 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Mar 07 07:33:19 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-z[38341]: 2024-03-07T07:33:19.022+0000 7f71827fa200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-07T07:33:19.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:19 smithi178 ceph-mon[36650]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:19.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:19 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:19.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:19 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:19.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:19 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:19.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:19 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-07T07:33:19.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:19 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-07T07:33:19.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:19 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-07T07:33:19.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:19 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:33:19.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:19 smithi160 ceph-mon[31963]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:19.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:19 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:19.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:19 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:19.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:19 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:19.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:19 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-07T07:33:19.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:19 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-07T07:33:19.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:19 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-07T07:33:19.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:19 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:33:19.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:19 smithi177 ceph-mon[36578]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:19.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:19 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:19.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:19 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:19.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:19 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:19.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:19 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-07T07:33:19.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:19 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-07T07:33:19.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:19 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-07T07:33:19.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:19 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:33:19.992 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Mar 07 07:33:19 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-z[38341]: 2024-03-07T07:33:19.657+0000 7f71827fa200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-07T07:33:19.992 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Mar 07 07:33:19 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-z[38341]: 2024-03-07T07:33:19.748+0000 7f71827fa200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-07T07:33:19.993 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Mar 07 07:33:19 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-z[38341]: 2024-03-07T07:33:19.894+0000 7f71827fa200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-07T07:33:20.262 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Mar 07 07:33:19 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-z[38341]: 2024-03-07T07:33:19.992+0000 7f71827fa200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-07T07:33:20.262 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Mar 07 07:33:20 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-z[38341]: 2024-03-07T07:33:20.078+0000 7f71827fa200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-07T07:33:20.322 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:20 smithi160 ceph-mon[31963]: Deploying daemon mgr.y on smithi177 2024-03-07T07:33:20.554 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:20 smithi178 ceph-mon[36650]: Deploying daemon mgr.y on smithi177 2024-03-07T07:33:20.554 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Mar 07 07:33:20 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-z[38341]: 2024-03-07T07:33:20.352+0000 7f71827fa200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-07T07:33:20.554 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Mar 07 07:33:20 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-z[38341]: 2024-03-07T07:33:20.466+0000 7f71827fa200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-07T07:33:20.554 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Mar 07 07:33:20 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-z[38341]: 2024-03-07T07:33:20.553+0000 7f71827fa200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-07T07:33:20.586 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:20 smithi177 ceph-mon[36578]: Deploying daemon mgr.y on smithi177 2024-03-07T07:33:20.586 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Mar 07 07:33:20 smithi177 systemd[1]: Starting Ceph mgr.y for 826f6582-dc54-11ee-95c6-87774f69a715... 2024-03-07T07:33:20.806 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Mar 07 07:33:20 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-z[38341]: 2024-03-07T07:33:20.649+0000 7f71827fa200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-07T07:33:20.857 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Mar 07 07:33:20 smithi177 podman[37941]: 2024-03-07 07:33:20.488930785 +0000 UTC m=+0.017823790 image pull e362cd7cebfee51077fb90b7f01895811140c9a4272edd83ad9910b05e3a46f6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2 2024-03-07T07:33:20.857 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Mar 07 07:33:20 smithi177 podman[37941]: 2024-03-07 07:33:20.589558346 +0000 UTC m=+0.118451333 container create 44e7a1cb82b18b91af95635cea4807e8f3b7ae37c4746bd0f75645d6e3cecd7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-y, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-03-07T07:33:20.857 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Mar 07 07:33:20 smithi177 podman[37941]: 2024-03-07 07:33:20.808307099 +0000 UTC m=+0.337200098 container init 44e7a1cb82b18b91af95635cea4807e8f3b7ae37c4746bd0f75645d6e3cecd7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-y, maintainer=Guillaume Abrioux , GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True) 2024-03-07T07:33:20.857 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Mar 07 07:33:20 smithi177 podman[37941]: 2024-03-07 07:33:20.81092004 +0000 UTC m=+0.339813028 container start 44e7a1cb82b18b91af95635cea4807e8f3b7ae37c4746bd0f75645d6e3cecd7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-y, RELEASE=wip-lusov-qdb-8b896a9, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-07T07:33:20.857 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Mar 07 07:33:20 smithi177 ceph-mgr[37960]: -- 172.21.15.177:0/1251471630 <== mon.2 v2:172.21.15.177:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557f542405a0 con 0x557f54214800 2024-03-07T07:33:20.877 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-07T07:33:20.878 INFO:teuthology.orchestra.run.smithi160.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-07T07:33:20.878 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 10+0 records in 2024-03-07T07:33:20.878 INFO:teuthology.orchestra.run.smithi160.stderr:10+0 records out 2024-03-07T07:33:20.878 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112924 s, 929 MB/s 2024-03-07T07:33:20.878 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping successful for: 2024-03-07T07:33:21.263 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Mar 07 07:33:20 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-z[38341]: 2024-03-07T07:33:20.829+0000 7f71827fa200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-07T07:33:21.263 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Mar 07 07:33:20 smithi177 bash[37941]: 44e7a1cb82b18b91af95635cea4807e8f3b7ae37c4746bd0f75645d6e3cecd7c 2024-03-07T07:33:21.264 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Mar 07 07:33:20 smithi177 systemd[1]: Started Ceph mgr.y for 826f6582-dc54-11ee-95c6-87774f69a715. 2024-03-07T07:33:21.494 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi160:vg_nvme/lv_4 2024-03-07T07:33:21.519 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:21 smithi160 ceph-mon[31963]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:21.519 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:21 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:21.519 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:21 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:21.519 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:21 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:21.519 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:21 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:21.519 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:21 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:33:21.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:21 smithi178 ceph-mon[36650]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:21.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:21 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:21.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:21 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:21.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:21 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:21.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:21 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:21.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:21 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:33:21.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:21 smithi177 ceph-mon[36578]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:21.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:21 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:21.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:21 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:21.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:21 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:21.608 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:21 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:21.608 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:21 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:33:21.608 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Mar 07 07:33:21 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-y[37956]: 2024-03-07T07:33:21.391+0000 7f6c90067200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-07T07:33:21.608 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Mar 07 07:33:21 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-y[37956]: 2024-03-07T07:33:21.494+0000 7f6c90067200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-07T07:33:21.733 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:33:22.306 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Mar 07 07:33:21 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-z[38341]: 2024-03-07T07:33:21.841+0000 7f71827fa200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-07T07:33:22.306 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Mar 07 07:33:21 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-z[38341]: 2024-03-07T07:33:21.970+0000 7f71827fa200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-07T07:33:22.448 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Mar 07 07:33:22 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-y[37956]: 2024-03-07T07:33:22.194+0000 7f6c90067200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-07T07:33:22.448 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Mar 07 07:33:22 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-y[37956]: 2024-03-07T07:33:22.291+0000 7f6c90067200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-07T07:33:22.604 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Mar 07 07:33:22 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-z[38341]: 2024-03-07T07:33:22.311+0000 7f71827fa200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-07T07:33:22.604 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Mar 07 07:33:22 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-z[38341]: 2024-03-07T07:33:22.401+0000 7f71827fa200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-07T07:33:22.604 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Mar 07 07:33:22 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-z[38341]: 2024-03-07T07:33:22.603+0000 7f71827fa200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-07T07:33:22.856 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Mar 07 07:33:22 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-y[37956]: 2024-03-07T07:33:22.447+0000 7f6c90067200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-07T07:33:22.857 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Mar 07 07:33:22 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-y[37956]: 2024-03-07T07:33:22.547+0000 7f6c90067200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-07T07:33:22.857 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Mar 07 07:33:22 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-y[37956]: 2024-03-07T07:33:22.631+0000 7f6c90067200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-07T07:33:23.056 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Mar 07 07:33:22 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-z[38341]: 2024-03-07T07:33:22.783+0000 7f71827fa200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-07T07:33:23.182 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Mar 07 07:33:22 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-y[37956]: 2024-03-07T07:33:22.893+0000 7f6c90067200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-07T07:33:23.182 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Mar 07 07:33:23 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-y[37956]: 2024-03-07T07:33:23.002+0000 7f6c90067200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-07T07:33:23.182 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Mar 07 07:33:23 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-y[37956]: 2024-03-07T07:33:23.087+0000 7f6c90067200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-07T07:33:23.182 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Mar 07 07:33:23 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-y[37956]: 2024-03-07T07:33:23.181+0000 7f6c90067200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-07T07:33:23.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:23.283+0000 7f707de96700 1 -- 172.21.15.160:0/2190322613 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70781033a0 msgr2=0x7f7078103800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:33:23.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:23.283+0000 7f707de96700 1 --2- 172.21.15.160:0/2190322613 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70781033a0 0x7f7078103800 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f70600099c0 tx=0x7f7060009cd0 comp rx=0 tx=0).stop 2024-03-07T07:33:23.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:23.283+0000 7f707de96700 1 -- 172.21.15.160:0/2190322613 shutdown_connections 2024-03-07T07:33:23.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:23.283+0000 7f707de96700 1 --2- 172.21.15.160:0/2190322613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7078103d40 0x7f707810a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:33:23.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:23.283+0000 7f707de96700 1 --2- 172.21.15.160:0/2190322613 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70781021a0 0x7f70781025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:33:23.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:23.283+0000 7f707de96700 1 --2- 172.21.15.160:0/2190322613 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70781033a0 0x7f7078103800 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:33:23.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:23.283+0000 7f707de96700 1 -- 172.21.15.160:0/2190322613 >> 172.21.15.160:0/2190322613 conn(0x7f70780fd760 msgr2=0x7f70780ffb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:33:23.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:23.283+0000 7f707de96700 1 -- 172.21.15.160:0/2190322613 shutdown_connections 2024-03-07T07:33:23.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:23.283+0000 7f707de96700 1 -- 172.21.15.160:0/2190322613 wait complete. 2024-03-07T07:33:23.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:23.284+0000 7f707de96700 1 Processor -- start 2024-03-07T07:33:23.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:23.284+0000 7f707de96700 1 -- start start 2024-03-07T07:33:23.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:23.284+0000 7f707de96700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70781021a0 0x7f7078198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:33:23.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:23.284+0000 7f707de96700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70781033a0 0x7f70781988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:33:23.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:23.284+0000 7f707de96700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7078103d40 0x7f707819f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:33:23.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:23.284+0000 7f707de96700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707810ddb0 con 0x7f7078103d40 2024-03-07T07:33:23.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:23.284+0000 7f707de96700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707810dc30 con 0x7f70781021a0 2024-03-07T07:33:23.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:23.284+0000 7f707de96700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707810df30 con 0x7f70781033a0 2024-03-07T07:33:23.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:23.284+0000 7f7077fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7078103d40 0x7f707819f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:33:23.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:23.284+0000 7f7077fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7078103d40 0x7f707819f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:38400/0 (socket says 172.21.15.160:38400) 2024-03-07T07:33:23.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:23.284+0000 7f7077fff700 1 -- 172.21.15.160:0/324009877 learned_addr learned my addr 172.21.15.160:0/324009877 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:33:23.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:23.284+0000 7f706effd700 1 --2- 172.21.15.160:0/324009877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70781033a0 0x7f70781988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:33:23.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:23.284+0000 7f70777fe700 1 --2- 172.21.15.160:0/324009877 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70781021a0 0x7f7078198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:33:23.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:23.284+0000 7f7077fff700 1 -- 172.21.15.160:0/324009877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70781033a0 msgr2=0x7f70781988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:33:23.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:23.284+0000 7f7077fff700 1 --2- 172.21.15.160:0/324009877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70781033a0 0x7f70781988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:33:23.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:23.284+0000 7f7077fff700 1 -- 172.21.15.160:0/324009877 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70781021a0 msgr2=0x7f7078198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:33:23.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:23.284+0000 7f7077fff700 1 --2- 172.21.15.160:0/324009877 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70781021a0 0x7f7078198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:33:23.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:23.284+0000 7f7077fff700 1 -- 172.21.15.160:0/324009877 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f707819fff0 con 0x7f7078103d40 2024-03-07T07:33:23.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:23.285+0000 7f7077fff700 1 --2- 172.21.15.160:0/324009877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7078103d40 0x7f707819f950 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7070007f10 tx=0x7f707000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:33:23.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:23.285+0000 7f70757fa700 1 -- 172.21.15.160:0/324009877 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707000dd20 con 0x7f7078103d40 2024-03-07T07:33:23.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:23.285+0000 7f70757fa700 1 -- 172.21.15.160:0/324009877 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7070004020 con 0x7f7078103d40 2024-03-07T07:33:23.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:23.285+0000 7f707de96700 1 -- 172.21.15.160:0/324009877 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70780743e0 con 0x7f7078103d40 2024-03-07T07:33:23.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:23.285+0000 7f70757fa700 1 -- 172.21.15.160:0/324009877 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7070015660 con 0x7f7078103d40 2024-03-07T07:33:23.288 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:23.285+0000 7f707de96700 1 -- 172.21.15.160:0/324009877 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7078074920 con 0x7f7078103d40 2024-03-07T07:33:23.288 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:23.286+0000 7f70757fa700 1 -- 172.21.15.160:0/324009877 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7f7070015800 con 0x7f7078103d40 2024-03-07T07:33:23.288 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:23.286+0000 7f70757fa700 1 --2- 172.21.15.160:0/324009877 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f7064038e80 0x7f706403b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:33:23.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:23.286+0000 7f70757fa700 1 -- 172.21.15.160:0/324009877 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f707004b730 con 0x7f7078103d40 2024-03-07T07:33:23.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:23.286+0000 7f70777fe700 1 --2- 172.21.15.160:0/324009877 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f7064038e80 0x7f706403b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:33:23.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:23.286+0000 7f707de96700 1 -- 172.21.15.160:0/324009877 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7078068e10 con 0x7f7078103d40 2024-03-07T07:33:23.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:23.286+0000 7f70777fe700 1 --2- 172.21.15.160:0/324009877 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f7064038e80 0x7f706403b340 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7078103200 tx=0x7f706800f820 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:33:23.291 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:23.291+0000 7f70757fa700 1 -- 172.21.15.160:0/324009877 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7070010070 con 0x7f7078103d40 2024-03-07T07:33:23.318 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Mar 07 07:33:23 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-z[38341]: 2024-03-07T07:33:23.059+0000 7f71827fa200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-07T07:33:23.319 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:23 smithi178 ceph-mon[36650]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:23.319 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:23 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:23.319 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:23 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:23.319 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:23 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:23.319 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:23 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:23.319 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:23 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:33:23.319 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:23 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:33:23.319 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:23 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:23.319 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:23 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-07T07:33:23.319 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:23 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-07T07:33:23.319 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:23 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:33:23.319 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:23 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:23.392 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:23 smithi160 ceph-mon[31963]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:23.393 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:23 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:23.393 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:23 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:23.393 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:23 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:23.393 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:23 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:23.393 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:23 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:33:23.393 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:23 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:33:23.393 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:23 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:23.393 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:23 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-07T07:33:23.393 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:23 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-07T07:33:23.393 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:23 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:33:23.393 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:23 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:23.504 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:23.503+0000 7f707de96700 1 -- 172.21.15.160:0/324009877 --> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f707802ddb0 con 0x7f7064038e80 2024-03-07T07:33:23.608 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Mar 07 07:33:23 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-y[37956]: 2024-03-07T07:33:23.361+0000 7f6c90067200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-07T07:33:23.608 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:23 smithi177 ceph-mon[36578]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:23.608 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:23 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:23.608 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:23 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:23.608 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:23 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:23.608 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:23 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:23.608 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:23 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:33:23.608 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:23 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:33:23.608 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:23 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:23.609 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:23 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-07T07:33:23.609 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:23 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-07T07:33:23.609 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:23 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:33:23.609 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:23 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:23.806 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Mar 07 07:33:23 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-z[38341]: 2024-03-07T07:33:23.318+0000 7f71827fa200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-07T07:33:23.806 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Mar 07 07:33:23 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-z[38341]: 2024-03-07T07:33:23.547+0000 7f71827fa200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-07T07:33:24.543 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:24 smithi160 ceph-mon[31963]: Reconfiguring mgr.x (unknown last config time)... 2024-03-07T07:33:24.544 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:24 smithi160 ceph-mon[31963]: Reconfiguring daemon mgr.x on smithi160 2024-03-07T07:33:24.544 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:24 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-07T07:33:24.544 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:24 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-07T07:33:24.544 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:24 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:33:24.556 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Mar 07 07:33:24 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-z[38341]: 2024-03-07T07:33:24.326+0000 7f71827fa200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-07T07:33:24.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:24 smithi178 ceph-mon[36650]: Reconfiguring mgr.x (unknown last config time)... 2024-03-07T07:33:24.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:24 smithi178 ceph-mon[36650]: Reconfiguring daemon mgr.x on smithi160 2024-03-07T07:33:24.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:24 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-07T07:33:24.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:24 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-07T07:33:24.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:24 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:33:24.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:24 smithi177 ceph-mon[36578]: Reconfiguring mgr.x (unknown last config time)... 2024-03-07T07:33:24.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:24 smithi177 ceph-mon[36578]: Reconfiguring daemon mgr.x on smithi160 2024-03-07T07:33:24.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:24 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-07T07:33:24.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:24 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-07T07:33:24.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:24 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:33:24.607 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Mar 07 07:33:24 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-y[37956]: 2024-03-07T07:33:24.325+0000 7f6c90067200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-07T07:33:24.607 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Mar 07 07:33:24 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-y[37956]: 2024-03-07T07:33:24.453+0000 7f6c90067200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-07T07:33:25.071 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Mar 07 07:33:24 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-y[37956]: 2024-03-07T07:33:24.789+0000 7f6c90067200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-07T07:33:25.071 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Mar 07 07:33:24 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-y[37956]: 2024-03-07T07:33:24.878+0000 7f6c90067200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-07T07:33:25.071 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Mar 07 07:33:25 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-y[37956]: 2024-03-07T07:33:25.070+0000 7f6c90067200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-07T07:33:25.192 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Mar 07 07:33:24 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-z[38341]: 2024-03-07T07:33:24.888+0000 7f71827fa200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-07T07:33:25.192 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Mar 07 07:33:24 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-z[38341]: 2024-03-07T07:33:24.983+0000 7f71827fa200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-07T07:33:25.193 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Mar 07 07:33:25 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-z[38341]: 2024-03-07T07:33:25.192+0000 7f71827fa200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-07T07:33:25.356 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Mar 07 07:33:25 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-y[37956]: 2024-03-07T07:33:25.252+0000 7f6c90067200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-07T07:33:25.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:25 smithi177 ceph-mon[36578]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:33:25.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:25 smithi177 ceph-mon[36578]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:25.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:25 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:25.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:25 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:25.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:25 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:33:25.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:25 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:33:25.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:25 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:33:25.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:25 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:25.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:25 smithi177 ceph-mon[36578]: Standby manager daemon z started 2024-03-07T07:33:25.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:25 smithi177 ceph-mon[36578]: from='mgr.? 172.21.15.178:0/3733232346' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-07T07:33:25.358 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:25 smithi177 ceph-mon[36578]: from='mgr.? 172.21.15.178:0/3733232346' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-07T07:33:25.358 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:25 smithi177 ceph-mon[36578]: from='mgr.? 172.21.15.178:0/3733232346' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-07T07:33:25.358 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:25 smithi177 ceph-mon[36578]: from='mgr.? 172.21.15.178:0/3733232346' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-07T07:33:25.493 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:25.492+0000 7f70757fa700 1 -- 172.21.15.160:0/324009877 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mgrmap(e 14) v1 ==== 91341+0+0 (secure 0 0 0) 0x7f70700148c0 con 0x7f7078103d40 2024-03-07T07:33:25.493 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:25 smithi160 ceph-mon[31963]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:33:25.493 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:25 smithi160 ceph-mon[31963]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:25.493 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:25 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:25.493 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:25 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:25.494 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:25 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:33:25.494 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:25 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:33:25.494 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:25 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:33:25.494 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:25 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:25.494 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:25 smithi160 ceph-mon[31963]: Standby manager daemon z started 2024-03-07T07:33:25.494 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:25 smithi160 ceph-mon[31963]: from='mgr.? 172.21.15.178:0/3733232346' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-07T07:33:25.494 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:25 smithi160 ceph-mon[31963]: from='mgr.? 172.21.15.178:0/3733232346' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-07T07:33:25.494 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:25 smithi160 ceph-mon[31963]: from='mgr.? 172.21.15.178:0/3733232346' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-07T07:33:25.494 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:25 smithi160 ceph-mon[31963]: from='mgr.? 172.21.15.178:0/3733232346' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-07T07:33:25.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:25 smithi178 ceph-mon[36650]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:33:25.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:25 smithi178 ceph-mon[36650]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:25.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:25 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:25.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:25 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:25.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:25 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:33:25.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:25 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:33:25.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:25 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:33:25.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:25 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:25.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:25 smithi178 ceph-mon[36650]: Standby manager daemon z started 2024-03-07T07:33:25.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:25 smithi178 ceph-mon[36650]: from='mgr.? 172.21.15.178:0/3733232346' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-07T07:33:25.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:25 smithi178 ceph-mon[36650]: from='mgr.? 172.21.15.178:0/3733232346' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-07T07:33:25.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:25 smithi178 ceph-mon[36650]: from='mgr.? 172.21.15.178:0/3733232346' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-07T07:33:25.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:25 smithi178 ceph-mon[36650]: from='mgr.? 172.21.15.178:0/3733232346' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-07T07:33:25.796 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Mar 07 07:33:25 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-y[37956]: 2024-03-07T07:33:25.535+0000 7f6c90067200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-07T07:33:25.796 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Mar 07 07:33:25 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-y[37956]: 2024-03-07T07:33:25.796+0000 7f6c90067200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-07T07:33:26.106 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Mar 07 07:33:26 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-y[37956]: 2024-03-07T07:33:26.028+0000 7f6c90067200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-07T07:33:26.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:26 smithi178 ceph-mon[36650]: mgrmap e14: x(active, since 77s), standbys: z 2024-03-07T07:33:26.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:26 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-07T07:33:26.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:26 smithi178 ceph-mon[36650]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:26.826 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:26 smithi177 ceph-mon[36578]: mgrmap e14: x(active, since 77s), standbys: z 2024-03-07T07:33:26.826 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:26 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-07T07:33:26.826 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:26 smithi177 ceph-mon[36578]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:26.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:26 smithi160 ceph-mon[31963]: mgrmap e14: x(active, since 77s), standbys: z 2024-03-07T07:33:26.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:26 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-07T07:33:26.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:26 smithi160 ceph-mon[31963]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:27.106 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Mar 07 07:33:26 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-y[37956]: 2024-03-07T07:33:26.825+0000 7f6c90067200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-07T07:33:27.707 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:27 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/1075322087' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bee66c67-fb6e-4717-9685-ca62587d5f90"}]: dispatch 2024-03-07T07:33:27.708 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:27 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/1075322087' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bee66c67-fb6e-4717-9685-ca62587d5f90"}]': finished 2024-03-07T07:33:27.708 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:27 smithi177 ceph-mon[36578]: osdmap e5: 1 total, 0 up, 1 in 2024-03-07T07:33:27.708 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:27 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-07T07:33:27.708 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:27 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/1266387279' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-07T07:33:27.708 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Mar 07 07:33:27 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-y[37956]: 2024-03-07T07:33:27.392+0000 7f6c90067200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-07T07:33:27.708 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Mar 07 07:33:27 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-y[37956]: 2024-03-07T07:33:27.489+0000 7f6c90067200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-07T07:33:27.708 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Mar 07 07:33:27 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-y[37956]: 2024-03-07T07:33:27.704+0000 7f6c90067200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-07T07:33:27.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:27 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/1075322087' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bee66c67-fb6e-4717-9685-ca62587d5f90"}]: dispatch 2024-03-07T07:33:27.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:27 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/1075322087' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bee66c67-fb6e-4717-9685-ca62587d5f90"}]': finished 2024-03-07T07:33:27.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:27 smithi178 ceph-mon[36650]: osdmap e5: 1 total, 0 up, 1 in 2024-03-07T07:33:27.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:27 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-07T07:33:27.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:27 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/1266387279' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-07T07:33:27.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:27 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/1075322087' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bee66c67-fb6e-4717-9685-ca62587d5f90"}]: dispatch 2024-03-07T07:33:27.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:27 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/1075322087' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bee66c67-fb6e-4717-9685-ca62587d5f90"}]': finished 2024-03-07T07:33:27.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:27 smithi160 ceph-mon[31963]: osdmap e5: 1 total, 0 up, 1 in 2024-03-07T07:33:27.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:27 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-07T07:33:27.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:27 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/1266387279' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-07T07:33:28.506 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:28.505+0000 7f70757fa700 1 -- 172.21.15.160:0/324009877 <== mon.0 v2:172.21.15.160:3300/0 8 ==== mgrmap(e 15) v1 ==== 136698+0+0 (secure 0 0 0) 0x7f70700170d0 con 0x7f7078103d40 2024-03-07T07:33:28.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:28 smithi178 ceph-mon[36650]: Standby manager daemon y started 2024-03-07T07:33:28.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:28 smithi178 ceph-mon[36650]: from='mgr.? 172.21.15.177:0/3539139359' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-07T07:33:28.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:28 smithi178 ceph-mon[36650]: from='mgr.? 172.21.15.177:0/3539139359' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-07T07:33:28.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:28 smithi178 ceph-mon[36650]: from='mgr.? 172.21.15.177:0/3539139359' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-07T07:33:28.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:28 smithi178 ceph-mon[36650]: from='mgr.? 172.21.15.177:0/3539139359' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-07T07:33:28.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:28 smithi178 ceph-mon[36650]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:28.829 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:28 smithi160 ceph-mon[31963]: Standby manager daemon y started 2024-03-07T07:33:28.829 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:28 smithi160 ceph-mon[31963]: from='mgr.? 172.21.15.177:0/3539139359' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-07T07:33:28.829 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:28 smithi160 ceph-mon[31963]: from='mgr.? 172.21.15.177:0/3539139359' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-07T07:33:28.829 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:28 smithi160 ceph-mon[31963]: from='mgr.? 172.21.15.177:0/3539139359' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-07T07:33:28.829 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:28 smithi160 ceph-mon[31963]: from='mgr.? 172.21.15.177:0/3539139359' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-07T07:33:28.829 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:28 smithi160 ceph-mon[31963]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:28.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:28 smithi177 ceph-mon[36578]: Standby manager daemon y started 2024-03-07T07:33:28.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:28 smithi177 ceph-mon[36578]: from='mgr.? 172.21.15.177:0/3539139359' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-07T07:33:28.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:28 smithi177 ceph-mon[36578]: from='mgr.? 172.21.15.177:0/3539139359' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-07T07:33:28.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:28 smithi177 ceph-mon[36578]: from='mgr.? 172.21.15.177:0/3539139359' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-07T07:33:28.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:28 smithi177 ceph-mon[36578]: from='mgr.? 172.21.15.177:0/3539139359' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-07T07:33:28.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:28 smithi177 ceph-mon[36578]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:29.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:29 smithi178 ceph-mon[36650]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-03-07T07:33:29.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:29 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-07T07:33:29.846 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:29 smithi160 ceph-mon[31963]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-03-07T07:33:29.846 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:29 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-07T07:33:29.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:29 smithi177 ceph-mon[36578]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-03-07T07:33:29.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:29 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-07T07:33:30.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:30 smithi178 ceph-mon[36650]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:30.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:30 smithi177 ceph-mon[36578]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:30.882 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:30 smithi160 ceph-mon[31963]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:33.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:33 smithi160 ceph-mon[31963]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:33.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:33 smithi178 ceph-mon[36650]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:33.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:33 smithi177 ceph-mon[36578]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:34.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:34 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-07T07:33:34.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:34 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:33:34.581 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:34 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-07T07:33:34.581 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:34 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:33:34.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:34 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-07T07:33:34.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:34 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:33:35.427 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:35 smithi160 ceph-mon[31963]: Deploying daemon osd.0 on smithi160 2024-03-07T07:33:35.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:35 smithi160 ceph-mon[31963]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:35.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:35 smithi178 ceph-mon[36650]: Deploying daemon osd.0 on smithi160 2024-03-07T07:33:35.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:35 smithi178 ceph-mon[36650]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:35.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:35 smithi177 ceph-mon[36578]: Deploying daemon osd.0 on smithi160 2024-03-07T07:33:35.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:35 smithi177 ceph-mon[36578]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:37.529 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:37 smithi160 ceph-mon[31963]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:37.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:37 smithi178 ceph-mon[36650]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:37.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:37 smithi177 ceph-mon[36578]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:39.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:39 smithi178 ceph-mon[36650]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:39.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:39 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:39.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:39 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:39.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:39 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:33:39.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:39 smithi177 ceph-mon[36578]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:39.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:39 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:39.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:39 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:39.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:39 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:33:39.615 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:39 smithi160 ceph-mon[31963]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:39.615 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:39 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:39.615 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:39 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:39.615 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:39 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:33:41.405 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:41 smithi160 ceph-mon[31963]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:41.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:41 smithi178 ceph-mon[36650]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:41.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:41 smithi177 ceph-mon[36578]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:42.663 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:42.662+0000 7f70757fa700 1 -- 172.21.15.160:0/324009877 <== mgr.14152 v2:172.21.15.160:6800/419630619 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f707802ddb0 con 0x7f7064038e80 2024-03-07T07:33:42.663 INFO:teuthology.orchestra.run.smithi160.stdout:Created osd(s) 0 on host 'smithi160' 2024-03-07T07:33:42.665 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:42.664+0000 7f707de96700 1 -- 172.21.15.160:0/324009877 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f7064038e80 msgr2=0x7f706403b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:33:42.665 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:42.664+0000 7f707de96700 1 --2- 172.21.15.160:0/324009877 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f7064038e80 0x7f706403b340 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7078103200 tx=0x7f706800f820 comp rx=0 tx=0).stop 2024-03-07T07:33:42.665 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:42.664+0000 7f707de96700 1 -- 172.21.15.160:0/324009877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7078103d40 msgr2=0x7f707819f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:33:42.665 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:42.664+0000 7f707de96700 1 --2- 172.21.15.160:0/324009877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7078103d40 0x7f707819f950 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7070007f10 tx=0x7f707000d3b0 comp rx=0 tx=0).stop 2024-03-07T07:33:42.665 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:42.665+0000 7f707de96700 1 -- 172.21.15.160:0/324009877 shutdown_connections 2024-03-07T07:33:42.666 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:42.665+0000 7f707de96700 1 --2- 172.21.15.160:0/324009877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7078103d40 0x7f707819f950 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:33:42.666 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:42.665+0000 7f707de96700 1 --2- 172.21.15.160:0/324009877 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70781021a0 0x7f7078198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:33:42.666 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:42.665+0000 7f707de96700 1 --2- 172.21.15.160:0/324009877 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f7064038e80 0x7f706403b340 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:33:42.666 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:42.665+0000 7f707de96700 1 --2- 172.21.15.160:0/324009877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70781033a0 0x7f70781988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:33:42.666 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:42.665+0000 7f707de96700 1 -- 172.21.15.160:0/324009877 >> 172.21.15.160:0/324009877 conn(0x7f70780fd760 msgr2=0x7f70781066a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:33:42.666 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:42.665+0000 7f707de96700 1 -- 172.21.15.160:0/324009877 shutdown_connections 2024-03-07T07:33:42.666 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:33:42.665+0000 7f707de96700 1 -- 172.21.15.160:0/324009877 wait complete. 2024-03-07T07:33:43.304 DEBUG:teuthology.orchestra.run.smithi160:osd.0> sudo journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@osd.0.service 2024-03-07T07:33:43.347 INFO:tasks.cephadm:Deploying osd.1 on smithi177 with /dev/vg_nvme/lv_4... 2024-03-07T07:33:43.347 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-07T07:33:43.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:43 smithi178 ceph-mon[36650]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:43.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:43 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:43.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:43 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:43.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:43 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:43.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:43 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:43.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:43 smithi178 ceph-mon[36650]: from='osd.0 [v2:172.21.15.160:6802/1368482926,v1:172.21.15.160:6803/1368482926]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-07T07:33:43.588 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.b/config 2024-03-07T07:33:43.594 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:43 smithi160 ceph-mon[31963]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:43.595 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:43 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:43.595 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:43 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:43.595 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:43 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:43.595 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:43 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:43.595 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:43 smithi160 ceph-mon[31963]: from='osd.0 [v2:172.21.15.160:6802/1368482926,v1:172.21.15.160:6803/1368482926]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-07T07:33:43.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:43 smithi177 ceph-mon[36578]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:43.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:43 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:43.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:43 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:43.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:43 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:43.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:43 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:43.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:43 smithi177 ceph-mon[36578]: from='osd.0 [v2:172.21.15.160:6802/1368482926,v1:172.21.15.160:6803/1368482926]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-07T07:33:44.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:44 smithi177 ceph-mon[36578]: from='osd.0 [v2:172.21.15.160:6802/1368482926,v1:172.21.15.160:6803/1368482926]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-07T07:33:44.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:44 smithi177 ceph-mon[36578]: osdmap e6: 1 total, 0 up, 1 in 2024-03-07T07:33:44.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:44 smithi177 ceph-mon[36578]: from='osd.0 [v2:172.21.15.160:6802/1368482926,v1:172.21.15.160:6803/1368482926]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-03-07T07:33:44.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:44 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-07T07:33:44.858 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:44 smithi177 ceph-mon[36578]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:45.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:44 smithi178 ceph-mon[36650]: from='osd.0 [v2:172.21.15.160:6802/1368482926,v1:172.21.15.160:6803/1368482926]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-07T07:33:45.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:44 smithi178 ceph-mon[36650]: osdmap e6: 1 total, 0 up, 1 in 2024-03-07T07:33:45.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:44 smithi178 ceph-mon[36650]: from='osd.0 [v2:172.21.15.160:6802/1368482926,v1:172.21.15.160:6803/1368482926]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-03-07T07:33:45.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:44 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-07T07:33:45.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:44 smithi178 ceph-mon[36650]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:45.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:44 smithi160 ceph-mon[31963]: from='osd.0 [v2:172.21.15.160:6802/1368482926,v1:172.21.15.160:6803/1368482926]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-07T07:33:45.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:44 smithi160 ceph-mon[31963]: osdmap e6: 1 total, 0 up, 1 in 2024-03-07T07:33:45.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:44 smithi160 ceph-mon[31963]: from='osd.0 [v2:172.21.15.160:6802/1368482926,v1:172.21.15.160:6803/1368482926]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-03-07T07:33:45.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:44 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-07T07:33:45.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:44 smithi160 ceph-mon[31963]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:45.179 INFO:journalctl@ceph.osd.0.smithi160.stdout:Mar 07 07:33:44 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-0[46912]: 2024-03-07T07:33:44.771+0000 7f3054fdb700 -1 osd.0 0 waiting for initial osdmap 2024-03-07T07:33:45.179 INFO:journalctl@ceph.osd.0.smithi160.stdout:Mar 07 07:33:44 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-0[46912]: 2024-03-07T07:33:44.774+0000 7f3050dce700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-07T07:33:45.444 INFO:teuthology.orchestra.run.smithi177.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-07T07:33:45.444 INFO:teuthology.orchestra.run.smithi177.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-07T07:33:45.444 INFO:teuthology.orchestra.run.smithi177.stderr: stderr: 10+0 records in 2024-03-07T07:33:45.444 INFO:teuthology.orchestra.run.smithi177.stderr:10+0 records out 2024-03-07T07:33:45.445 INFO:teuthology.orchestra.run.smithi177.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0122455 s, 856 MB/s 2024-03-07T07:33:45.445 INFO:teuthology.orchestra.run.smithi177.stderr:--> Zapping successful for: 2024-03-07T07:33:46.053 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:45 smithi177 ceph-mon[36578]: from='osd.0 [v2:172.21.15.160:6802/1368482926,v1:172.21.15.160:6803/1368482926]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-03-07T07:33:46.053 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:45 smithi177 ceph-mon[36578]: osdmap e7: 1 total, 0 up, 1 in 2024-03-07T07:33:46.054 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:45 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-07T07:33:46.054 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:45 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-07T07:33:46.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:45 smithi178 ceph-mon[36650]: from='osd.0 [v2:172.21.15.160:6802/1368482926,v1:172.21.15.160:6803/1368482926]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-03-07T07:33:46.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:45 smithi178 ceph-mon[36650]: osdmap e7: 1 total, 0 up, 1 in 2024-03-07T07:33:46.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:45 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-07T07:33:46.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:45 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-07T07:33:46.081 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi177:vg_nvme/lv_4 2024-03-07T07:33:46.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:45 smithi160 ceph-mon[31963]: from='osd.0 [v2:172.21.15.160:6802/1368482926,v1:172.21.15.160:6803/1368482926]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-03-07T07:33:46.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:45 smithi160 ceph-mon[31963]: osdmap e7: 1 total, 0 up, 1 in 2024-03-07T07:33:46.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:45 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-07T07:33:46.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:45 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-07T07:33:46.329 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.b/config 2024-03-07T07:33:47.042 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:46 smithi177 ceph-mon[36578]: purged_snaps scrub starts 2024-03-07T07:33:47.043 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:46 smithi177 ceph-mon[36578]: purged_snaps scrub ok 2024-03-07T07:33:47.043 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:46 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-07T07:33:47.043 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:46 smithi177 ceph-mon[36578]: osd.0 [v2:172.21.15.160:6802/1368482926,v1:172.21.15.160:6803/1368482926] boot 2024-03-07T07:33:47.043 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:46 smithi177 ceph-mon[36578]: osdmap e8: 1 total, 1 up, 1 in 2024-03-07T07:33:47.043 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:46 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-07T07:33:47.043 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:46 smithi177 ceph-mon[36578]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:47.043 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:46 smithi177 ceph-mon[36578]: Detected new or changed devices on smithi160 2024-03-07T07:33:47.043 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:46 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:47.043 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:46 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:47.043 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:46 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:33:47.043 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:46 smithi177 ceph-mon[36578]: Adjusting osd_memory_target on smithi160 to 17088M 2024-03-07T07:33:47.043 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:46 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:47.044 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:46 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:33:47.044 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:46 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:33:47.044 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:46 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:47.044 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:46 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:33:47.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:46 smithi178 ceph-mon[36650]: purged_snaps scrub starts 2024-03-07T07:33:47.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:46 smithi178 ceph-mon[36650]: purged_snaps scrub ok 2024-03-07T07:33:47.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:46 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-07T07:33:47.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:46 smithi178 ceph-mon[36650]: osd.0 [v2:172.21.15.160:6802/1368482926,v1:172.21.15.160:6803/1368482926] boot 2024-03-07T07:33:47.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:46 smithi178 ceph-mon[36650]: osdmap e8: 1 total, 1 up, 1 in 2024-03-07T07:33:47.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:46 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-07T07:33:47.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:46 smithi178 ceph-mon[36650]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:47.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:46 smithi178 ceph-mon[36650]: Detected new or changed devices on smithi160 2024-03-07T07:33:47.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:46 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:47.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:46 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:47.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:46 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:33:47.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:46 smithi178 ceph-mon[36650]: Adjusting osd_memory_target on smithi160 to 17088M 2024-03-07T07:33:47.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:46 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:47.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:46 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:33:47.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:46 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:33:47.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:46 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:47.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:46 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:33:47.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:46 smithi160 ceph-mon[31963]: purged_snaps scrub starts 2024-03-07T07:33:47.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:46 smithi160 ceph-mon[31963]: purged_snaps scrub ok 2024-03-07T07:33:47.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:46 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-07T07:33:47.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:46 smithi160 ceph-mon[31963]: osd.0 [v2:172.21.15.160:6802/1368482926,v1:172.21.15.160:6803/1368482926] boot 2024-03-07T07:33:47.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:46 smithi160 ceph-mon[31963]: osdmap e8: 1 total, 1 up, 1 in 2024-03-07T07:33:47.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:46 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-07T07:33:47.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:46 smithi160 ceph-mon[31963]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-07T07:33:47.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:46 smithi160 ceph-mon[31963]: Detected new or changed devices on smithi160 2024-03-07T07:33:47.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:46 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:47.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:46 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:47.179 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:46 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:33:47.179 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:46 smithi160 ceph-mon[31963]: Adjusting osd_memory_target on smithi160 to 17088M 2024-03-07T07:33:47.179 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:46 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:47.179 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:46 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:33:47.179 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:46 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:33:47.179 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:46 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:47.179 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:46 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:33:47.826 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:33:47.824+0000 7f288aed4700 1 -- 172.21.15.177:0/3623801239 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28841022f0 msgr2=0x7f2884102750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:33:47.826 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:33:47.824+0000 7f288aed4700 1 --2- 172.21.15.177:0/3623801239 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28841022f0 0x7f2884102750 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f28700099c0 tx=0x7f2870009cd0 comp rx=0 tx=0).stop 2024-03-07T07:33:47.827 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:33:47.824+0000 7f288aed4700 1 -- 172.21.15.177:0/3623801239 shutdown_connections 2024-03-07T07:33:47.827 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:33:47.824+0000 7f288aed4700 1 --2- 172.21.15.177:0/3623801239 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2884102c90 0x7f2884109540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:33:47.827 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:33:47.824+0000 7f288aed4700 1 --2- 172.21.15.177:0/3623801239 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28841010f0 0x7f2884101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:33:47.827 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:33:47.824+0000 7f288aed4700 1 --2- 172.21.15.177:0/3623801239 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28841022f0 0x7f2884102750 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:33:47.827 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:33:47.824+0000 7f288aed4700 1 -- 172.21.15.177:0/3623801239 >> 172.21.15.177:0/3623801239 conn(0x7f28840fc6b0 msgr2=0x7f28840fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:33:47.827 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:33:47.824+0000 7f288aed4700 1 -- 172.21.15.177:0/3623801239 shutdown_connections 2024-03-07T07:33:47.827 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:33:47.824+0000 7f288aed4700 1 -- 172.21.15.177:0/3623801239 wait complete. 2024-03-07T07:33:47.827 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:33:47.825+0000 7f288aed4700 1 Processor -- start 2024-03-07T07:33:47.827 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:33:47.825+0000 7f288aed4700 1 -- start start 2024-03-07T07:33:47.827 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:33:47.825+0000 7f288aed4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28841010f0 0x7f288406b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:33:47.827 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:33:47.825+0000 7f288aed4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28841022f0 0x7f288419d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:33:47.827 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:33:47.825+0000 7f288aed4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2884102c90 0x7f288419f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:33:47.827 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:33:47.825+0000 7f288aed4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288410cc50 con 0x7f28841010f0 2024-03-07T07:33:47.828 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:33:47.825+0000 7f288aed4700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288410cad0 con 0x7f28841022f0 2024-03-07T07:33:47.828 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:33:47.825+0000 7f288aed4700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288410cdd0 con 0x7f2884102c90 2024-03-07T07:33:47.828 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:33:47.825+0000 7f2883fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28841022f0 0x7f288419d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:33:47.828 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:33:47.825+0000 7f2883fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28841022f0 0x7f288419d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:56132/0 (socket says 172.21.15.177:56132) 2024-03-07T07:33:47.828 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:33:47.825+0000 7f2883fff700 1 -- 172.21.15.177:0/1232837407 learned_addr learned my addr 172.21.15.177:0/1232837407 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-03-07T07:33:47.828 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:33:47.825+0000 7f2889471700 1 --2- 172.21.15.177:0/1232837407 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2884102c90 0x7f288419f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:33:47.828 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:33:47.825+0000 7f2888c70700 1 --2- 172.21.15.177:0/1232837407 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28841010f0 0x7f288406b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:33:47.828 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:33:47.825+0000 7f2883fff700 1 -- 172.21.15.177:0/1232837407 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2884102c90 msgr2=0x7f288419f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:33:47.828 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:33:47.825+0000 7f2883fff700 1 --2- 172.21.15.177:0/1232837407 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2884102c90 0x7f288419f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:33:47.828 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:33:47.826+0000 7f2883fff700 1 -- 172.21.15.177:0/1232837407 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28841010f0 msgr2=0x7f288406b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:33:47.828 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:33:47.826+0000 7f2883fff700 1 --2- 172.21.15.177:0/1232837407 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28841010f0 0x7f288406b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:33:47.828 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:33:47.826+0000 7f2883fff700 1 -- 172.21.15.177:0/1232837407 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f288419ffb0 con 0x7f28841022f0 2024-03-07T07:33:47.828 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:33:47.826+0000 7f2883fff700 1 --2- 172.21.15.177:0/1232837407 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28841022f0 0x7f288419d0a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2870005910 tx=0x7f2870009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:33:47.829 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:33:47.826+0000 7f2881ffb700 1 -- 172.21.15.177:0/1232837407 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2870027070 con 0x7f28841022f0 2024-03-07T07:33:47.829 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:33:47.826+0000 7f288aed4700 1 -- 172.21.15.177:0/1232837407 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2884196e70 con 0x7f28841022f0 2024-03-07T07:33:47.829 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:33:47.826+0000 7f2881ffb700 1 -- 172.21.15.177:0/1232837407 <== mon.2 v2:172.21.15.177:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2870004150 con 0x7f28841022f0 2024-03-07T07:33:47.829 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:33:47.826+0000 7f2881ffb700 1 -- 172.21.15.177:0/1232837407 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287000f460 con 0x7f28841022f0 2024-03-07T07:33:47.829 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:33:47.826+0000 7f288aed4700 1 -- 172.21.15.177:0/1232837407 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28841973a0 con 0x7f28841022f0 2024-03-07T07:33:47.830 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:33:47.827+0000 7f288aed4700 1 -- 172.21.15.177:0/1232837407 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2884068e10 con 0x7f28841022f0 2024-03-07T07:33:47.830 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:33:47.828+0000 7f2881ffb700 1 -- 172.21.15.177:0/1232837407 <== mon.2 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 15) v1 ==== 136698+0+0 (secure 0 0 0) 0x7f287002a070 con 0x7f28841022f0 2024-03-07T07:33:47.830 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:33:47.828+0000 7f2881ffb700 1 --2- 172.21.15.177:0/1232837407 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f28740a1d80 0x7f28740a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:33:47.830 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:33:47.829+0000 7f2881ffb700 1 -- 172.21.15.177:0/1232837407 <== mon.2 v2:172.21.15.177:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f28700cd700 con 0x7f28841022f0 2024-03-07T07:33:47.830 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:33:47.829+0000 7f2888c70700 1 --2- 172.21.15.177:0/1232837407 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f28740a1d80 0x7f28740a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:33:47.830 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:33:47.829+0000 7f2888c70700 1 --2- 172.21.15.177:0/1232837407 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f28740a1d80 0x7f28740a4240 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2884102150 tx=0x7f2878005cf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:33:47.832 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:33:47.832+0000 7f2881ffb700 1 -- 172.21.15.177:0/1232837407 <== mon.2 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f2870031070 con 0x7f28841022f0 2024-03-07T07:33:48.032 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:33:48.031+0000 7f288aed4700 1 -- 172.21.15.177:0/1232837407 --> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f2884062590 con 0x7f28740a1d80 2024-03-07T07:33:48.032 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:47 smithi177 ceph-mon[36578]: osdmap e9: 1 total, 1 up, 1 in 2024-03-07T07:33:48.032 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:47 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:33:48.032 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:47 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:33:48.032 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:47 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:48.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:47 smithi178 ceph-mon[36650]: osdmap e9: 1 total, 1 up, 1 in 2024-03-07T07:33:48.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:47 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:33:48.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:47 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:33:48.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:47 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:48.177 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:47 smithi160 ceph-mon[31963]: osdmap e9: 1 total, 1 up, 1 in 2024-03-07T07:33:48.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:47 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:33:48.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:47 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:33:48.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:47 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:33:48.915 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:48 smithi177 ceph-mon[36578]: from='client.24152 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:33:48.915 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:48 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-07T07:33:48.915 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:48 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-07T07:33:48.915 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:48 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:33:48.916 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:48 smithi177 ceph-mon[36578]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-07T07:33:49.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:48 smithi178 ceph-mon[36650]: from='client.24152 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:33:49.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:48 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-07T07:33:49.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:48 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-07T07:33:49.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:48 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:33:49.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:48 smithi178 ceph-mon[36650]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-07T07:33:49.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:48 smithi160 ceph-mon[31963]: from='client.24152 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:33:49.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:48 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-07T07:33:49.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:48 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-07T07:33:49.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:48 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:33:49.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:48 smithi160 ceph-mon[31963]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-07T07:33:51.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:51 smithi178 ceph-mon[36650]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-07T07:33:51.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:51 smithi178 ceph-mon[36650]: from='client.? 172.21.15.177:0/582834690' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49236251-f750-424e-992b-bddf55762556"}]: dispatch 2024-03-07T07:33:51.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:51 smithi178 ceph-mon[36650]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49236251-f750-424e-992b-bddf55762556"}]: dispatch 2024-03-07T07:33:51.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:51 smithi178 ceph-mon[36650]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "49236251-f750-424e-992b-bddf55762556"}]': finished 2024-03-07T07:33:51.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:51 smithi178 ceph-mon[36650]: osdmap e10: 2 total, 1 up, 2 in 2024-03-07T07:33:51.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:51 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-07T07:33:51.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:51 smithi178 ceph-mon[36650]: from='client.? 172.21.15.177:0/3106926255' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-07T07:33:51.608 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:51 smithi177 ceph-mon[36578]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-07T07:33:51.608 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:51 smithi177 ceph-mon[36578]: from='client.? 172.21.15.177:0/582834690' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49236251-f750-424e-992b-bddf55762556"}]: dispatch 2024-03-07T07:33:51.609 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:51 smithi177 ceph-mon[36578]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49236251-f750-424e-992b-bddf55762556"}]: dispatch 2024-03-07T07:33:51.609 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:51 smithi177 ceph-mon[36578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "49236251-f750-424e-992b-bddf55762556"}]': finished 2024-03-07T07:33:51.609 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:51 smithi177 ceph-mon[36578]: osdmap e10: 2 total, 1 up, 2 in 2024-03-07T07:33:51.609 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:51 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-07T07:33:51.609 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:51 smithi177 ceph-mon[36578]: from='client.? 172.21.15.177:0/3106926255' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-07T07:33:51.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:51 smithi160 ceph-mon[31963]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-07T07:33:51.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:51 smithi160 ceph-mon[31963]: from='client.? 172.21.15.177:0/582834690' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49236251-f750-424e-992b-bddf55762556"}]: dispatch 2024-03-07T07:33:51.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:51 smithi160 ceph-mon[31963]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49236251-f750-424e-992b-bddf55762556"}]: dispatch 2024-03-07T07:33:51.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:51 smithi160 ceph-mon[31963]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "49236251-f750-424e-992b-bddf55762556"}]': finished 2024-03-07T07:33:51.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:51 smithi160 ceph-mon[31963]: osdmap e10: 2 total, 1 up, 2 in 2024-03-07T07:33:51.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:51 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-07T07:33:51.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:51 smithi160 ceph-mon[31963]: from='client.? 172.21.15.177:0/3106926255' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-07T07:33:53.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:53 smithi178 ceph-mon[36650]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-07T07:33:53.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:53 smithi177 ceph-mon[36578]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-07T07:33:53.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:53 smithi160 ceph-mon[31963]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-07T07:33:55.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:55 smithi178 ceph-mon[36650]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-07T07:33:55.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:55 smithi177 ceph-mon[36578]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-07T07:33:55.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:55 smithi160 ceph-mon[31963]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-07T07:33:57.439 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:57 smithi177 ceph-mon[36578]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-07T07:33:57.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:57 smithi178 ceph-mon[36650]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-07T07:33:57.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:57 smithi160 ceph-mon[31963]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-07T07:33:58.348 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:58 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-07T07:33:58.348 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:58 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:33:58.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:58 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-07T07:33:58.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:58 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:33:58.579 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:58 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-07T07:33:58.579 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:58 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:33:59.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:59 smithi178 ceph-mon[36650]: Deploying daemon osd.1 on smithi177 2024-03-07T07:33:59.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:33:59 smithi178 ceph-mon[36650]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-07T07:33:59.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:59 smithi177 ceph-mon[36578]: Deploying daemon osd.1 on smithi177 2024-03-07T07:33:59.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:33:59 smithi177 ceph-mon[36578]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-07T07:33:59.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:59 smithi160 ceph-mon[31963]: Deploying daemon osd.1 on smithi177 2024-03-07T07:33:59.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:33:59 smithi160 ceph-mon[31963]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-07T07:34:01.356 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:01 smithi177 ceph-mon[36578]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-07T07:34:01.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:01 smithi178 ceph-mon[36650]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-07T07:34:01.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:01 smithi160 ceph-mon[31963]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-07T07:34:03.544 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:03 smithi177 ceph-mon[36578]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-07T07:34:03.544 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:03 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:03.545 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:03 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:03.545 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:03 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:34:03.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:03 smithi178 ceph-mon[36650]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-07T07:34:03.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:03 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:03.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:03 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:03.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:03 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:34:03.679 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:03 smithi160 ceph-mon[31963]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-07T07:34:03.679 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:03 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:03.679 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:03 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:03.679 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:03 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:34:05.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:05 smithi178 ceph-mon[36650]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-07T07:34:05.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:05 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:05.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:05 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:05.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:05 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:34:05.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:05 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:34:05.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:05 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:05.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:05 smithi177 ceph-mon[36578]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-07T07:34:05.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:05 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:05.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:05 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:05.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:05 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:34:05.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:05 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:34:05.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:05 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:05.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:05 smithi160 ceph-mon[31963]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-07T07:34:05.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:05 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:05.679 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:05 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:05.679 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:05 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:34:05.679 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:05 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:34:05.679 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:05 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:06.343 INFO:teuthology.orchestra.run.smithi177.stdout:Created osd(s) 1 on host 'smithi177' 2024-03-07T07:34:06.344 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:34:06.343+0000 7f2881ffb700 1 -- 172.21.15.177:0/1232837407 <== mgr.14152 v2:172.21.15.160:6800/419630619 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2884062590 con 0x7f28740a1d80 2024-03-07T07:34:06.346 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:34:06.345+0000 7f288aed4700 1 -- 172.21.15.177:0/1232837407 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f28740a1d80 msgr2=0x7f28740a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:34:06.346 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:34:06.345+0000 7f288aed4700 1 --2- 172.21.15.177:0/1232837407 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f28740a1d80 0x7f28740a4240 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2884102150 tx=0x7f2878005cf0 comp rx=0 tx=0).stop 2024-03-07T07:34:06.346 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:34:06.345+0000 7f288aed4700 1 -- 172.21.15.177:0/1232837407 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28841022f0 msgr2=0x7f288419d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:34:06.346 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:34:06.345+0000 7f288aed4700 1 --2- 172.21.15.177:0/1232837407 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28841022f0 0x7f288419d0a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2870005910 tx=0x7f2870009580 comp rx=0 tx=0).stop 2024-03-07T07:34:06.346 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:34:06.345+0000 7f288aed4700 1 -- 172.21.15.177:0/1232837407 shutdown_connections 2024-03-07T07:34:06.346 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:34:06.345+0000 7f288aed4700 1 --2- 172.21.15.177:0/1232837407 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f28740a1d80 0x7f28740a4240 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:34:06.347 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:34:06.345+0000 7f288aed4700 1 --2- 172.21.15.177:0/1232837407 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2884102c90 0x7f288419f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:34:06.347 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:34:06.345+0000 7f288aed4700 1 --2- 172.21.15.177:0/1232837407 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28841010f0 0x7f288406b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:34:06.347 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:34:06.345+0000 7f288aed4700 1 --2- 172.21.15.177:0/1232837407 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28841022f0 0x7f288419d0a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:34:06.347 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:34:06.345+0000 7f288aed4700 1 -- 172.21.15.177:0/1232837407 >> 172.21.15.177:0/1232837407 conn(0x7f28840fc6b0 msgr2=0x7f2884105670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:34:06.347 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:34:06.345+0000 7f288aed4700 1 -- 172.21.15.177:0/1232837407 shutdown_connections 2024-03-07T07:34:06.347 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:34:06.345+0000 7f288aed4700 1 -- 172.21.15.177:0/1232837407 wait complete. 2024-03-07T07:34:06.932 DEBUG:teuthology.orchestra.run.smithi177:osd.1> sudo journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@osd.1.service 2024-03-07T07:34:06.934 INFO:tasks.cephadm:Deploying osd.2 on smithi178 with /dev/vg_nvme/lv_4... 2024-03-07T07:34:06.934 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-07T07:34:07.181 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.c/config 2024-03-07T07:34:07.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:07 smithi178 ceph-mon[36650]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-07T07:34:07.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:07 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:07.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:07 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:07.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:07 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:34:07.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:07 smithi178 ceph-mon[36650]: from='osd.1 [v2:172.21.15.177:6800/2922391884,v1:172.21.15.177:6801/2922391884]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-07T07:34:07.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:07 smithi178 ceph-mon[36650]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-07T07:34:07.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:07 smithi177 ceph-mon[36578]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-07T07:34:07.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:07 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:07.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:07 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:07.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:07 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:34:07.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:07 smithi177 ceph-mon[36578]: from='osd.1 [v2:172.21.15.177:6800/2922391884,v1:172.21.15.177:6801/2922391884]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-07T07:34:07.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:07 smithi177 ceph-mon[36578]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-07T07:34:07.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:07 smithi160 ceph-mon[31963]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-07T07:34:07.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:07 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:07.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:07 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:07.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:07 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:34:07.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:07 smithi160 ceph-mon[31963]: from='osd.1 [v2:172.21.15.177:6800/2922391884,v1:172.21.15.177:6801/2922391884]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-07T07:34:07.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:07 smithi160 ceph-mon[31963]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-07T07:34:08.399 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:08 smithi178 ceph-mon[36650]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-07T07:34:08.400 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:08 smithi178 ceph-mon[36650]: from='osd.1 [v2:172.21.15.177:6800/2922391884,v1:172.21.15.177:6801/2922391884]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-03-07T07:34:08.400 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:08 smithi178 ceph-mon[36650]: osdmap e11: 2 total, 1 up, 2 in 2024-03-07T07:34:08.400 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:08 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-07T07:34:08.400 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:08 smithi178 ceph-mon[36650]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-03-07T07:34:08.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:08 smithi177 ceph-mon[36578]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-07T07:34:08.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:08 smithi177 ceph-mon[36578]: from='osd.1 [v2:172.21.15.177:6800/2922391884,v1:172.21.15.177:6801/2922391884]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-03-07T07:34:08.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:08 smithi177 ceph-mon[36578]: osdmap e11: 2 total, 1 up, 2 in 2024-03-07T07:34:08.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:08 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-07T07:34:08.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:08 smithi177 ceph-mon[36578]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-03-07T07:34:08.608 INFO:journalctl@ceph.osd.1.smithi177.stdout:Mar 07 07:34:08 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-1[41162]: 2024-03-07T07:34:08.317+0000 7fed8538a700 -1 osd.1 0 waiting for initial osdmap 2024-03-07T07:34:08.608 INFO:journalctl@ceph.osd.1.smithi177.stdout:Mar 07 07:34:08 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-1[41162]: 2024-03-07T07:34:08.321+0000 7fed8117d700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-07T07:34:08.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:08 smithi160 ceph-mon[31963]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-07T07:34:08.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:08 smithi160 ceph-mon[31963]: from='osd.1 [v2:172.21.15.177:6800/2922391884,v1:172.21.15.177:6801/2922391884]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-03-07T07:34:08.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:08 smithi160 ceph-mon[31963]: osdmap e11: 2 total, 1 up, 2 in 2024-03-07T07:34:08.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:08 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-07T07:34:08.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:08 smithi160 ceph-mon[31963]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-03-07T07:34:08.984 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-07T07:34:08.985 INFO:teuthology.orchestra.run.smithi178.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-07T07:34:08.985 INFO:teuthology.orchestra.run.smithi178.stderr: stderr: 10+0 records in 2024-03-07T07:34:08.985 INFO:teuthology.orchestra.run.smithi178.stderr:10+0 records out 2024-03-07T07:34:08.985 INFO:teuthology.orchestra.run.smithi178.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00858377 s, 1.2 GB/s 2024-03-07T07:34:08.985 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping successful for: 2024-03-07T07:34:09.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:09 smithi178 ceph-mon[36650]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-07T07:34:09.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:09 smithi178 ceph-mon[36650]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2024-03-07T07:34:09.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:09 smithi178 ceph-mon[36650]: osdmap e12: 2 total, 1 up, 2 in 2024-03-07T07:34:09.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:09 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-07T07:34:09.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:09 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-07T07:34:09.584 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi178:vg_nvme/lv_4 2024-03-07T07:34:09.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:09 smithi177 ceph-mon[36578]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-07T07:34:09.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:09 smithi177 ceph-mon[36578]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2024-03-07T07:34:09.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:09 smithi177 ceph-mon[36578]: osdmap e12: 2 total, 1 up, 2 in 2024-03-07T07:34:09.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:09 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-07T07:34:09.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:09 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-07T07:34:09.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:09 smithi160 ceph-mon[31963]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-07T07:34:09.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:09 smithi160 ceph-mon[31963]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2024-03-07T07:34:09.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:09 smithi160 ceph-mon[31963]: osdmap e12: 2 total, 1 up, 2 in 2024-03-07T07:34:09.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:09 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-07T07:34:09.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:09 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-07T07:34:09.847 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.c/config 2024-03-07T07:34:10.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:10 smithi178 ceph-mon[36650]: purged_snaps scrub starts 2024-03-07T07:34:10.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:10 smithi178 ceph-mon[36650]: purged_snaps scrub ok 2024-03-07T07:34:10.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:10 smithi178 ceph-mon[36650]: osd.1 [v2:172.21.15.177:6800/2922391884,v1:172.21.15.177:6801/2922391884] boot 2024-03-07T07:34:10.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:10 smithi178 ceph-mon[36650]: osdmap e13: 2 total, 2 up, 2 in 2024-03-07T07:34:10.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:10 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-07T07:34:10.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:10 smithi178 ceph-mon[36650]: Detected new or changed devices on smithi177 2024-03-07T07:34:10.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:10 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:10.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:10 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:10.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:10 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:34:10.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:10 smithi178 ceph-mon[36650]: Adjusting osd_memory_target on smithi177 to 17080M 2024-03-07T07:34:10.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:10 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:10.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:10 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:34:10.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:10 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:34:10.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:10 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:10.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:10 smithi178 ceph-mon[36650]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-07T07:34:10.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:10 smithi177 ceph-mon[36578]: purged_snaps scrub starts 2024-03-07T07:34:10.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:10 smithi177 ceph-mon[36578]: purged_snaps scrub ok 2024-03-07T07:34:10.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:10 smithi177 ceph-mon[36578]: osd.1 [v2:172.21.15.177:6800/2922391884,v1:172.21.15.177:6801/2922391884] boot 2024-03-07T07:34:10.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:10 smithi177 ceph-mon[36578]: osdmap e13: 2 total, 2 up, 2 in 2024-03-07T07:34:10.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:10 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-07T07:34:10.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:10 smithi177 ceph-mon[36578]: Detected new or changed devices on smithi177 2024-03-07T07:34:10.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:10 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:10.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:10 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:10.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:10 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:34:10.608 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:10 smithi177 ceph-mon[36578]: Adjusting osd_memory_target on smithi177 to 17080M 2024-03-07T07:34:10.608 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:10 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:10.608 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:10 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:34:10.608 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:10 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:34:10.608 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:10 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:10.608 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:10 smithi177 ceph-mon[36578]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-07T07:34:10.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:10 smithi160 ceph-mon[31963]: purged_snaps scrub starts 2024-03-07T07:34:10.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:10 smithi160 ceph-mon[31963]: purged_snaps scrub ok 2024-03-07T07:34:10.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:10 smithi160 ceph-mon[31963]: osd.1 [v2:172.21.15.177:6800/2922391884,v1:172.21.15.177:6801/2922391884] boot 2024-03-07T07:34:10.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:10 smithi160 ceph-mon[31963]: osdmap e13: 2 total, 2 up, 2 in 2024-03-07T07:34:10.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:10 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-07T07:34:10.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:10 smithi160 ceph-mon[31963]: Detected new or changed devices on smithi177 2024-03-07T07:34:10.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:10 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:10.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:10 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:10.679 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:10 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:34:10.679 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:10 smithi160 ceph-mon[31963]: Adjusting osd_memory_target on smithi177 to 17080M 2024-03-07T07:34:10.679 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:10 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:10.679 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:10 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:34:10.679 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:10 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:34:10.679 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:10 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:10.679 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:10 smithi160 ceph-mon[31963]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-07T07:34:11.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:11.337+0000 7f3435b35700 1 -- 172.21.15.178:0/404773148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3430102c00 msgr2=0x7f3430104ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:34:11.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:11.337+0000 7f3435b35700 1 --2- 172.21.15.178:0/404773148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3430102c00 0x7f3430104ff0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3420009990 tx=0x7f3420009ca0 comp rx=0 tx=0).stop 2024-03-07T07:34:11.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:11.337+0000 7f3435b35700 1 -- 172.21.15.178:0/404773148 shutdown_connections 2024-03-07T07:34:11.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:11.337+0000 7f3435b35700 1 --2- 172.21.15.178:0/404773148 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3430107e60 0x7f343010a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:34:11.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:11.337+0000 7f3435b35700 1 --2- 172.21.15.178:0/404773148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3430102c00 0x7f3430104ff0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:34:11.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:11.337+0000 7f3435b35700 1 --2- 172.21.15.178:0/404773148 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3430105530 0x7f3430107920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:34:11.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:11.337+0000 7f3435b35700 1 -- 172.21.15.178:0/404773148 >> 172.21.15.178:0/404773148 conn(0x7f34300fc610 msgr2=0x7f34300fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:34:11.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:11.337+0000 7f3435b35700 1 -- 172.21.15.178:0/404773148 shutdown_connections 2024-03-07T07:34:11.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:11.338+0000 7f3435b35700 1 -- 172.21.15.178:0/404773148 wait complete. 2024-03-07T07:34:11.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:11.338+0000 7f3435b35700 1 Processor -- start 2024-03-07T07:34:11.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:11.338+0000 7f3435b35700 1 -- start start 2024-03-07T07:34:11.341 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:11.338+0000 7f3435b35700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3430102c00 0x7f3430102560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:34:11.341 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:11.338+0000 7f3435b35700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3430105530 0x7f3430100bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:34:11.341 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:11.338+0000 7f3435b35700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3430107e60 0x7f34301010f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:34:11.341 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:11.338+0000 7f3435b35700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343010dda0 con 0x7f3430107e60 2024-03-07T07:34:11.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:11.338+0000 7f3435b35700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343010dc20 con 0x7f3430105530 2024-03-07T07:34:11.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:11.338+0000 7f3435b35700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343010df20 con 0x7f3430102c00 2024-03-07T07:34:11.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:11.338+0000 7f342f7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3430102c00 0x7f3430102560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:34:11.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:11.338+0000 7f342f7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3430102c00 0x7f3430102560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:40004/0 (socket says 172.21.15.178:40004) 2024-03-07T07:34:11.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:11.339+0000 7f342f7fe700 1 -- 172.21.15.178:0/656743187 learned_addr learned my addr 172.21.15.178:0/656743187 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-03-07T07:34:11.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:11.339+0000 7f342effd700 1 --2- 172.21.15.178:0/656743187 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3430105530 0x7f3430100bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:34:11.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:11.339+0000 7f342ffff700 1 --2- 172.21.15.178:0/656743187 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3430107e60 0x7f34301010f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:34:11.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:11.339+0000 7f342f7fe700 1 -- 172.21.15.178:0/656743187 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3430105530 msgr2=0x7f3430100bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:34:11.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:11.339+0000 7f342f7fe700 1 --2- 172.21.15.178:0/656743187 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3430105530 0x7f3430100bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:34:11.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:11.339+0000 7f342f7fe700 1 -- 172.21.15.178:0/656743187 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3430107e60 msgr2=0x7f34301010f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:34:11.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:11.339+0000 7f342f7fe700 1 --2- 172.21.15.178:0/656743187 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3430107e60 0x7f34301010f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:34:11.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:11.339+0000 7f342f7fe700 1 -- 172.21.15.178:0/656743187 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34301019a0 con 0x7f3430102c00 2024-03-07T07:34:11.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:11.339+0000 7f342f7fe700 1 --2- 172.21.15.178:0/656743187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3430102c00 0x7f3430102560 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3420003890 tx=0x7f34200047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:34:11.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:11.339+0000 7f342cff9700 1 -- 172.21.15.178:0/656743187 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3420026070 con 0x7f3430102c00 2024-03-07T07:34:11.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:11.339+0000 7f342cff9700 1 -- 172.21.15.178:0/656743187 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3420004150 con 0x7f3430102c00 2024-03-07T07:34:11.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:11.339+0000 7f3435b35700 1 -- 172.21.15.178:0/656743187 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34301a24e0 con 0x7f3430102c00 2024-03-07T07:34:11.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:11.339+0000 7f342cff9700 1 -- 172.21.15.178:0/656743187 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342000fe00 con 0x7f3430102c00 2024-03-07T07:34:11.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:11.339+0000 7f3435b35700 1 -- 172.21.15.178:0/656743187 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34301a2930 con 0x7f3430102c00 2024-03-07T07:34:11.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:11.340+0000 7f3435b35700 1 -- 172.21.15.178:0/656743187 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3430068e10 con 0x7f3430102c00 2024-03-07T07:34:11.343 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:11 smithi178 ceph-mon[36650]: osdmap e14: 2 total, 2 up, 2 in 2024-03-07T07:34:11.344 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:11.342+0000 7f342cff9700 1 -- 172.21.15.178:0/656743187 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 15) v1 ==== 136698+0+0 (secure 0 0 0) 0x7f3420029070 con 0x7f3430102c00 2024-03-07T07:34:11.344 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:11.343+0000 7f342cff9700 1 --2- 172.21.15.178:0/656743187 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f34180a1d80 0x7f34180a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:34:11.344 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:11.343+0000 7f342cff9700 1 -- 172.21.15.178:0/656743187 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f34200cd110 con 0x7f3430102c00 2024-03-07T07:34:11.344 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:11.343+0000 7f342effd700 1 --2- 172.21.15.178:0/656743187 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f34180a1d80 0x7f34180a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:34:11.344 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:11.343+0000 7f342effd700 1 --2- 172.21.15.178:0/656743187 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f34180a1d80 0x7f34180a4240 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3424005950 tx=0x7f342400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:34:11.346 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:11.346+0000 7f342cff9700 1 -- 172.21.15.178:0/656743187 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3420030080 con 0x7f3430102c00 2024-03-07T07:34:11.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:11.549+0000 7f3435b35700 1 -- 172.21.15.178:0/656743187 --> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f3430062590 con 0x7f34180a1d80 2024-03-07T07:34:11.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:11 smithi177 ceph-mon[36578]: osdmap e14: 2 total, 2 up, 2 in 2024-03-07T07:34:11.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:11 smithi160 ceph-mon[31963]: osdmap e14: 2 total, 2 up, 2 in 2024-03-07T07:34:12.413 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:12 smithi178 ceph-mon[36650]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:34:12.413 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:12 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-07T07:34:12.413 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:12 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-07T07:34:12.413 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:12 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:34:12.413 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:12 smithi178 ceph-mon[36650]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-07T07:34:12.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:12 smithi177 ceph-mon[36578]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:34:12.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:12 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-07T07:34:12.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:12 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-07T07:34:12.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:12 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:34:12.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:12 smithi177 ceph-mon[36578]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-07T07:34:12.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:12 smithi160 ceph-mon[31963]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:34:12.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:12 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-07T07:34:12.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:12 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-07T07:34:12.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:12 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:34:12.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:12 smithi160 ceph-mon[31963]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-07T07:34:14.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:14 smithi177 ceph-mon[36578]: from='client.? 172.21.15.178:0/1274664720' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7a432b28-f7ee-43fe-9196-7aaeccfecf77"}]: dispatch 2024-03-07T07:34:14.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:14 smithi177 ceph-mon[36578]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7a432b28-f7ee-43fe-9196-7aaeccfecf77"}]: dispatch 2024-03-07T07:34:14.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:14 smithi177 ceph-mon[36578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7a432b28-f7ee-43fe-9196-7aaeccfecf77"}]': finished 2024-03-07T07:34:14.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:14 smithi177 ceph-mon[36578]: osdmap e15: 3 total, 2 up, 3 in 2024-03-07T07:34:14.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:14 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-07T07:34:14.359 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:14 smithi178 ceph-mon[36650]: from='client.? 172.21.15.178:0/1274664720' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7a432b28-f7ee-43fe-9196-7aaeccfecf77"}]: dispatch 2024-03-07T07:34:14.359 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:14 smithi178 ceph-mon[36650]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7a432b28-f7ee-43fe-9196-7aaeccfecf77"}]: dispatch 2024-03-07T07:34:14.359 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:14 smithi178 ceph-mon[36650]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7a432b28-f7ee-43fe-9196-7aaeccfecf77"}]': finished 2024-03-07T07:34:14.359 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:14 smithi178 ceph-mon[36650]: osdmap e15: 3 total, 2 up, 3 in 2024-03-07T07:34:14.359 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:14 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-07T07:34:14.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:14 smithi160 ceph-mon[31963]: from='client.? 172.21.15.178:0/1274664720' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7a432b28-f7ee-43fe-9196-7aaeccfecf77"}]: dispatch 2024-03-07T07:34:14.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:14 smithi160 ceph-mon[31963]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7a432b28-f7ee-43fe-9196-7aaeccfecf77"}]: dispatch 2024-03-07T07:34:14.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:14 smithi160 ceph-mon[31963]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7a432b28-f7ee-43fe-9196-7aaeccfecf77"}]': finished 2024-03-07T07:34:14.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:14 smithi160 ceph-mon[31963]: osdmap e15: 3 total, 2 up, 3 in 2024-03-07T07:34:14.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:14 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-07T07:34:15.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:15 smithi177 ceph-mon[36578]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-07T07:34:15.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:15 smithi177 ceph-mon[36578]: from='client.? 172.21.15.178:0/3213780010' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-07T07:34:15.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:15 smithi160 ceph-mon[31963]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-07T07:34:15.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:15 smithi160 ceph-mon[31963]: from='client.? 172.21.15.178:0/3213780010' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-07T07:34:15.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:15 smithi178 ceph-mon[36650]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-07T07:34:15.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:15 smithi178 ceph-mon[36650]: from='client.? 172.21.15.178:0/3213780010' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-07T07:34:17.559 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:17 smithi178 ceph-mon[36650]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-07T07:34:17.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:17 smithi177 ceph-mon[36578]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-07T07:34:17.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:17 smithi160 ceph-mon[31963]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-07T07:34:19.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:19 smithi178 ceph-mon[36650]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-07T07:34:19.608 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:19 smithi177 ceph-mon[36578]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-07T07:34:19.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:19 smithi160 ceph-mon[31963]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-07T07:34:21.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:21 smithi178 ceph-mon[36650]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-07T07:34:21.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:21 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-07T07:34:21.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:21 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:34:21.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:21 smithi177 ceph-mon[36578]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-07T07:34:21.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:21 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-07T07:34:21.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:21 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:34:21.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:21 smithi160 ceph-mon[31963]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-07T07:34:21.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:21 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-07T07:34:21.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:21 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:34:22.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:22 smithi178 ceph-mon[36650]: Deploying daemon osd.2 on smithi178 2024-03-07T07:34:22.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:22 smithi177 ceph-mon[36578]: Deploying daemon osd.2 on smithi178 2024-03-07T07:34:22.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:22 smithi160 ceph-mon[31963]: Deploying daemon osd.2 on smithi178 2024-03-07T07:34:23.529 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:23 smithi178 ceph-mon[36650]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-07T07:34:23.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:23 smithi177 ceph-mon[36578]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-07T07:34:23.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:23 smithi160 ceph-mon[31963]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-07T07:34:25.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:25 smithi178 ceph-mon[36650]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-07T07:34:25.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:25 smithi177 ceph-mon[36578]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-07T07:34:25.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:25 smithi160 ceph-mon[31963]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-07T07:34:27.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:27 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:27.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:27 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:27.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:27 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:34:27.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:27 smithi160 ceph-mon[31963]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-07T07:34:27.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:27 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:27.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:27 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:27.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:27 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:34:27.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:27 smithi178 ceph-mon[36650]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-07T07:34:27.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:27 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:27.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:27 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:27.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:27 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:34:27.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:27 smithi177 ceph-mon[36578]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-07T07:34:29.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:29 smithi178 ceph-mon[36650]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-07T07:34:29.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:29 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:29.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:29 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:29.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:29 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:34:29.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:29 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:34:29.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:29 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:29.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:29 smithi177 ceph-mon[36578]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-07T07:34:29.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:29 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:29.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:29 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:29.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:29 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:34:29.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:29 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:34:29.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:29 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:29.618 INFO:teuthology.orchestra.run.smithi178.stdout:Created osd(s) 2 on host 'smithi178' 2024-03-07T07:34:29.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:29.617+0000 7f342cff9700 1 -- 172.21.15.178:0/656743187 <== mgr.14152 v2:172.21.15.160:6800/419630619 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f3430062590 con 0x7f34180a1d80 2024-03-07T07:34:29.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:29.619+0000 7f3435b35700 1 -- 172.21.15.178:0/656743187 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f34180a1d80 msgr2=0x7f34180a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:34:29.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:29.619+0000 7f3435b35700 1 --2- 172.21.15.178:0/656743187 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f34180a1d80 0x7f34180a4240 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3424005950 tx=0x7f342400b410 comp rx=0 tx=0).stop 2024-03-07T07:34:29.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:29.619+0000 7f3435b35700 1 -- 172.21.15.178:0/656743187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3430102c00 msgr2=0x7f3430102560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:34:29.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:29.619+0000 7f3435b35700 1 --2- 172.21.15.178:0/656743187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3430102c00 0x7f3430102560 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3420003890 tx=0x7f34200047b0 comp rx=0 tx=0).stop 2024-03-07T07:34:29.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:29.619+0000 7f3435b35700 1 -- 172.21.15.178:0/656743187 shutdown_connections 2024-03-07T07:34:29.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:29.619+0000 7f3435b35700 1 --2- 172.21.15.178:0/656743187 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3430107e60 0x7f34301010f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:34:29.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:29.619+0000 7f3435b35700 1 --2- 172.21.15.178:0/656743187 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f34180a1d80 0x7f34180a4240 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:34:29.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:29.619+0000 7f3435b35700 1 --2- 172.21.15.178:0/656743187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3430102c00 0x7f3430102560 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:34:29.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:29.619+0000 7f3435b35700 1 --2- 172.21.15.178:0/656743187 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3430105530 0x7f3430100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:34:29.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:29.619+0000 7f3435b35700 1 -- 172.21.15.178:0/656743187 >> 172.21.15.178:0/656743187 conn(0x7f34300fc610 msgr2=0x7f34301036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:34:29.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:29.620+0000 7f3435b35700 1 -- 172.21.15.178:0/656743187 shutdown_connections 2024-03-07T07:34:29.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:34:29.620+0000 7f3435b35700 1 -- 172.21.15.178:0/656743187 wait complete. 2024-03-07T07:34:29.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:29 smithi160 ceph-mon[31963]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-07T07:34:29.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:29 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:29.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:29 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:29.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:29 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:34:29.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:29 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:34:29.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:29 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:30.243 DEBUG:teuthology.orchestra.run.smithi178:osd.2> sudo journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@osd.2.service 2024-03-07T07:34:30.245 INFO:tasks.cephadm:Deploying osd.3 on smithi160 with /dev/vg_nvme/lv_3... 2024-03-07T07:34:30.245 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-07T07:34:30.499 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:34:30.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:30 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:30.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:30 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:30.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:30 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:34:30.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:30 smithi178 ceph-mon[36650]: from='osd.2 [v2:172.21.15.178:6800/1217901551,v1:172.21.15.178:6801/1217901551]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-07T07:34:30.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:30 smithi178 ceph-mon[36650]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-07T07:34:30.848 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:30 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:30.848 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:30 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:30.848 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:30 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:34:30.848 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:30 smithi160 ceph-mon[31963]: from='osd.2 [v2:172.21.15.178:6800/1217901551,v1:172.21.15.178:6801/1217901551]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-07T07:34:30.849 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:30 smithi160 ceph-mon[31963]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-07T07:34:30.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:30 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:30.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:30 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:30.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:30 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:34:30.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:30 smithi177 ceph-mon[36578]: from='osd.2 [v2:172.21.15.178:6800/1217901551,v1:172.21.15.178:6801/1217901551]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-07T07:34:30.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:30 smithi177 ceph-mon[36578]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-07T07:34:31.733 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:31 smithi178 ceph-mon[36650]: from='osd.2 [v2:172.21.15.178:6800/1217901551,v1:172.21.15.178:6801/1217901551]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-07T07:34:31.734 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:31 smithi178 ceph-mon[36650]: osdmap e16: 3 total, 2 up, 3 in 2024-03-07T07:34:31.734 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:31 smithi178 ceph-mon[36650]: from='osd.2 [v2:172.21.15.178:6800/1217901551,v1:172.21.15.178:6801/1217901551]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-03-07T07:34:31.734 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:31 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-07T07:34:31.783 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:31 smithi160 ceph-mon[31963]: from='osd.2 [v2:172.21.15.178:6800/1217901551,v1:172.21.15.178:6801/1217901551]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-07T07:34:31.783 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:31 smithi160 ceph-mon[31963]: osdmap e16: 3 total, 2 up, 3 in 2024-03-07T07:34:31.783 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:31 smithi160 ceph-mon[31963]: from='osd.2 [v2:172.21.15.178:6800/1217901551,v1:172.21.15.178:6801/1217901551]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-03-07T07:34:31.784 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:31 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-07T07:34:32.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:31 smithi177 ceph-mon[36578]: from='osd.2 [v2:172.21.15.178:6800/1217901551,v1:172.21.15.178:6801/1217901551]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-07T07:34:32.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:31 smithi177 ceph-mon[36578]: osdmap e16: 3 total, 2 up, 3 in 2024-03-07T07:34:32.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:31 smithi177 ceph-mon[36578]: from='osd.2 [v2:172.21.15.178:6800/1217901551,v1:172.21.15.178:6801/1217901551]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-03-07T07:34:32.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:31 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-07T07:34:32.455 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-07T07:34:32.455 INFO:teuthology.orchestra.run.smithi160.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-07T07:34:32.455 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 10+0 records in 2024-03-07T07:34:32.455 INFO:teuthology.orchestra.run.smithi160.stderr:10+0 records out 2024-03-07T07:34:32.455 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111936 s, 937 MB/s 2024-03-07T07:34:32.455 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping successful for: 2024-03-07T07:34:32.556 INFO:journalctl@ceph.osd.2.smithi178.stdout:Mar 07 07:34:32 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-2[41777]: 2024-03-07T07:34:32.073+0000 7fa74888b700 -1 osd.2 0 waiting for initial osdmap 2024-03-07T07:34:32.556 INFO:journalctl@ceph.osd.2.smithi178.stdout:Mar 07 07:34:32 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-2[41777]: 2024-03-07T07:34:32.078+0000 7fa740676700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-07T07:34:32.824 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:32 smithi160 ceph-mon[31963]: from='osd.2 [v2:172.21.15.178:6800/1217901551,v1:172.21.15.178:6801/1217901551]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-03-07T07:34:32.824 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:32 smithi160 ceph-mon[31963]: osdmap e17: 3 total, 2 up, 3 in 2024-03-07T07:34:32.824 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:32 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-07T07:34:32.825 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:32 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-07T07:34:32.825 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:32 smithi160 ceph-mon[31963]: from='osd.2 [v2:172.21.15.178:6800/1217901551,v1:172.21.15.178:6801/1217901551]' entity='osd.2' 2024-03-07T07:34:32.825 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:32 smithi160 ceph-mon[31963]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-07T07:34:32.890 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:32 smithi178 ceph-mon[36650]: from='osd.2 [v2:172.21.15.178:6800/1217901551,v1:172.21.15.178:6801/1217901551]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-03-07T07:34:32.891 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:32 smithi178 ceph-mon[36650]: osdmap e17: 3 total, 2 up, 3 in 2024-03-07T07:34:32.891 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:32 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-07T07:34:32.891 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:32 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-07T07:34:32.891 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:32 smithi178 ceph-mon[36650]: from='osd.2 [v2:172.21.15.178:6800/1217901551,v1:172.21.15.178:6801/1217901551]' entity='osd.2' 2024-03-07T07:34:32.891 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:32 smithi178 ceph-mon[36650]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-07T07:34:33.084 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi160:vg_nvme/lv_3 2024-03-07T07:34:33.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:32 smithi177 ceph-mon[36578]: from='osd.2 [v2:172.21.15.178:6800/1217901551,v1:172.21.15.178:6801/1217901551]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-03-07T07:34:33.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:32 smithi177 ceph-mon[36578]: osdmap e17: 3 total, 2 up, 3 in 2024-03-07T07:34:33.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:32 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-07T07:34:33.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:32 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-07T07:34:33.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:32 smithi177 ceph-mon[36578]: from='osd.2 [v2:172.21.15.178:6800/1217901551,v1:172.21.15.178:6801/1217901551]' entity='osd.2' 2024-03-07T07:34:33.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:32 smithi177 ceph-mon[36578]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-07T07:34:33.332 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:34:33.795 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:33 smithi160 ceph-mon[31963]: purged_snaps scrub starts 2024-03-07T07:34:33.795 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:33 smithi160 ceph-mon[31963]: purged_snaps scrub ok 2024-03-07T07:34:33.795 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:33 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-07T07:34:33.795 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:33 smithi160 ceph-mon[31963]: osd.2 [v2:172.21.15.178:6800/1217901551,v1:172.21.15.178:6801/1217901551] boot 2024-03-07T07:34:33.795 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:33 smithi160 ceph-mon[31963]: osdmap e18: 3 total, 3 up, 3 in 2024-03-07T07:34:33.795 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:33 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-07T07:34:33.796 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:33 smithi160 ceph-mon[31963]: Detected new or changed devices on smithi178 2024-03-07T07:34:33.796 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:33 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:33.796 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:33 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:33.796 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:33 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:34:33.796 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:33 smithi160 ceph-mon[31963]: Adjusting osd_memory_target on smithi178 to 17080M 2024-03-07T07:34:33.796 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:33 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:33.796 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:33 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:34:33.796 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:33 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:34:33.796 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:33 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:34.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:33 smithi178 ceph-mon[36650]: purged_snaps scrub starts 2024-03-07T07:34:34.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:33 smithi178 ceph-mon[36650]: purged_snaps scrub ok 2024-03-07T07:34:34.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:33 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-07T07:34:34.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:33 smithi178 ceph-mon[36650]: osd.2 [v2:172.21.15.178:6800/1217901551,v1:172.21.15.178:6801/1217901551] boot 2024-03-07T07:34:34.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:33 smithi178 ceph-mon[36650]: osdmap e18: 3 total, 3 up, 3 in 2024-03-07T07:34:34.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:33 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-07T07:34:34.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:33 smithi178 ceph-mon[36650]: Detected new or changed devices on smithi178 2024-03-07T07:34:34.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:33 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:34.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:33 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:34.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:33 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:34:34.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:33 smithi178 ceph-mon[36650]: Adjusting osd_memory_target on smithi178 to 17080M 2024-03-07T07:34:34.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:33 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:34.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:33 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:34:34.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:33 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:34:34.058 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:33 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:34.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:33 smithi177 ceph-mon[36578]: purged_snaps scrub starts 2024-03-07T07:34:34.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:33 smithi177 ceph-mon[36578]: purged_snaps scrub ok 2024-03-07T07:34:34.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:33 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-07T07:34:34.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:33 smithi177 ceph-mon[36578]: osd.2 [v2:172.21.15.178:6800/1217901551,v1:172.21.15.178:6801/1217901551] boot 2024-03-07T07:34:34.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:33 smithi177 ceph-mon[36578]: osdmap e18: 3 total, 3 up, 3 in 2024-03-07T07:34:34.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:33 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-07T07:34:34.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:33 smithi177 ceph-mon[36578]: Detected new or changed devices on smithi178 2024-03-07T07:34:34.108 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:33 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:34.108 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:33 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:34.108 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:33 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:34:34.112 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:33 smithi177 ceph-mon[36578]: Adjusting osd_memory_target on smithi178 to 17080M 2024-03-07T07:34:34.112 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:33 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:34.112 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:33 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:34:34.112 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:33 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:34:34.112 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:33 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:34.759 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:34 smithi160 ceph-mon[31963]: pgmap v81: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-07T07:34:34.759 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:34 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-07T07:34:34.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:34.971+0000 7fd6464ba700 1 -- 172.21.15.160:0/609772807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd640103380 msgr2=0x7fd6401037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:34:34.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:34.971+0000 7fd6464ba700 1 --2- 172.21.15.160:0/609772807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd640103380 0x7fd6401037e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd6340099c0 tx=0x7fd634009cd0 comp rx=0 tx=0).stop 2024-03-07T07:34:34.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:34.971+0000 7fd6464ba700 1 -- 172.21.15.160:0/609772807 shutdown_connections 2024-03-07T07:34:34.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:34.971+0000 7fd6464ba700 1 --2- 172.21.15.160:0/609772807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd640103d20 0x7fd64010a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:34:34.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:34.971+0000 7fd6464ba700 1 --2- 172.21.15.160:0/609772807 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd640102180 0x7fd6401025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:34:34.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:34.971+0000 7fd6464ba700 1 --2- 172.21.15.160:0/609772807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd640103380 0x7fd6401037e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:34:34.975 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:34.971+0000 7fd6464ba700 1 -- 172.21.15.160:0/609772807 >> 172.21.15.160:0/609772807 conn(0x7fd6400fd760 msgr2=0x7fd6400ffb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:34:34.975 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:34.971+0000 7fd6464ba700 1 -- 172.21.15.160:0/609772807 shutdown_connections 2024-03-07T07:34:34.975 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:34.971+0000 7fd6464ba700 1 -- 172.21.15.160:0/609772807 wait complete. 2024-03-07T07:34:34.975 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:34.971+0000 7fd6464ba700 1 Processor -- start 2024-03-07T07:34:34.975 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:34.971+0000 7fd6464ba700 1 -- start start 2024-03-07T07:34:34.975 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:34.972+0000 7fd6464ba700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd640102180 0x7fd640198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:34:34.975 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:34.972+0000 7fd6464ba700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd640103380 0x7fd6401988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:34:34.975 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:34.972+0000 7fd6464ba700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd640103d20 0x7fd64019f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:34:34.975 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:34.972+0000 7fd6464ba700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64010dd70 con 0x7fd640103d20 2024-03-07T07:34:34.976 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:34.972+0000 7fd6464ba700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64010dbf0 con 0x7fd640103380 2024-03-07T07:34:34.976 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:34.972+0000 7fd6464ba700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64010def0 con 0x7fd640102180 2024-03-07T07:34:34.976 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:34.972+0000 7fd644a57700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd640103d20 0x7fd64019f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:34:34.976 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:34.972+0000 7fd644a57700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd640103d20 0x7fd64019f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34244/0 (socket says 172.21.15.160:34244) 2024-03-07T07:34:34.976 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:34.972+0000 7fd644a57700 1 -- 172.21.15.160:0/1070642828 learned_addr learned my addr 172.21.15.160:0/1070642828 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:34:34.976 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:34.972+0000 7fd63f7fe700 1 --2- 172.21.15.160:0/1070642828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd640103380 0x7fd6401988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:34:34.976 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:34.972+0000 7fd63ffff700 1 --2- 172.21.15.160:0/1070642828 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd640102180 0x7fd640198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:34:34.976 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:34.972+0000 7fd644a57700 1 -- 172.21.15.160:0/1070642828 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd640102180 msgr2=0x7fd640198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:34:34.976 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:34.972+0000 7fd644a57700 1 --2- 172.21.15.160:0/1070642828 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd640102180 0x7fd640198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:34:34.976 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:34.972+0000 7fd644a57700 1 -- 172.21.15.160:0/1070642828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd640103380 msgr2=0x7fd6401988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:34:34.976 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:34.972+0000 7fd644a57700 1 --2- 172.21.15.160:0/1070642828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd640103380 0x7fd6401988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:34:34.976 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:34.972+0000 7fd644a57700 1 -- 172.21.15.160:0/1070642828 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd64019ffe0 con 0x7fd640103d20 2024-03-07T07:34:34.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:34.972+0000 7fd63ffff700 1 --2- 172.21.15.160:0/1070642828 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd640102180 0x7fd640198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:34:34.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:34.972+0000 7fd644a57700 1 --2- 172.21.15.160:0/1070642828 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd640103d20 0x7fd64019f940 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd638007f10 tx=0x7fd63800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:34:34.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:34.973+0000 7fd63d7fa700 1 -- 172.21.15.160:0/1070642828 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd638004bb0 con 0x7fd640103d20 2024-03-07T07:34:34.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:34.973+0000 7fd6464ba700 1 -- 172.21.15.160:0/1070642828 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6401a02d0 con 0x7fd640103d20 2024-03-07T07:34:34.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:34.973+0000 7fd63d7fa700 1 -- 172.21.15.160:0/1070642828 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd638004d50 con 0x7fd640103d20 2024-03-07T07:34:34.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:34.973+0000 7fd63d7fa700 1 -- 172.21.15.160:0/1070642828 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd638005a60 con 0x7fd640103d20 2024-03-07T07:34:34.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:34.973+0000 7fd6464ba700 1 -- 172.21.15.160:0/1070642828 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6401a0860 con 0x7fd640103d20 2024-03-07T07:34:34.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:34.974+0000 7fd6464ba700 1 -- 172.21.15.160:0/1070642828 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd640068e10 con 0x7fd640103d20 2024-03-07T07:34:34.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:34.975+0000 7fd63d7fa700 1 -- 172.21.15.160:0/1070642828 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 15) v1 ==== 136698+0+0 (secure 0 0 0) 0x7fd63801a020 con 0x7fd640103d20 2024-03-07T07:34:34.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:34.975+0000 7fd63d7fa700 1 --2- 172.21.15.160:0/1070642828 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fd6280a1d80 0x7fd6280a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:34:34.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:34.975+0000 7fd63ffff700 1 --2- 172.21.15.160:0/1070642828 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fd6280a1d80 0x7fd6280a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:34:34.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:34.975+0000 7fd63d7fa700 1 -- 172.21.15.160:0/1070642828 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3226+0+0 (secure 0 0 0) 0x7fd6380cc050 con 0x7fd640103d20 2024-03-07T07:34:34.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:34.976+0000 7fd63ffff700 1 --2- 172.21.15.160:0/1070642828 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fd6280a1d80 0x7fd6280a4240 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd6401031e0 tx=0x7fd630005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:34:34.980 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:34.980+0000 7fd63d7fa700 1 -- 172.21.15.160:0/1070642828 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd638010070 con 0x7fd640103d20 2024-03-07T07:34:35.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:34 smithi178 ceph-mon[36650]: pgmap v81: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-07T07:34:35.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:34 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-07T07:34:35.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:34 smithi177 ceph-mon[36578]: pgmap v81: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-07T07:34:35.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:34 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-07T07:34:35.183 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:35.183+0000 7fd6464ba700 1 -- 172.21.15.160:0/1070642828 --> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fd640062590 con 0x7fd6280a1d80 2024-03-07T07:34:35.876 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:35 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-07T07:34:35.876 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:35 smithi160 ceph-mon[31963]: osdmap e19: 3 total, 3 up, 3 in 2024-03-07T07:34:35.876 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:35 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-07T07:34:35.876 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:35 smithi160 ceph-mon[31963]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:34:35.876 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:35 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-07T07:34:35.876 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:35 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-07T07:34:35.876 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:35 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:34:36.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:35 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-07T07:34:36.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:35 smithi178 ceph-mon[36650]: osdmap e19: 3 total, 3 up, 3 in 2024-03-07T07:34:36.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:35 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-07T07:34:36.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:35 smithi178 ceph-mon[36650]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:34:36.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:35 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-07T07:34:36.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:35 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-07T07:34:36.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:35 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:34:36.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:35 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-07T07:34:36.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:35 smithi177 ceph-mon[36578]: osdmap e19: 3 total, 3 up, 3 in 2024-03-07T07:34:36.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:35 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-07T07:34:36.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:35 smithi177 ceph-mon[36578]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:34:36.108 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:35 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-07T07:34:36.108 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:35 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-07T07:34:36.108 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:35 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:34:36.641 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:36 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-07T07:34:36.641 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:36 smithi160 ceph-mon[31963]: osdmap e20: 3 total, 3 up, 3 in 2024-03-07T07:34:36.641 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:36 smithi160 ceph-mon[31963]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-07T07:34:36.641 INFO:journalctl@ceph.osd.0.smithi160.stdout:Mar 07 07:34:36 smithi160 sudo[51313]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-07T07:34:36.641 INFO:journalctl@ceph.osd.0.smithi160.stdout:Mar 07 07:34:36 smithi160 sudo[51313]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-07T07:34:36.641 INFO:journalctl@ceph.osd.0.smithi160.stdout:Mar 07 07:34:36 smithi160 sudo[51313]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-07T07:34:36.641 INFO:journalctl@ceph.osd.0.smithi160.stdout:Mar 07 07:34:36 smithi160 sudo[51313]: pam_unix(sudo:session): session closed for user root 2024-03-07T07:34:36.901 INFO:journalctl@ceph.osd.0.smithi160.stdout:Mar 07 07:34:36 smithi160 sudo[51316]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-07T07:34:36.902 INFO:journalctl@ceph.osd.0.smithi160.stdout:Mar 07 07:34:36 smithi160 sudo[51316]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-07T07:34:36.902 INFO:journalctl@ceph.osd.0.smithi160.stdout:Mar 07 07:34:36 smithi160 sudo[51316]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-07T07:34:36.902 INFO:journalctl@ceph.osd.0.smithi160.stdout:Mar 07 07:34:36 smithi160 sudo[51316]: pam_unix(sudo:session): session closed for user root 2024-03-07T07:34:36.906 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:36 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-07T07:34:36.906 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:36 smithi177 ceph-mon[36578]: osdmap e20: 3 total, 3 up, 3 in 2024-03-07T07:34:36.906 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:36 smithi177 ceph-mon[36578]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-07T07:34:37.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:36 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-07T07:34:37.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:36 smithi178 ceph-mon[36650]: osdmap e20: 3 total, 3 up, 3 in 2024-03-07T07:34:37.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:36 smithi178 ceph-mon[36650]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-07T07:34:37.158 INFO:journalctl@ceph.osd.1.smithi177.stdout:Mar 07 07:34:36 smithi177 sudo[45318]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-07T07:34:37.158 INFO:journalctl@ceph.osd.1.smithi177.stdout:Mar 07 07:34:36 smithi177 sudo[45318]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-07T07:34:37.158 INFO:journalctl@ceph.osd.1.smithi177.stdout:Mar 07 07:34:36 smithi177 sudo[45318]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-07T07:34:37.158 INFO:journalctl@ceph.osd.1.smithi177.stdout:Mar 07 07:34:36 smithi177 sudo[45318]: pam_unix(sudo:session): session closed for user root 2024-03-07T07:34:37.606 INFO:journalctl@ceph.osd.1.smithi177.stdout:Mar 07 07:34:37 smithi177 sudo[45321]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-07T07:34:37.607 INFO:journalctl@ceph.osd.1.smithi177.stdout:Mar 07 07:34:37 smithi177 sudo[45321]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-07T07:34:37.607 INFO:journalctl@ceph.osd.1.smithi177.stdout:Mar 07 07:34:37 smithi177 sudo[45321]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-07T07:34:37.607 INFO:journalctl@ceph.osd.1.smithi177.stdout:Mar 07 07:34:37 smithi177 sudo[45321]: pam_unix(sudo:session): session closed for user root 2024-03-07T07:34:37.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:37 smithi178 ceph-mon[36650]: osdmap e21: 3 total, 3 up, 3 in 2024-03-07T07:34:37.677 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:37 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:37.677 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:37 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:37.677 INFO:journalctl@ceph.osd.2.smithi178.stdout:Mar 07 07:34:37 smithi178 sudo[45865]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-07T07:34:37.677 INFO:journalctl@ceph.osd.2.smithi178.stdout:Mar 07 07:34:37 smithi178 sudo[45865]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-07T07:34:37.677 INFO:journalctl@ceph.osd.2.smithi178.stdout:Mar 07 07:34:37 smithi178 sudo[45865]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-07T07:34:37.677 INFO:journalctl@ceph.osd.2.smithi178.stdout:Mar 07 07:34:37 smithi178 sudo[45865]: pam_unix(sudo:session): session closed for user root 2024-03-07T07:34:37.678 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:37.678+0000 7fd63d7fa700 1 -- 172.21.15.160:0/1070642828 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7fd6380968b0 con 0x7fd640103d20 2024-03-07T07:34:37.927 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:37 smithi160 ceph-mon[31963]: osdmap e21: 3 total, 3 up, 3 in 2024-03-07T07:34:37.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:37 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:37.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:37 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:38.057 INFO:journalctl@ceph.osd.2.smithi178.stdout:Mar 07 07:34:37 smithi178 sudo[45868]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-07T07:34:38.057 INFO:journalctl@ceph.osd.2.smithi178.stdout:Mar 07 07:34:37 smithi178 sudo[45868]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-07T07:34:38.057 INFO:journalctl@ceph.osd.2.smithi178.stdout:Mar 07 07:34:37 smithi178 sudo[45868]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-07T07:34:38.057 INFO:journalctl@ceph.osd.2.smithi178.stdout:Mar 07 07:34:37 smithi178 sudo[45868]: pam_unix(sudo:session): session closed for user root 2024-03-07T07:34:38.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:37 smithi177 ceph-mon[36578]: osdmap e21: 3 total, 3 up, 3 in 2024-03-07T07:34:38.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:37 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:38.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:37 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:38.207 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:37 smithi160 sudo[51392]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-07T07:34:38.207 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:37 smithi160 sudo[51392]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-07T07:34:38.208 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:37 smithi160 sudo[51392]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-07T07:34:38.208 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:37 smithi160 sudo[51392]: pam_unix(sudo:session): session closed for user root 2024-03-07T07:34:38.583 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:38 smithi160 sudo[51395]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-07T07:34:38.584 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:38 smithi160 sudo[51395]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-07T07:34:38.584 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:38 smithi160 sudo[51395]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-07T07:34:38.584 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:38 smithi160 sudo[51395]: pam_unix(sudo:session): session closed for user root 2024-03-07T07:34:38.701 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:38 smithi178 sudo[45871]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-07T07:34:38.701 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:38 smithi178 sudo[45871]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-07T07:34:38.701 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:38 smithi178 sudo[45871]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-07T07:34:38.701 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:38 smithi178 sudo[45871]: pam_unix(sudo:session): session closed for user root 2024-03-07T07:34:38.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:38 smithi160 ceph-mon[31963]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-07T07:34:38.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:38 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:38.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:38 smithi160 ceph-mon[31963]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-07T07:34:38.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:38 smithi160 ceph-mon[31963]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-07T07:34:38.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:38 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-07T07:34:38.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:38 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/955557028' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ad22a20-f1be-4002-a48d-ecd3953e63f3"}]: dispatch 2024-03-07T07:34:38.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:38 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-07T07:34:38.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:38 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-07T07:34:38.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:38 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/955557028' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4ad22a20-f1be-4002-a48d-ecd3953e63f3"}]': finished 2024-03-07T07:34:38.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:38 smithi160 ceph-mon[31963]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-07T07:34:38.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:38 smithi160 ceph-mon[31963]: osdmap e22: 4 total, 3 up, 4 in 2024-03-07T07:34:38.929 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:38 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-07T07:34:38.929 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:38 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-07T07:34:38.929 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:38 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-07T07:34:38.929 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:38 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-07T07:34:38.962 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:38 smithi177 ceph-mon[36578]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-07T07:34:38.962 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:38 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:38.962 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:38 smithi177 ceph-mon[36578]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-07T07:34:38.962 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:38 smithi177 ceph-mon[36578]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-07T07:34:38.962 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:38 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-07T07:34:38.962 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:38 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/955557028' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ad22a20-f1be-4002-a48d-ecd3953e63f3"}]: dispatch 2024-03-07T07:34:38.962 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:38 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-07T07:34:38.963 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:38 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-07T07:34:38.963 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:38 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/955557028' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4ad22a20-f1be-4002-a48d-ecd3953e63f3"}]': finished 2024-03-07T07:34:38.963 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:38 smithi177 ceph-mon[36578]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-07T07:34:38.963 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:38 smithi177 ceph-mon[36578]: osdmap e22: 4 total, 3 up, 4 in 2024-03-07T07:34:38.963 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:38 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-07T07:34:38.963 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:38 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-07T07:34:38.963 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:38 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-07T07:34:38.963 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:38 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-07T07:34:38.963 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:38 smithi177 sudo[45324]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-07T07:34:38.963 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:38 smithi177 sudo[45324]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-07T07:34:38.963 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:38 smithi177 sudo[45324]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-07T07:34:38.963 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:38 smithi177 sudo[45324]: pam_unix(sudo:session): session closed for user root 2024-03-07T07:34:39.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:38 smithi178 sudo[45874]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-07T07:34:39.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:38 smithi178 sudo[45874]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-07T07:34:39.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:38 smithi178 sudo[45874]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-07T07:34:39.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:38 smithi178 sudo[45874]: pam_unix(sudo:session): session closed for user root 2024-03-07T07:34:39.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:38 smithi178 ceph-mon[36650]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-07T07:34:39.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:38 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:39.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:38 smithi178 ceph-mon[36650]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-07T07:34:39.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:38 smithi178 ceph-mon[36650]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-07T07:34:39.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:38 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-07T07:34:39.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:38 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/955557028' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ad22a20-f1be-4002-a48d-ecd3953e63f3"}]: dispatch 2024-03-07T07:34:39.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:38 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-07T07:34:39.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:38 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-07T07:34:39.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:38 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/955557028' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4ad22a20-f1be-4002-a48d-ecd3953e63f3"}]': finished 2024-03-07T07:34:39.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:38 smithi178 ceph-mon[36650]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-07T07:34:39.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:38 smithi178 ceph-mon[36650]: osdmap e22: 4 total, 3 up, 4 in 2024-03-07T07:34:39.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:38 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-07T07:34:39.058 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:38 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-07T07:34:39.058 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:38 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-07T07:34:39.058 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:38 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-07T07:34:39.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:39 smithi177 sudo[45327]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-07T07:34:39.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:39 smithi177 sudo[45327]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-07T07:34:39.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:39 smithi177 sudo[45327]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-07T07:34:39.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:39 smithi177 sudo[45327]: pam_unix(sudo:session): session closed for user root 2024-03-07T07:34:40.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:39 smithi178 ceph-mon[36650]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-07T07:34:40.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:39 smithi178 ceph-mon[36650]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-07T07:34:40.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:39 smithi178 ceph-mon[36650]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-07T07:34:40.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:39 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-07T07:34:40.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:39 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-07T07:34:40.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:39 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-07T07:34:40.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:39 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/2866903813' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-07T07:34:40.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:39 smithi178 ceph-mon[36650]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-07T07:34:40.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:39 smithi177 ceph-mon[36578]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-07T07:34:40.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:39 smithi177 ceph-mon[36578]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-07T07:34:40.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:39 smithi177 ceph-mon[36578]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-07T07:34:40.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:39 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-07T07:34:40.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:39 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-07T07:34:40.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:39 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-07T07:34:40.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:39 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/2866903813' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-07T07:34:40.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:39 smithi177 ceph-mon[36578]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-07T07:34:40.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:39 smithi160 ceph-mon[31963]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-07T07:34:40.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:39 smithi160 ceph-mon[31963]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-07T07:34:40.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:39 smithi160 ceph-mon[31963]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-07T07:34:40.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:39 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-07T07:34:40.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:39 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-07T07:34:40.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:39 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-07T07:34:40.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:39 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/2866903813' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-07T07:34:40.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:39 smithi160 ceph-mon[31963]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-07T07:34:41.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:40 smithi178 ceph-mon[36650]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-07T07:34:41.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:40 smithi177 ceph-mon[36578]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-07T07:34:41.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:40 smithi160 ceph-mon[31963]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-07T07:34:43.497 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:43 smithi160 ceph-mon[31963]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-07T07:34:43.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:43 smithi178 ceph-mon[36650]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-07T07:34:43.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:43 smithi177 ceph-mon[36578]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-07T07:34:45.540 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:45 smithi160 ceph-mon[31963]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-07T07:34:45.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:45 smithi178 ceph-mon[36650]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-07T07:34:45.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:45 smithi177 ceph-mon[36578]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-07T07:34:46.435 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:46 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-07T07:34:46.435 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:46 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:34:46.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:46 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-07T07:34:46.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:46 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:34:46.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:46 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-07T07:34:46.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:46 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:34:47.457 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:47 smithi160 ceph-mon[31963]: Deploying daemon osd.3 on smithi160 2024-03-07T07:34:47.457 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:47 smithi160 ceph-mon[31963]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-07T07:34:47.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:47 smithi178 ceph-mon[36650]: Deploying daemon osd.3 on smithi160 2024-03-07T07:34:47.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:47 smithi178 ceph-mon[36650]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-07T07:34:47.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:47 smithi177 ceph-mon[36578]: Deploying daemon osd.3 on smithi160 2024-03-07T07:34:47.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:47 smithi177 ceph-mon[36578]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-07T07:34:49.537 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:49 smithi160 ceph-mon[31963]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-07T07:34:49.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:49 smithi178 ceph-mon[36650]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-07T07:34:49.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:49 smithi177 ceph-mon[36578]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-07T07:34:51.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:51 smithi178 ceph-mon[36650]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-07T07:34:51.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:51 smithi177 ceph-mon[36578]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-07T07:34:51.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:51 smithi160 ceph-mon[31963]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-07T07:34:52.301 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:52 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:34:52.301 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:52 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:52.301 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:52 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:52.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:52 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:34:52.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:52 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:52.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:52 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:52.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:52 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:34:52.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:52 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:52.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:52 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:53.317 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:53 smithi160 ceph-mon[31963]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-07T07:34:53.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:53 smithi178 ceph-mon[36650]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-07T07:34:53.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:53 smithi177 ceph-mon[36578]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-07T07:34:55.243 INFO:teuthology.orchestra.run.smithi160.stdout:Created osd(s) 3 on host 'smithi160' 2024-03-07T07:34:55.244 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:55.242+0000 7fd63d7fa700 1 -- 172.21.15.160:0/1070642828 <== mgr.14152 v2:172.21.15.160:6800/419630619 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd640062590 con 0x7fd6280a1d80 2024-03-07T07:34:55.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:55.245+0000 7fd6464ba700 1 -- 172.21.15.160:0/1070642828 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fd6280a1d80 msgr2=0x7fd6280a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:34:55.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:55.245+0000 7fd6464ba700 1 --2- 172.21.15.160:0/1070642828 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fd6280a1d80 0x7fd6280a4240 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd6401031e0 tx=0x7fd630005f90 comp rx=0 tx=0).stop 2024-03-07T07:34:55.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:55.245+0000 7fd6464ba700 1 -- 172.21.15.160:0/1070642828 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd640103d20 msgr2=0x7fd64019f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:34:55.245 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:55.245+0000 7fd6464ba700 1 --2- 172.21.15.160:0/1070642828 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd640103d20 0x7fd64019f940 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd638007f10 tx=0x7fd63800d3b0 comp rx=0 tx=0).stop 2024-03-07T07:34:55.246 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:55.245+0000 7fd6464ba700 1 -- 172.21.15.160:0/1070642828 shutdown_connections 2024-03-07T07:34:55.246 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:55.245+0000 7fd6464ba700 1 --2- 172.21.15.160:0/1070642828 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd640103d20 0x7fd64019f940 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:34:55.246 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:55.245+0000 7fd6464ba700 1 --2- 172.21.15.160:0/1070642828 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fd6280a1d80 0x7fd6280a4240 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:34:55.246 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:55.245+0000 7fd6464ba700 1 --2- 172.21.15.160:0/1070642828 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd640102180 0x7fd640198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:34:55.246 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:55.245+0000 7fd6464ba700 1 --2- 172.21.15.160:0/1070642828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd640103380 0x7fd6401988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:34:55.246 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:55.245+0000 7fd6464ba700 1 -- 172.21.15.160:0/1070642828 >> 172.21.15.160:0/1070642828 conn(0x7fd6400fd760 msgr2=0x7fd6401066d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:34:55.246 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:55.245+0000 7fd6464ba700 1 -- 172.21.15.160:0/1070642828 shutdown_connections 2024-03-07T07:34:55.246 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:34:55.245+0000 7fd6464ba700 1 -- 172.21.15.160:0/1070642828 wait complete. 2024-03-07T07:34:55.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:55 smithi178 ceph-mon[36650]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-07T07:34:55.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:55 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:55.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:55 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:55.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:55 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:34:55.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:55 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:34:55.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:55 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:55.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:55 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:55.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:55 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:55.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:55 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:34:55.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:55 smithi178 ceph-mon[36650]: from='osd.3 [v2:172.21.15.160:6810/42543989,v1:172.21.15.160:6811/42543989]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-07T07:34:55.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:55 smithi177 ceph-mon[36578]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-07T07:34:55.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:55 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:55.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:55 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:55.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:55 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:34:55.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:55 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:34:55.608 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:55 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:55.608 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:55 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:55.608 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:55 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:55.608 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:55 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:34:55.608 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:55 smithi177 ceph-mon[36578]: from='osd.3 [v2:172.21.15.160:6810/42543989,v1:172.21.15.160:6811/42543989]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-07T07:34:55.622 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:55 smithi160 ceph-mon[31963]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-07T07:34:55.622 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:55 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:55.622 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:55 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:55.622 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:55 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:34:55.622 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:55 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:34:55.623 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:55 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:55.623 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:55 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:55.623 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:55 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:55.623 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:55 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:34:55.623 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:55 smithi160 ceph-mon[31963]: from='osd.3 [v2:172.21.15.160:6810/42543989,v1:172.21.15.160:6811/42543989]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-07T07:34:55.981 DEBUG:teuthology.orchestra.run.smithi160:osd.3> sudo journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@osd.3.service 2024-03-07T07:34:55.983 INFO:tasks.cephadm:Deploying osd.4 on smithi177 with /dev/vg_nvme/lv_3... 2024-03-07T07:34:55.984 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-07T07:34:56.228 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.b/config 2024-03-07T07:34:57.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:57 smithi160 ceph-mon[31963]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-07T07:34:57.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:57 smithi160 ceph-mon[31963]: from='osd.3 [v2:172.21.15.160:6810/42543989,v1:172.21.15.160:6811/42543989]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-07T07:34:57.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:57 smithi160 ceph-mon[31963]: osdmap e23: 4 total, 3 up, 4 in 2024-03-07T07:34:57.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:57 smithi160 ceph-mon[31963]: from='osd.3 [v2:172.21.15.160:6810/42543989,v1:172.21.15.160:6811/42543989]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-03-07T07:34:57.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:57 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-07T07:34:57.429 INFO:journalctl@ceph.osd.3.smithi160.stdout:Mar 07 07:34:57 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-3[53426]: 2024-03-07T07:34:57.240+0000 7fb69d866700 -1 osd.3 0 waiting for initial osdmap 2024-03-07T07:34:57.429 INFO:journalctl@ceph.osd.3.smithi160.stdout:Mar 07 07:34:57 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-3[53426]: 2024-03-07T07:34:57.246+0000 7fb699659700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-07T07:34:57.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:57 smithi178 ceph-mon[36650]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-07T07:34:57.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:57 smithi178 ceph-mon[36650]: from='osd.3 [v2:172.21.15.160:6810/42543989,v1:172.21.15.160:6811/42543989]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-07T07:34:57.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:57 smithi178 ceph-mon[36650]: osdmap e23: 4 total, 3 up, 4 in 2024-03-07T07:34:57.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:57 smithi178 ceph-mon[36650]: from='osd.3 [v2:172.21.15.160:6810/42543989,v1:172.21.15.160:6811/42543989]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-03-07T07:34:57.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:57 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-07T07:34:57.561 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:57 smithi177 ceph-mon[36578]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-07T07:34:57.561 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:57 smithi177 ceph-mon[36578]: from='osd.3 [v2:172.21.15.160:6810/42543989,v1:172.21.15.160:6811/42543989]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-07T07:34:57.561 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:57 smithi177 ceph-mon[36578]: osdmap e23: 4 total, 3 up, 4 in 2024-03-07T07:34:57.561 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:57 smithi177 ceph-mon[36578]: from='osd.3 [v2:172.21.15.160:6810/42543989,v1:172.21.15.160:6811/42543989]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-03-07T07:34:57.561 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:57 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-07T07:34:58.212 INFO:teuthology.orchestra.run.smithi177.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-07T07:34:58.212 INFO:teuthology.orchestra.run.smithi177.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-07T07:34:58.212 INFO:teuthology.orchestra.run.smithi177.stderr: stderr: 10+0 records in 2024-03-07T07:34:58.212 INFO:teuthology.orchestra.run.smithi177.stderr:10+0 records out 2024-03-07T07:34:58.212 INFO:teuthology.orchestra.run.smithi177.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111441 s, 941 MB/s 2024-03-07T07:34:58.212 INFO:teuthology.orchestra.run.smithi177.stderr:--> Zapping successful for: 2024-03-07T07:34:58.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:58 smithi178 ceph-mon[36650]: from='osd.3 [v2:172.21.15.160:6810/42543989,v1:172.21.15.160:6811/42543989]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-03-07T07:34:58.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:58 smithi178 ceph-mon[36650]: osdmap e24: 4 total, 3 up, 4 in 2024-03-07T07:34:58.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:58 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-07T07:34:58.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:58 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-07T07:34:58.598 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:58 smithi177 ceph-mon[36578]: from='osd.3 [v2:172.21.15.160:6810/42543989,v1:172.21.15.160:6811/42543989]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-03-07T07:34:58.598 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:58 smithi177 ceph-mon[36578]: osdmap e24: 4 total, 3 up, 4 in 2024-03-07T07:34:58.598 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:58 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-07T07:34:58.598 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:58 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-07T07:34:58.652 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:58 smithi160 ceph-mon[31963]: from='osd.3 [v2:172.21.15.160:6810/42543989,v1:172.21.15.160:6811/42543989]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-03-07T07:34:58.652 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:58 smithi160 ceph-mon[31963]: osdmap e24: 4 total, 3 up, 4 in 2024-03-07T07:34:58.653 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:58 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-07T07:34:58.653 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:58 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-07T07:34:58.908 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi177:vg_nvme/lv_3 2024-03-07T07:34:59.143 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.b/config 2024-03-07T07:34:59.535 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:59 smithi177 ceph-mon[36578]: purged_snaps scrub starts 2024-03-07T07:34:59.535 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:59 smithi177 ceph-mon[36578]: purged_snaps scrub ok 2024-03-07T07:34:59.535 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:59 smithi177 ceph-mon[36578]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-07T07:34:59.535 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:59 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-07T07:34:59.535 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:59 smithi177 ceph-mon[36578]: osd.3 [v2:172.21.15.160:6810/42543989,v1:172.21.15.160:6811/42543989] boot 2024-03-07T07:34:59.535 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:59 smithi177 ceph-mon[36578]: osdmap e25: 4 total, 4 up, 4 in 2024-03-07T07:34:59.535 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:59 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-07T07:34:59.535 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:59 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:59.535 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:59 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:59.535 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:59 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:34:59.535 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:59 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:34:59.536 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:59 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:59.536 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:59 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:34:59.536 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:59 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:34:59.536 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:34:59 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:59.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:59 smithi178 ceph-mon[36650]: purged_snaps scrub starts 2024-03-07T07:34:59.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:59 smithi178 ceph-mon[36650]: purged_snaps scrub ok 2024-03-07T07:34:59.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:59 smithi178 ceph-mon[36650]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-07T07:34:59.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:59 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-07T07:34:59.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:59 smithi178 ceph-mon[36650]: osd.3 [v2:172.21.15.160:6810/42543989,v1:172.21.15.160:6811/42543989] boot 2024-03-07T07:34:59.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:59 smithi178 ceph-mon[36650]: osdmap e25: 4 total, 4 up, 4 in 2024-03-07T07:34:59.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:59 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-07T07:34:59.558 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:59 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:59.558 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:59 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:59.558 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:59 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:34:59.558 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:59 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:34:59.558 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:59 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:59.558 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:59 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:34:59.559 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:59 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:34:59.559 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:34:59 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:59.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:59 smithi160 ceph-mon[31963]: purged_snaps scrub starts 2024-03-07T07:34:59.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:59 smithi160 ceph-mon[31963]: purged_snaps scrub ok 2024-03-07T07:34:59.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:59 smithi160 ceph-mon[31963]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-07T07:34:59.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:59 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-07T07:34:59.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:59 smithi160 ceph-mon[31963]: osd.3 [v2:172.21.15.160:6810/42543989,v1:172.21.15.160:6811/42543989] boot 2024-03-07T07:34:59.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:59 smithi160 ceph-mon[31963]: osdmap e25: 4 total, 4 up, 4 in 2024-03-07T07:34:59.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:59 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-07T07:34:59.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:59 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:59.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:59 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:59.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:59 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:34:59.679 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:59 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:34:59.679 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:59 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:34:59.679 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:59 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:34:59.679 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:59 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:34:59.679 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:34:59 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:00.520 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:00 smithi160 ceph-mon[31963]: Detected new or changed devices on smithi160 2024-03-07T07:35:00.520 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:00 smithi160 ceph-mon[31963]: Adjusting osd_memory_target on smithi160 to 8544M 2024-03-07T07:35:00.520 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:00 smithi160 ceph-mon[31963]: osdmap e26: 4 total, 4 up, 4 in 2024-03-07T07:35:00.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:00 smithi178 ceph-mon[36650]: Detected new or changed devices on smithi160 2024-03-07T07:35:00.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:00 smithi178 ceph-mon[36650]: Adjusting osd_memory_target on smithi160 to 8544M 2024-03-07T07:35:00.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:00 smithi178 ceph-mon[36650]: osdmap e26: 4 total, 4 up, 4 in 2024-03-07T07:35:00.567 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:00 smithi177 ceph-mon[36578]: Detected new or changed devices on smithi160 2024-03-07T07:35:00.568 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:00 smithi177 ceph-mon[36578]: Adjusting osd_memory_target on smithi160 to 8544M 2024-03-07T07:35:00.568 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:00 smithi177 ceph-mon[36578]: osdmap e26: 4 total, 4 up, 4 in 2024-03-07T07:35:00.794 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:00.792+0000 7f0e43fbc700 1 -- 172.21.15.177:0/210708170 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e3c103d20 msgr2=0x7f0e3c10a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:35:00.794 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:00.792+0000 7f0e43fbc700 1 --2- 172.21.15.177:0/210708170 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e3c103d20 0x7f0e3c10a5d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0e3400b0d0 tx=0x7f0e3400b3e0 comp rx=0 tx=0).stop 2024-03-07T07:35:00.794 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:00.792+0000 7f0e43fbc700 1 -- 172.21.15.177:0/210708170 shutdown_connections 2024-03-07T07:35:00.794 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:00.792+0000 7f0e43fbc700 1 --2- 172.21.15.177:0/210708170 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e3c103d20 0x7f0e3c10a5d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:35:00.794 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:00.792+0000 7f0e43fbc700 1 --2- 172.21.15.177:0/210708170 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e3c102180 0x7f0e3c1025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:35:00.794 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:00.792+0000 7f0e43fbc700 1 --2- 172.21.15.177:0/210708170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e3c103380 0x7f0e3c1037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:35:00.795 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:00.792+0000 7f0e43fbc700 1 -- 172.21.15.177:0/210708170 >> 172.21.15.177:0/210708170 conn(0x7f0e3c0fd760 msgr2=0x7f0e3c0ffb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:35:00.795 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:00.792+0000 7f0e43fbc700 1 -- 172.21.15.177:0/210708170 shutdown_connections 2024-03-07T07:35:00.795 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:00.792+0000 7f0e43fbc700 1 -- 172.21.15.177:0/210708170 wait complete. 2024-03-07T07:35:00.795 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:00.792+0000 7f0e43fbc700 1 Processor -- start 2024-03-07T07:35:00.795 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:00.792+0000 7f0e43fbc700 1 -- start start 2024-03-07T07:35:00.795 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:00.792+0000 7f0e43fbc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e3c102180 0x7f0e3c198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:35:00.795 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:00.792+0000 7f0e43fbc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e3c103380 0x7f0e3c1987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:35:00.795 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:00.792+0000 7f0e43fbc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e3c103d20 0x7f0e3c19f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:35:00.795 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:00.792+0000 7f0e43fbc700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c10dd60 con 0x7f0e3c103d20 2024-03-07T07:35:00.795 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:00.792+0000 7f0e43fbc700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c10dbe0 con 0x7f0e3c103380 2024-03-07T07:35:00.795 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:00.792+0000 7f0e43fbc700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c10dee0 con 0x7f0e3c102180 2024-03-07T07:35:00.795 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:00.793+0000 7f0e41557700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e3c103380 0x7f0e3c1987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:35:00.795 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:00.793+0000 7f0e41557700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e3c103380 0x7f0e3c1987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:42308/0 (socket says 172.21.15.177:42308) 2024-03-07T07:35:00.796 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:00.793+0000 7f0e42559700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e3c103d20 0x7f0e3c19f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:35:00.796 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:00.793+0000 7f0e41557700 1 -- 172.21.15.177:0/4136849185 learned_addr learned my addr 172.21.15.177:0/4136849185 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-03-07T07:35:00.796 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:00.793+0000 7f0e41d58700 1 --2- 172.21.15.177:0/4136849185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e3c102180 0x7f0e3c198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:35:00.796 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:00.793+0000 7f0e42559700 1 -- 172.21.15.177:0/4136849185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e3c102180 msgr2=0x7f0e3c198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:35:00.796 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:00.793+0000 7f0e42559700 1 --2- 172.21.15.177:0/4136849185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e3c102180 0x7f0e3c198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:35:00.796 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:00.793+0000 7f0e42559700 1 -- 172.21.15.177:0/4136849185 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e3c103380 msgr2=0x7f0e3c1987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:35:00.796 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:00.793+0000 7f0e42559700 1 --2- 172.21.15.177:0/4136849185 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e3c103380 0x7f0e3c1987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:35:00.796 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:00.793+0000 7f0e42559700 1 -- 172.21.15.177:0/4136849185 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e3c19fee0 con 0x7f0e3c103d20 2024-03-07T07:35:00.796 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:00.793+0000 7f0e41557700 1 --2- 172.21.15.177:0/4136849185 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e3c103380 0x7f0e3c1987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:35:00.796 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:00.793+0000 7f0e42559700 1 --2- 172.21.15.177:0/4136849185 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e3c103d20 0x7f0e3c19f840 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0e3400b0d0 tx=0x7f0e34007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:35:00.796 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:00.793+0000 7f0e32ffd700 1 -- 172.21.15.177:0/4136849185 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e34021020 con 0x7f0e3c103d20 2024-03-07T07:35:00.796 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:00.793+0000 7f0e43fbc700 1 -- 172.21.15.177:0/4136849185 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e3c1a0170 con 0x7f0e3c103d20 2024-03-07T07:35:00.796 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:00.793+0000 7f0e32ffd700 1 -- 172.21.15.177:0/4136849185 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0e3401d950 con 0x7f0e3c103d20 2024-03-07T07:35:00.797 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:00.793+0000 7f0e32ffd700 1 -- 172.21.15.177:0/4136849185 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e340053b0 con 0x7f0e3c103d20 2024-03-07T07:35:00.797 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:00.793+0000 7f0e43fbc700 1 -- 172.21.15.177:0/4136849185 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e3c1a06b0 con 0x7f0e3c103d20 2024-03-07T07:35:00.798 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:00.794+0000 7f0e43fbc700 1 -- 172.21.15.177:0/4136849185 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e3c068e10 con 0x7f0e3c103d20 2024-03-07T07:35:00.798 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:00.796+0000 7f0e32ffd700 1 -- 172.21.15.177:0/4136849185 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f0e34024020 con 0x7f0e3c103d20 2024-03-07T07:35:00.798 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:00.796+0000 7f0e32ffd700 1 --2- 172.21.15.177:0/4136849185 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f0e280a1e10 0x7f0e280a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:35:00.798 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:00.796+0000 7f0e32ffd700 1 -- 172.21.15.177:0/4136849185 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 3698+0+0 (secure 0 0 0) 0x7f0e3400f020 con 0x7f0e3c103d20 2024-03-07T07:35:00.798 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:00.796+0000 7f0e41d58700 1 --2- 172.21.15.177:0/4136849185 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f0e280a1e10 0x7f0e280a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:35:00.798 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:00.797+0000 7f0e41d58700 1 --2- 172.21.15.177:0/4136849185 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f0e280a1e10 0x7f0e280a42d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c1031e0 tx=0x7f0e3800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:35:00.804 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:00.804+0000 7f0e32ffd700 1 -- 172.21.15.177:0/4136849185 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f0e3409cb00 con 0x7f0e3c103d20 2024-03-07T07:35:01.007 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:01.006+0000 7f0e43fbc700 1 -- 172.21.15.177:0/4136849185 --> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e3c062590 con 0x7f0e280a1e10 2024-03-07T07:35:01.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:01 smithi178 ceph-mon[36650]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-07T07:35:01.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:01 smithi178 ceph-mon[36650]: osdmap e27: 4 total, 4 up, 4 in 2024-03-07T07:35:01.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:01 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-07T07:35:01.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:01 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-07T07:35:01.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:01 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:35:01.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:01 smithi177 ceph-mon[36578]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-07T07:35:01.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:01 smithi177 ceph-mon[36578]: osdmap e27: 4 total, 4 up, 4 in 2024-03-07T07:35:01.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:01 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-07T07:35:01.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:01 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-07T07:35:01.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:01 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:35:01.680 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:01 smithi160 ceph-mon[31963]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-07T07:35:01.680 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:01 smithi160 ceph-mon[31963]: osdmap e27: 4 total, 4 up, 4 in 2024-03-07T07:35:01.680 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:01 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-07T07:35:01.680 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:01 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-07T07:35:01.681 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:01 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:35:02.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:02 smithi178 ceph-mon[36650]: from='client.14325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:35:02.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:02 smithi178 ceph-mon[36650]: osdmap e28: 4 total, 4 up, 4 in 2024-03-07T07:35:02.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:02 smithi177 ceph-mon[36578]: from='client.14325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:35:02.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:02 smithi177 ceph-mon[36578]: osdmap e28: 4 total, 4 up, 4 in 2024-03-07T07:35:02.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:02 smithi160 ceph-mon[31963]: from='client.14325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:35:02.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:02 smithi160 ceph-mon[31963]: osdmap e28: 4 total, 4 up, 4 in 2024-03-07T07:35:03.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:03 smithi178 ceph-mon[36650]: pgmap v105: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-07T07:35:03.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:03 smithi178 ceph-mon[36650]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-03-07T07:35:03.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:03 smithi178 ceph-mon[36650]: osdmap e29: 4 total, 4 up, 4 in 2024-03-07T07:35:03.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:03 smithi177 ceph-mon[36578]: pgmap v105: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-07T07:35:03.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:03 smithi177 ceph-mon[36578]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-03-07T07:35:03.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:03 smithi177 ceph-mon[36578]: osdmap e29: 4 total, 4 up, 4 in 2024-03-07T07:35:03.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:03 smithi160 ceph-mon[31963]: pgmap v105: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-07T07:35:03.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:03 smithi160 ceph-mon[31963]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-03-07T07:35:03.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:03 smithi160 ceph-mon[31963]: osdmap e29: 4 total, 4 up, 4 in 2024-03-07T07:35:04.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:04 smithi178 ceph-mon[36650]: from='client.? 172.21.15.177:0/477774863' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b65b104-ac82-4ce3-b6c5-11a399103045"}]: dispatch 2024-03-07T07:35:04.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:04 smithi178 ceph-mon[36650]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b65b104-ac82-4ce3-b6c5-11a399103045"}]: dispatch 2024-03-07T07:35:04.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:04 smithi178 ceph-mon[36650]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2b65b104-ac82-4ce3-b6c5-11a399103045"}]': finished 2024-03-07T07:35:04.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:04 smithi178 ceph-mon[36650]: osdmap e30: 5 total, 4 up, 5 in 2024-03-07T07:35:04.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:04 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-07T07:35:04.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:04 smithi177 ceph-mon[36578]: from='client.? 172.21.15.177:0/477774863' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b65b104-ac82-4ce3-b6c5-11a399103045"}]: dispatch 2024-03-07T07:35:04.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:04 smithi177 ceph-mon[36578]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b65b104-ac82-4ce3-b6c5-11a399103045"}]: dispatch 2024-03-07T07:35:04.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:04 smithi177 ceph-mon[36578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2b65b104-ac82-4ce3-b6c5-11a399103045"}]': finished 2024-03-07T07:35:04.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:04 smithi177 ceph-mon[36578]: osdmap e30: 5 total, 4 up, 5 in 2024-03-07T07:35:04.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:04 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-07T07:35:04.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:04 smithi160 ceph-mon[31963]: from='client.? 172.21.15.177:0/477774863' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b65b104-ac82-4ce3-b6c5-11a399103045"}]: dispatch 2024-03-07T07:35:04.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:04 smithi160 ceph-mon[31963]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b65b104-ac82-4ce3-b6c5-11a399103045"}]: dispatch 2024-03-07T07:35:04.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:04 smithi160 ceph-mon[31963]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2b65b104-ac82-4ce3-b6c5-11a399103045"}]': finished 2024-03-07T07:35:04.679 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:04 smithi160 ceph-mon[31963]: osdmap e30: 5 total, 4 up, 5 in 2024-03-07T07:35:04.679 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:04 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-07T07:35:05.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:05 smithi178 ceph-mon[36650]: pgmap v108: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-07T07:35:05.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:05 smithi178 ceph-mon[36650]: from='client.? 172.21.15.177:0/874781229' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-07T07:35:05.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:05 smithi177 ceph-mon[36578]: pgmap v108: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-07T07:35:05.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:05 smithi177 ceph-mon[36578]: from='client.? 172.21.15.177:0/874781229' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-07T07:35:05.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:05 smithi160 ceph-mon[31963]: pgmap v108: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-07T07:35:05.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:05 smithi160 ceph-mon[31963]: from='client.? 172.21.15.177:0/874781229' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-07T07:35:06.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:06 smithi177 ceph-mon[36578]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-03-07T07:35:06.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:06 smithi177 ceph-mon[36578]: Cluster is now healthy 2024-03-07T07:35:06.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:06 smithi160 ceph-mon[31963]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-03-07T07:35:06.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:06 smithi160 ceph-mon[31963]: Cluster is now healthy 2024-03-07T07:35:06.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:06 smithi178 ceph-mon[36650]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-03-07T07:35:06.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:06 smithi178 ceph-mon[36650]: Cluster is now healthy 2024-03-07T07:35:07.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:07 smithi177 ceph-mon[36578]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-03-07T07:35:07.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:07 smithi160 ceph-mon[31963]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-03-07T07:35:07.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:07 smithi178 ceph-mon[36650]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-03-07T07:35:08.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:08 smithi177 ceph-mon[36578]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-03-07T07:35:08.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:08 smithi160 ceph-mon[31963]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-03-07T07:35:08.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:08 smithi178 ceph-mon[36650]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-03-07T07:35:11.542 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:11 smithi160 ceph-mon[31963]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-07T07:35:11.543 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:11 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-07T07:35:11.543 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:11 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:35:11.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:11 smithi178 ceph-mon[36650]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-07T07:35:11.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:11 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-07T07:35:11.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:11 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:35:11.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:11 smithi177 ceph-mon[36578]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-07T07:35:11.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:11 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-07T07:35:11.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:11 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:35:12.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:12 smithi177 ceph-mon[36578]: Deploying daemon osd.4 on smithi177 2024-03-07T07:35:12.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:12 smithi178 ceph-mon[36650]: Deploying daemon osd.4 on smithi177 2024-03-07T07:35:12.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:12 smithi160 ceph-mon[31963]: Deploying daemon osd.4 on smithi177 2024-03-07T07:35:13.356 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:13 smithi177 ceph-mon[36578]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-07T07:35:13.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:13 smithi178 ceph-mon[36650]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-07T07:35:13.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:13 smithi160 ceph-mon[31963]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-07T07:35:15.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:15 smithi178 ceph-mon[36650]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-03-07T07:35:15.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:15 smithi177 ceph-mon[36578]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-03-07T07:35:15.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:15 smithi160 ceph-mon[31963]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-03-07T07:35:17.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:17 smithi178 ceph-mon[36650]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-07T07:35:17.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:17 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:17.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:17 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:35:17.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:17 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:17.585 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:17 smithi177 ceph-mon[36578]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-07T07:35:17.585 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:17 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:17.585 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:17 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:35:17.585 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:17 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:17.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:17 smithi160 ceph-mon[31963]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-07T07:35:17.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:17 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:17.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:17 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:35:17.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:17 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:19.392 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:19 smithi177 ceph-mon[36578]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-07T07:35:19.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:19 smithi178 ceph-mon[36650]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-07T07:35:19.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:19 smithi160 ceph-mon[31963]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-07T07:35:20.322 INFO:teuthology.orchestra.run.smithi177.stdout:Created osd(s) 4 on host 'smithi177' 2024-03-07T07:35:20.322 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:20.321+0000 7f0e32ffd700 1 -- 172.21.15.177:0/4136849185 <== mgr.14152 v2:172.21.15.160:6800/419630619 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0e3c062590 con 0x7f0e280a1e10 2024-03-07T07:35:20.324 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:20.323+0000 7f0e43fbc700 1 -- 172.21.15.177:0/4136849185 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f0e280a1e10 msgr2=0x7f0e280a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:35:20.324 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:20.323+0000 7f0e43fbc700 1 --2- 172.21.15.177:0/4136849185 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f0e280a1e10 0x7f0e280a42d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c1031e0 tx=0x7f0e3800b410 comp rx=0 tx=0).stop 2024-03-07T07:35:20.324 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:20.323+0000 7f0e43fbc700 1 -- 172.21.15.177:0/4136849185 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e3c103d20 msgr2=0x7f0e3c19f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:35:20.324 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:20.323+0000 7f0e43fbc700 1 --2- 172.21.15.177:0/4136849185 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e3c103d20 0x7f0e3c19f840 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0e3400b0d0 tx=0x7f0e34007d90 comp rx=0 tx=0).stop 2024-03-07T07:35:20.324 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:20.323+0000 7f0e43fbc700 1 -- 172.21.15.177:0/4136849185 shutdown_connections 2024-03-07T07:35:20.324 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:20.323+0000 7f0e43fbc700 1 --2- 172.21.15.177:0/4136849185 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e3c103d20 0x7f0e3c19f840 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:35:20.324 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:20.323+0000 7f0e43fbc700 1 --2- 172.21.15.177:0/4136849185 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f0e280a1e10 0x7f0e280a42d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:35:20.324 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:20.323+0000 7f0e43fbc700 1 --2- 172.21.15.177:0/4136849185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e3c102180 0x7f0e3c198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:35:20.325 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:20.323+0000 7f0e43fbc700 1 --2- 172.21.15.177:0/4136849185 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e3c103380 0x7f0e3c1987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:35:20.325 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:20.323+0000 7f0e43fbc700 1 -- 172.21.15.177:0/4136849185 >> 172.21.15.177:0/4136849185 conn(0x7f0e3c0fd760 msgr2=0x7f0e3c106620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:35:20.325 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:20.323+0000 7f0e43fbc700 1 -- 172.21.15.177:0/4136849185 shutdown_connections 2024-03-07T07:35:20.325 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:35:20.323+0000 7f0e43fbc700 1 -- 172.21.15.177:0/4136849185 wait complete. 2024-03-07T07:35:20.572 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:20 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:20.573 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:20 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:20.573 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:20 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:35:20.573 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:20 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:35:20.573 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:20 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:20.573 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:20 smithi177 ceph-mon[36578]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-07T07:35:20.573 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:20 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:20.573 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:20 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:20.573 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:20 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:35:20.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:20 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:20.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:20 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:20.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:20 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:35:20.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:20 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:35:20.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:20 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:20.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:20 smithi178 ceph-mon[36650]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-07T07:35:20.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:20 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:20.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:20 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:20.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:20 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:35:20.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:20 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:20.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:20 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:20.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:20 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:35:20.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:20 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:35:20.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:20 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:20.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:20 smithi160 ceph-mon[31963]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-07T07:35:20.929 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:20 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:20.929 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:20 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:20.929 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:20 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:35:21.002 DEBUG:teuthology.orchestra.run.smithi177:osd.4> sudo journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@osd.4.service 2024-03-07T07:35:21.005 INFO:tasks.cephadm:Deploying osd.5 on smithi178 with /dev/vg_nvme/lv_3... 2024-03-07T07:35:21.005 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-07T07:35:21.252 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.c/config 2024-03-07T07:35:21.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:21 smithi177 ceph-mon[36578]: from='osd.4 [v2:172.21.15.177:6808/88178553,v1:172.21.15.177:6809/88178553]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-07T07:35:21.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:21 smithi177 ceph-mon[36578]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-07T07:35:21.691 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:21 smithi178 ceph-mon[36650]: from='osd.4 [v2:172.21.15.177:6808/88178553,v1:172.21.15.177:6809/88178553]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-07T07:35:21.691 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:21 smithi178 ceph-mon[36650]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-07T07:35:21.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:21 smithi160 ceph-mon[31963]: from='osd.4 [v2:172.21.15.177:6808/88178553,v1:172.21.15.177:6809/88178553]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-07T07:35:21.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:21 smithi160 ceph-mon[31963]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-07T07:35:22.638 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:22 smithi178 ceph-mon[36650]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-07T07:35:22.638 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:22 smithi178 ceph-mon[36650]: from='osd.4 [v2:172.21.15.177:6808/88178553,v1:172.21.15.177:6809/88178553]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-03-07T07:35:22.638 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:22 smithi178 ceph-mon[36650]: osdmap e31: 5 total, 4 up, 5 in 2024-03-07T07:35:22.638 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:22 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-07T07:35:22.638 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:22 smithi178 ceph-mon[36650]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-03-07T07:35:22.638 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:22 smithi178 ceph-mon[36650]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-07T07:35:22.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:22 smithi177 ceph-mon[36578]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-07T07:35:22.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:22 smithi177 ceph-mon[36578]: from='osd.4 [v2:172.21.15.177:6808/88178553,v1:172.21.15.177:6809/88178553]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-03-07T07:35:22.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:22 smithi177 ceph-mon[36578]: osdmap e31: 5 total, 4 up, 5 in 2024-03-07T07:35:22.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:22 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-07T07:35:22.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:22 smithi177 ceph-mon[36578]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-03-07T07:35:22.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:22 smithi177 ceph-mon[36578]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-07T07:35:22.858 INFO:journalctl@ceph.osd.4.smithi177.stdout:Mar 07 07:35:22 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-4[48119]: 2024-03-07T07:35:22.539+0000 7f263478e700 -1 osd.4 0 waiting for initial osdmap 2024-03-07T07:35:22.858 INFO:journalctl@ceph.osd.4.smithi177.stdout:Mar 07 07:35:22 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-4[48119]: 2024-03-07T07:35:22.547+0000 7f2630581700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-07T07:35:22.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:22 smithi160 ceph-mon[31963]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-07T07:35:22.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:22 smithi160 ceph-mon[31963]: from='osd.4 [v2:172.21.15.177:6808/88178553,v1:172.21.15.177:6809/88178553]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-03-07T07:35:22.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:22 smithi160 ceph-mon[31963]: osdmap e31: 5 total, 4 up, 5 in 2024-03-07T07:35:22.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:22 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-07T07:35:22.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:22 smithi160 ceph-mon[31963]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-03-07T07:35:22.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:22 smithi160 ceph-mon[31963]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-07T07:35:23.223 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-07T07:35:23.223 INFO:teuthology.orchestra.run.smithi178.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-07T07:35:23.223 INFO:teuthology.orchestra.run.smithi178.stderr: stderr: 10+0 records in 2024-03-07T07:35:23.223 INFO:teuthology.orchestra.run.smithi178.stderr:10+0 records out 2024-03-07T07:35:23.223 INFO:teuthology.orchestra.run.smithi178.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00866485 s, 1.2 GB/s 2024-03-07T07:35:23.223 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping successful for: 2024-03-07T07:35:23.743 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:23 smithi178 ceph-mon[36650]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2024-03-07T07:35:23.743 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:23 smithi178 ceph-mon[36650]: osdmap e32: 5 total, 4 up, 5 in 2024-03-07T07:35:23.743 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:23 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-07T07:35:23.743 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:23 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-07T07:35:23.743 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:23 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-07T07:35:23.770 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi178:vg_nvme/lv_3 2024-03-07T07:35:23.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:23 smithi177 ceph-mon[36578]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2024-03-07T07:35:23.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:23 smithi177 ceph-mon[36578]: osdmap e32: 5 total, 4 up, 5 in 2024-03-07T07:35:23.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:23 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-07T07:35:23.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:23 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-07T07:35:23.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:23 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-07T07:35:23.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:23 smithi160 ceph-mon[31963]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2024-03-07T07:35:23.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:23 smithi160 ceph-mon[31963]: osdmap e32: 5 total, 4 up, 5 in 2024-03-07T07:35:23.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:23 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-07T07:35:23.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:23 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-07T07:35:23.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:23 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-07T07:35:24.024 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.c/config 2024-03-07T07:35:24.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:24 smithi178 ceph-mon[36650]: purged_snaps scrub starts 2024-03-07T07:35:24.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:24 smithi178 ceph-mon[36650]: purged_snaps scrub ok 2024-03-07T07:35:24.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:24 smithi178 ceph-mon[36650]: osd.4 [v2:172.21.15.177:6808/88178553,v1:172.21.15.177:6809/88178553] boot 2024-03-07T07:35:24.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:24 smithi178 ceph-mon[36650]: osdmap e33: 5 total, 5 up, 5 in 2024-03-07T07:35:24.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:24 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-07T07:35:24.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:24 smithi178 ceph-mon[36650]: Detected new or changed devices on smithi177 2024-03-07T07:35:24.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:24 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:24.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:24 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:24.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:24 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:35:24.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:24 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:35:24.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:24 smithi178 ceph-mon[36650]: Adjusting osd_memory_target on smithi177 to 8540M 2024-03-07T07:35:24.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:24 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:24.808 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:24 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:35:24.808 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:24 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:35:24.808 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:24 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:24.808 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:24 smithi178 ceph-mon[36650]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-07T07:35:24.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:24 smithi177 ceph-mon[36578]: purged_snaps scrub starts 2024-03-07T07:35:24.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:24 smithi177 ceph-mon[36578]: purged_snaps scrub ok 2024-03-07T07:35:24.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:24 smithi177 ceph-mon[36578]: osd.4 [v2:172.21.15.177:6808/88178553,v1:172.21.15.177:6809/88178553] boot 2024-03-07T07:35:24.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:24 smithi177 ceph-mon[36578]: osdmap e33: 5 total, 5 up, 5 in 2024-03-07T07:35:24.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:24 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-07T07:35:24.858 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:24 smithi177 ceph-mon[36578]: Detected new or changed devices on smithi177 2024-03-07T07:35:24.858 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:24 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:24.858 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:24 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:24.858 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:24 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:35:24.858 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:24 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:35:24.858 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:24 smithi177 ceph-mon[36578]: Adjusting osd_memory_target on smithi177 to 8540M 2024-03-07T07:35:24.858 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:24 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:24.858 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:24 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:35:24.858 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:24 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:35:24.858 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:24 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:24.858 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:24 smithi177 ceph-mon[36578]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-07T07:35:24.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:24 smithi160 ceph-mon[31963]: purged_snaps scrub starts 2024-03-07T07:35:24.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:24 smithi160 ceph-mon[31963]: purged_snaps scrub ok 2024-03-07T07:35:24.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:24 smithi160 ceph-mon[31963]: osd.4 [v2:172.21.15.177:6808/88178553,v1:172.21.15.177:6809/88178553] boot 2024-03-07T07:35:24.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:24 smithi160 ceph-mon[31963]: osdmap e33: 5 total, 5 up, 5 in 2024-03-07T07:35:24.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:24 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-07T07:35:24.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:24 smithi160 ceph-mon[31963]: Detected new or changed devices on smithi177 2024-03-07T07:35:24.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:24 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:24.929 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:24 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:24.929 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:24 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:35:24.929 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:24 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:35:24.929 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:24 smithi160 ceph-mon[31963]: Adjusting osd_memory_target on smithi177 to 8540M 2024-03-07T07:35:24.929 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:24 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:24.929 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:24 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:35:24.929 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:24 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:35:24.929 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:24 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:24.929 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:24 smithi160 ceph-mon[31963]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-07T07:35:25.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:25.614+0000 7f36a010f700 1 -- 172.21.15.178:0/3152855009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3698103d20 msgr2=0x7f369810a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:35:25.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:25.614+0000 7f36a010f700 1 --2- 172.21.15.178:0/3152855009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3698103d20 0x7f369810a5d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f369000b0a0 tx=0x7f369000b3b0 comp rx=0 tx=0).stop 2024-03-07T07:35:25.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:25.614+0000 7f36a010f700 1 -- 172.21.15.178:0/3152855009 shutdown_connections 2024-03-07T07:35:25.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:25.614+0000 7f36a010f700 1 --2- 172.21.15.178:0/3152855009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3698103d20 0x7f369810a5d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:35:25.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:25.614+0000 7f36a010f700 1 --2- 172.21.15.178:0/3152855009 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3698102180 0x7f36981025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:35:25.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:25.614+0000 7f36a010f700 1 --2- 172.21.15.178:0/3152855009 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3698103380 0x7f36981037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:35:25.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:25.614+0000 7f36a010f700 1 -- 172.21.15.178:0/3152855009 >> 172.21.15.178:0/3152855009 conn(0x7f36980fd760 msgr2=0x7f36980ffb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:35:25.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:25.615+0000 7f36a010f700 1 -- 172.21.15.178:0/3152855009 shutdown_connections 2024-03-07T07:35:25.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:25.615+0000 7f36a010f700 1 -- 172.21.15.178:0/3152855009 wait complete. 2024-03-07T07:35:25.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:25.615+0000 7f36a010f700 1 Processor -- start 2024-03-07T07:35:25.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:25.615+0000 7f36a010f700 1 -- start start 2024-03-07T07:35:25.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:25.615+0000 7f36a010f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3698102180 0x7f369819a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:35:25.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:25.615+0000 7f36a010f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3698103380 0x7f369819aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:35:25.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:25.615+0000 7f36a010f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3698103d20 0x7f36981a1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:35:25.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:25.615+0000 7f36a010f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f369810ddd0 con 0x7f3698103d20 2024-03-07T07:35:25.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:25.615+0000 7f36a010f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f369810dc50 con 0x7f3698102180 2024-03-07T07:35:25.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:25.615+0000 7f36a010f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f369810df50 con 0x7f3698103380 2024-03-07T07:35:25.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:25.615+0000 7f369d6aa700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3698103380 0x7f369819aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:35:25.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:25.615+0000 7f369d6aa700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3698103380 0x7f369819aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:33502/0 (socket says 172.21.15.178:33502) 2024-03-07T07:35:25.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:25.615+0000 7f369d6aa700 1 -- 172.21.15.178:0/3956535735 learned_addr learned my addr 172.21.15.178:0/3956535735 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-03-07T07:35:25.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:25.615+0000 7f369deab700 1 --2- 172.21.15.178:0/3956535735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3698102180 0x7f369819a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:35:25.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:25.616+0000 7f369d6aa700 1 -- 172.21.15.178:0/3956535735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3698102180 msgr2=0x7f369819a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:35:25.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:25.616+0000 7f369d6aa700 1 --2- 172.21.15.178:0/3956535735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3698102180 0x7f369819a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:35:25.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:25.616+0000 7f369d6aa700 1 -- 172.21.15.178:0/3956535735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3698103d20 msgr2=0x7f36981a1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:35:25.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:25.616+0000 7f369d6aa700 1 --2- 172.21.15.178:0/3956535735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3698103d20 0x7f36981a1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:35:25.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:25.616+0000 7f369d6aa700 1 -- 172.21.15.178:0/3956535735 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36981a22c0 con 0x7f3698103380 2024-03-07T07:35:25.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:25.616+0000 7f369d6aa700 1 --2- 172.21.15.178:0/3956535735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3698103380 0x7f369819aba0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f369400b560 tx=0x7f369400b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:35:25.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:25.616+0000 7f368effd700 1 -- 172.21.15.178:0/3956535735 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3694015020 con 0x7f3698103380 2024-03-07T07:35:25.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:25.616+0000 7f36a010f700 1 -- 172.21.15.178:0/3956535735 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36981a25b0 con 0x7f3698103380 2024-03-07T07:35:25.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:25.616+0000 7f368effd700 1 -- 172.21.15.178:0/3956535735 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3694011d10 con 0x7f3698103380 2024-03-07T07:35:25.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:25.616+0000 7f368effd700 1 -- 172.21.15.178:0/3956535735 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3694010e50 con 0x7f3698103380 2024-03-07T07:35:25.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:25.616+0000 7f36a010f700 1 -- 172.21.15.178:0/3956535735 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36981a2b40 con 0x7f3698103380 2024-03-07T07:35:25.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:25.617+0000 7f36a010f700 1 -- 172.21.15.178:0/3956535735 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3698068e10 con 0x7f3698103380 2024-03-07T07:35:25.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:25.618+0000 7f368effd700 1 -- 172.21.15.178:0/3956535735 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f3694018020 con 0x7f3698103380 2024-03-07T07:35:25.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:25.619+0000 7f368effd700 1 --2- 172.21.15.178:0/3956535735 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f36840a1fb0 0x7f36840a4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:35:25.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:25.619+0000 7f368effd700 1 -- 172.21.15.178:0/3956535735 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f36940cc990 con 0x7f3698103380 2024-03-07T07:35:25.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:25.619+0000 7f369deab700 1 --2- 172.21.15.178:0/3956535735 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f36840a1fb0 0x7f36840a4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:35:25.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:25.619+0000 7f369deab700 1 --2- 172.21.15.178:0/3956535735 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f36840a1fb0 0x7f36840a4470 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f36981031e0 tx=0x7f368800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:35:25.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:25.622+0000 7f368effd700 1 -- 172.21.15.178:0/3956535735 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f369401f020 con 0x7f3698103380 2024-03-07T07:35:25.827 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:25.826+0000 7f36a010f700 1 -- 172.21.15.178:0/3956535735 --> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f3698075aa0 con 0x7f36840a1fb0 2024-03-07T07:35:26.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:25 smithi178 ceph-mon[36650]: osdmap e34: 5 total, 5 up, 5 in 2024-03-07T07:35:26.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:25 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-07T07:35:26.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:25 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-07T07:35:26.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:25 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:35:26.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:25 smithi177 ceph-mon[36578]: osdmap e34: 5 total, 5 up, 5 in 2024-03-07T07:35:26.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:25 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-07T07:35:26.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:25 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-07T07:35:26.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:25 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:35:26.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:25 smithi160 ceph-mon[31963]: osdmap e34: 5 total, 5 up, 5 in 2024-03-07T07:35:26.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:25 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-07T07:35:26.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:25 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-07T07:35:26.429 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:25 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:35:27.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:26 smithi178 ceph-mon[36650]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:35:27.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:26 smithi178 ceph-mon[36650]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-07T07:35:27.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:26 smithi177 ceph-mon[36578]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:35:27.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:26 smithi177 ceph-mon[36578]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-07T07:35:27.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:26 smithi160 ceph-mon[31963]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:35:27.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:26 smithi160 ceph-mon[31963]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-07T07:35:29.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:29 smithi178 ceph-mon[36650]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-07T07:35:29.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:29 smithi178 ceph-mon[36650]: from='client.? 172.21.15.178:0/2563775078' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "30a1e714-08ff-44e8-9264-9ef164604b60"}]: dispatch 2024-03-07T07:35:29.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:29 smithi178 ceph-mon[36650]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "30a1e714-08ff-44e8-9264-9ef164604b60"}]: dispatch 2024-03-07T07:35:29.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:29 smithi178 ceph-mon[36650]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "30a1e714-08ff-44e8-9264-9ef164604b60"}]': finished 2024-03-07T07:35:29.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:29 smithi178 ceph-mon[36650]: osdmap e35: 6 total, 5 up, 6 in 2024-03-07T07:35:29.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:29 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-07T07:35:29.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:29 smithi178 ceph-mon[36650]: from='client.? 172.21.15.178:0/3015180181' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-07T07:35:29.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:29 smithi177 ceph-mon[36578]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-07T07:35:29.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:29 smithi177 ceph-mon[36578]: from='client.? 172.21.15.178:0/2563775078' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "30a1e714-08ff-44e8-9264-9ef164604b60"}]: dispatch 2024-03-07T07:35:29.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:29 smithi177 ceph-mon[36578]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "30a1e714-08ff-44e8-9264-9ef164604b60"}]: dispatch 2024-03-07T07:35:29.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:29 smithi177 ceph-mon[36578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "30a1e714-08ff-44e8-9264-9ef164604b60"}]': finished 2024-03-07T07:35:29.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:29 smithi177 ceph-mon[36578]: osdmap e35: 6 total, 5 up, 6 in 2024-03-07T07:35:29.608 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:29 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-07T07:35:29.608 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:29 smithi177 ceph-mon[36578]: from='client.? 172.21.15.178:0/3015180181' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-07T07:35:29.680 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:29 smithi160 ceph-mon[31963]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-07T07:35:29.681 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:29 smithi160 ceph-mon[31963]: from='client.? 172.21.15.178:0/2563775078' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "30a1e714-08ff-44e8-9264-9ef164604b60"}]: dispatch 2024-03-07T07:35:29.681 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:29 smithi160 ceph-mon[31963]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "30a1e714-08ff-44e8-9264-9ef164604b60"}]: dispatch 2024-03-07T07:35:29.681 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:29 smithi160 ceph-mon[31963]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "30a1e714-08ff-44e8-9264-9ef164604b60"}]': finished 2024-03-07T07:35:29.681 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:29 smithi160 ceph-mon[31963]: osdmap e35: 6 total, 5 up, 6 in 2024-03-07T07:35:29.681 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:29 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-07T07:35:29.681 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:29 smithi160 ceph-mon[31963]: from='client.? 172.21.15.178:0/3015180181' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-07T07:35:31.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:31 smithi178 ceph-mon[36650]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-07T07:35:31.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:31 smithi177 ceph-mon[36578]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-07T07:35:31.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:31 smithi160 ceph-mon[31963]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-07T07:35:33.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:33 smithi178 ceph-mon[36650]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-07T07:35:33.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:33 smithi177 ceph-mon[36578]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-07T07:35:33.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:33 smithi160 ceph-mon[31963]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-07T07:35:35.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:35 smithi178 ceph-mon[36650]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-07T07:35:35.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:35 smithi177 ceph-mon[36578]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-07T07:35:35.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:35 smithi160 ceph-mon[31963]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-07T07:35:36.551 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:36 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-07T07:35:36.551 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:36 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:35:36.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:36 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-07T07:35:36.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:36 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:35:36.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:36 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-07T07:35:36.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:36 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:35:37.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:37 smithi178 ceph-mon[36650]: Deploying daemon osd.5 on smithi178 2024-03-07T07:35:37.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:37 smithi178 ceph-mon[36650]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-07T07:35:37.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:37 smithi177 ceph-mon[36578]: Deploying daemon osd.5 on smithi178 2024-03-07T07:35:37.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:37 smithi177 ceph-mon[36578]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-07T07:35:37.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:37 smithi160 ceph-mon[31963]: Deploying daemon osd.5 on smithi178 2024-03-07T07:35:37.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:37 smithi160 ceph-mon[31963]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-07T07:35:39.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:39 smithi178 ceph-mon[36650]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-07T07:35:39.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:39 smithi177 ceph-mon[36578]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-07T07:35:39.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:39 smithi160 ceph-mon[31963]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-07T07:35:41.553 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:41 smithi178 ceph-mon[36650]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-07T07:35:41.554 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:41 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:41.554 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:41 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:35:41.554 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:41 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:41.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:41 smithi177 ceph-mon[36578]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-07T07:35:41.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:41 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:41.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:41 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:35:41.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:41 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:41.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:41 smithi160 ceph-mon[31963]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-07T07:35:41.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:41 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:41.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:41 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:35:41.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:41 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:43.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:43 smithi178 ceph-mon[36650]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-07T07:35:43.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:43 smithi177 ceph-mon[36578]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-07T07:35:43.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:43 smithi160 ceph-mon[31963]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-07T07:35:44.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:44.940+0000 7f368effd700 1 -- 172.21.15.178:0/3956535735 <== mgr.14152 v2:172.21.15.160:6800/419630619 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f3698075aa0 con 0x7f36840a1fb0 2024-03-07T07:35:44.941 INFO:teuthology.orchestra.run.smithi178.stdout:Created osd(s) 5 on host 'smithi178' 2024-03-07T07:35:44.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:44.942+0000 7f36a010f700 1 -- 172.21.15.178:0/3956535735 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f36840a1fb0 msgr2=0x7f36840a4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:35:44.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:44.942+0000 7f36a010f700 1 --2- 172.21.15.178:0/3956535735 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f36840a1fb0 0x7f36840a4470 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f36981031e0 tx=0x7f368800a380 comp rx=0 tx=0).stop 2024-03-07T07:35:44.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:44.943+0000 7f36a010f700 1 -- 172.21.15.178:0/3956535735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3698103380 msgr2=0x7f369819aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:35:44.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:44.943+0000 7f36a010f700 1 --2- 172.21.15.178:0/3956535735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3698103380 0x7f369819aba0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f369400b560 tx=0x7f369400b870 comp rx=0 tx=0).stop 2024-03-07T07:35:44.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:44.943+0000 7f36a010f700 1 -- 172.21.15.178:0/3956535735 shutdown_connections 2024-03-07T07:35:44.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:44.943+0000 7f36a010f700 1 --2- 172.21.15.178:0/3956535735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3698103d20 0x7f36981a1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:35:44.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:44.943+0000 7f36a010f700 1 --2- 172.21.15.178:0/3956535735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3698102180 0x7f369819a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:35:44.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:44.943+0000 7f36a010f700 1 --2- 172.21.15.178:0/3956535735 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f36840a1fb0 0x7f36840a4470 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:35:44.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:44.943+0000 7f36a010f700 1 --2- 172.21.15.178:0/3956535735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3698103380 0x7f369819aba0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:35:44.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:44.943+0000 7f36a010f700 1 -- 172.21.15.178:0/3956535735 >> 172.21.15.178:0/3956535735 conn(0x7f36980fd760 msgr2=0x7f3698074380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:35:44.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:44.943+0000 7f36a010f700 1 -- 172.21.15.178:0/3956535735 shutdown_connections 2024-03-07T07:35:44.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:35:44.943+0000 7f36a010f700 1 -- 172.21.15.178:0/3956535735 wait complete. 2024-03-07T07:35:45.268 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:45 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:45.268 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:45 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:45.268 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:45 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:35:45.268 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:45 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:35:45.268 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:45 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:45.268 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:45 smithi178 ceph-mon[36650]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-07T07:35:45.268 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:45 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:45.268 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:45 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:35:45.268 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:45 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:45.268 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:45 smithi178 ceph-mon[36650]: from='osd.5 [v2:172.21.15.178:6808/964732891,v1:172.21.15.178:6809/964732891]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-07T07:35:45.269 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:45 smithi178 ceph-mon[36650]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-07T07:35:45.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:45 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:45.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:45 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:45.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:45 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:35:45.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:45 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:35:45.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:45 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:45.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:45 smithi177 ceph-mon[36578]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-07T07:35:45.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:45 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:45.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:45 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:35:45.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:45 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:45.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:45 smithi177 ceph-mon[36578]: from='osd.5 [v2:172.21.15.178:6808/964732891,v1:172.21.15.178:6809/964732891]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-07T07:35:45.358 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:45 smithi177 ceph-mon[36578]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-07T07:35:45.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:45 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:45.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:45 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:45.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:45 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:35:45.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:45 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:35:45.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:45 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:45.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:45 smithi160 ceph-mon[31963]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-07T07:35:45.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:45 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:45.429 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:45 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:35:45.429 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:45 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:45.429 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:45 smithi160 ceph-mon[31963]: from='osd.5 [v2:172.21.15.178:6808/964732891,v1:172.21.15.178:6809/964732891]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-07T07:35:45.429 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:45 smithi160 ceph-mon[31963]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-07T07:35:45.611 DEBUG:teuthology.orchestra.run.smithi178:osd.5> sudo journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@osd.5.service 2024-03-07T07:35:45.614 INFO:tasks.cephadm:Deploying osd.6 on smithi160 with /dev/vg_nvme/lv_2... 2024-03-07T07:35:45.614 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-07T07:35:45.864 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:35:47.138 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:46 smithi178 ceph-mon[36650]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-07T07:35:47.139 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:46 smithi178 ceph-mon[36650]: osdmap e36: 6 total, 5 up, 6 in 2024-03-07T07:35:47.139 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:46 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-07T07:35:47.139 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:46 smithi178 ceph-mon[36650]: from='osd.5 [v2:172.21.15.178:6808/964732891,v1:172.21.15.178:6809/964732891]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-03-07T07:35:47.139 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:46 smithi178 ceph-mon[36650]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-03-07T07:35:47.139 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:46 smithi178 ceph-mon[36650]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-07T07:35:47.139 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:35:46 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-5[48766]: 2024-03-07T07:35:46.923+0000 7f38c4f02700 -1 osd.5 0 waiting for initial osdmap 2024-03-07T07:35:47.139 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:35:46 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-5[48766]: 2024-03-07T07:35:46.929+0000 7f38c0cf5700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-07T07:35:47.177 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:46 smithi160 ceph-mon[31963]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-07T07:35:47.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:46 smithi160 ceph-mon[31963]: osdmap e36: 6 total, 5 up, 6 in 2024-03-07T07:35:47.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:46 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-07T07:35:47.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:46 smithi160 ceph-mon[31963]: from='osd.5 [v2:172.21.15.178:6808/964732891,v1:172.21.15.178:6809/964732891]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-03-07T07:35:47.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:46 smithi160 ceph-mon[31963]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-03-07T07:35:47.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:46 smithi160 ceph-mon[31963]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-07T07:35:47.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:46 smithi177 ceph-mon[36578]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-07T07:35:47.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:46 smithi177 ceph-mon[36578]: osdmap e36: 6 total, 5 up, 6 in 2024-03-07T07:35:47.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:46 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-07T07:35:47.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:46 smithi177 ceph-mon[36578]: from='osd.5 [v2:172.21.15.178:6808/964732891,v1:172.21.15.178:6809/964732891]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-03-07T07:35:47.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:46 smithi177 ceph-mon[36578]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-03-07T07:35:47.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:46 smithi177 ceph-mon[36578]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-07T07:35:47.888 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-07T07:35:47.889 INFO:teuthology.orchestra.run.smithi160.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-07T07:35:47.889 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 10+0 records in 2024-03-07T07:35:47.889 INFO:teuthology.orchestra.run.smithi160.stderr:10+0 records out 2024-03-07T07:35:47.889 INFO:teuthology.orchestra.run.smithi160.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011301 s, 928 MB/s 2024-03-07T07:35:47.889 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping successful for: 2024-03-07T07:35:48.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:47 smithi160 ceph-mon[31963]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-03-07T07:35:48.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:47 smithi160 ceph-mon[31963]: osdmap e37: 6 total, 5 up, 6 in 2024-03-07T07:35:48.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:47 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-07T07:35:48.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:47 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-07T07:35:48.274 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:47 smithi178 ceph-mon[36650]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-03-07T07:35:48.274 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:47 smithi178 ceph-mon[36650]: osdmap e37: 6 total, 5 up, 6 in 2024-03-07T07:35:48.274 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:47 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-07T07:35:48.274 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:47 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-07T07:35:48.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:47 smithi177 ceph-mon[36578]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-03-07T07:35:48.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:47 smithi177 ceph-mon[36578]: osdmap e37: 6 total, 5 up, 6 in 2024-03-07T07:35:48.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:47 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-07T07:35:48.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:47 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-07T07:35:48.578 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi160:vg_nvme/lv_2 2024-03-07T07:35:48.829 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:35:49.152 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:48 smithi160 ceph-mon[31963]: purged_snaps scrub starts 2024-03-07T07:35:49.153 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:48 smithi160 ceph-mon[31963]: purged_snaps scrub ok 2024-03-07T07:35:49.153 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:48 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-07T07:35:49.153 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:48 smithi160 ceph-mon[31963]: osd.5 [v2:172.21.15.178:6808/964732891,v1:172.21.15.178:6809/964732891] boot 2024-03-07T07:35:49.153 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:48 smithi160 ceph-mon[31963]: osdmap e38: 6 total, 6 up, 6 in 2024-03-07T07:35:49.153 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:48 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-07T07:35:49.153 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:48 smithi160 ceph-mon[31963]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-07T07:35:49.153 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:48 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:49.153 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:48 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:49.153 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:48 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:35:49.153 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:48 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:35:49.153 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:48 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:49.153 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:48 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:35:49.154 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:48 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:35:49.154 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:48 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:49.258 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:48 smithi177 ceph-mon[36578]: purged_snaps scrub starts 2024-03-07T07:35:49.258 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:48 smithi177 ceph-mon[36578]: purged_snaps scrub ok 2024-03-07T07:35:49.258 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:48 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-07T07:35:49.259 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:48 smithi177 ceph-mon[36578]: osd.5 [v2:172.21.15.178:6808/964732891,v1:172.21.15.178:6809/964732891] boot 2024-03-07T07:35:49.259 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:48 smithi177 ceph-mon[36578]: osdmap e38: 6 total, 6 up, 6 in 2024-03-07T07:35:49.259 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:48 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-07T07:35:49.259 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:48 smithi177 ceph-mon[36578]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-07T07:35:49.259 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:48 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:49.259 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:48 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:49.259 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:48 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:35:49.259 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:48 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:35:49.259 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:48 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:49.259 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:48 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:35:49.259 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:48 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:35:49.259 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:48 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:49.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:48 smithi178 ceph-mon[36650]: purged_snaps scrub starts 2024-03-07T07:35:49.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:48 smithi178 ceph-mon[36650]: purged_snaps scrub ok 2024-03-07T07:35:49.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:48 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-07T07:35:49.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:48 smithi178 ceph-mon[36650]: osd.5 [v2:172.21.15.178:6808/964732891,v1:172.21.15.178:6809/964732891] boot 2024-03-07T07:35:49.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:48 smithi178 ceph-mon[36650]: osdmap e38: 6 total, 6 up, 6 in 2024-03-07T07:35:49.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:48 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-07T07:35:49.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:48 smithi178 ceph-mon[36650]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-07T07:35:49.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:48 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:49.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:48 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:49.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:48 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:35:49.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:48 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:35:49.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:48 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:49.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:48 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:35:49.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:48 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:35:49.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:48 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:35:50.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:49 smithi160 ceph-mon[31963]: Detected new or changed devices on smithi178 2024-03-07T07:35:50.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:49 smithi160 ceph-mon[31963]: Adjusting osd_memory_target on smithi178 to 8540M 2024-03-07T07:35:50.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:49 smithi160 ceph-mon[31963]: osdmap e39: 6 total, 6 up, 6 in 2024-03-07T07:35:50.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:49 smithi178 ceph-mon[36650]: Detected new or changed devices on smithi178 2024-03-07T07:35:50.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:49 smithi178 ceph-mon[36650]: Adjusting osd_memory_target on smithi178 to 8540M 2024-03-07T07:35:50.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:49 smithi178 ceph-mon[36650]: osdmap e39: 6 total, 6 up, 6 in 2024-03-07T07:35:50.356 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:49 smithi177 ceph-mon[36578]: Detected new or changed devices on smithi178 2024-03-07T07:35:50.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:49 smithi177 ceph-mon[36578]: Adjusting osd_memory_target on smithi178 to 8540M 2024-03-07T07:35:50.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:49 smithi177 ceph-mon[36578]: osdmap e39: 6 total, 6 up, 6 in 2024-03-07T07:35:50.443 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:35:50.437+0000 7fea09c2b700 1 -- 172.21.15.160:0/816553887 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea04103d20 msgr2=0x7fea0410a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:35:50.443 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:35:50.437+0000 7fea09c2b700 1 --2- 172.21.15.160:0/816553887 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea04103d20 0x7fea0410a5d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc00b0d0 tx=0x7fe9fc00b3e0 comp rx=0 tx=0).stop 2024-03-07T07:35:50.443 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:35:50.437+0000 7fea09c2b700 1 -- 172.21.15.160:0/816553887 shutdown_connections 2024-03-07T07:35:50.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:35:50.437+0000 7fea09c2b700 1 --2- 172.21.15.160:0/816553887 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea04103d20 0x7fea0410a5d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:35:50.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:35:50.437+0000 7fea09c2b700 1 --2- 172.21.15.160:0/816553887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea04102180 0x7fea041025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:35:50.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:35:50.437+0000 7fea09c2b700 1 --2- 172.21.15.160:0/816553887 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea04103380 0x7fea041037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:35:50.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:35:50.437+0000 7fea09c2b700 1 -- 172.21.15.160:0/816553887 >> 172.21.15.160:0/816553887 conn(0x7fea040fd760 msgr2=0x7fea040ffb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:35:50.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:35:50.438+0000 7fea09c2b700 1 -- 172.21.15.160:0/816553887 shutdown_connections 2024-03-07T07:35:50.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:35:50.438+0000 7fea09c2b700 1 -- 172.21.15.160:0/816553887 wait complete. 2024-03-07T07:35:50.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:35:50.438+0000 7fea09c2b700 1 Processor -- start 2024-03-07T07:35:50.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:35:50.438+0000 7fea09c2b700 1 -- start start 2024-03-07T07:35:50.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:35:50.438+0000 7fea09c2b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea04102180 0x7fea04198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:35:50.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:35:50.438+0000 7fea09c2b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea04103380 0x7fea04198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:35:50.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:35:50.438+0000 7fea09c2b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea04103d20 0x7fea0419f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:35:50.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:35:50.438+0000 7fea09c2b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0410dd70 con 0x7fea04103d20 2024-03-07T07:35:50.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:35:50.438+0000 7fea09c2b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0410dbf0 con 0x7fea04102180 2024-03-07T07:35:50.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:35:50.438+0000 7fea09c2b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0410def0 con 0x7fea04103380 2024-03-07T07:35:50.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:35:50.439+0000 7fea037fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea04102180 0x7fea04198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:35:50.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:35:50.439+0000 7fea03fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea04103d20 0x7fea0419f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:35:50.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:35:50.439+0000 7fea02ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea04103380 0x7fea04198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:35:50.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:35:50.439+0000 7fea037fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea04102180 0x7fea04198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.160:56686/0 (socket says 172.21.15.160:56686) 2024-03-07T07:35:50.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:35:50.439+0000 7fea037fe700 1 -- 172.21.15.160:0/2238518404 learned_addr learned my addr 172.21.15.160:0/2238518404 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:35:50.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:35:50.439+0000 7fea02ffd700 1 -- 172.21.15.160:0/2238518404 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea04102180 msgr2=0x7fea04198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:35:50.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:35:50.439+0000 7fea02ffd700 1 --2- 172.21.15.160:0/2238518404 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea04102180 0x7fea04198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:35:50.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:35:50.439+0000 7fea02ffd700 1 -- 172.21.15.160:0/2238518404 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea04103d20 msgr2=0x7fea0419f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:35:50.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:35:50.439+0000 7fea02ffd700 1 --2- 172.21.15.160:0/2238518404 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea04103d20 0x7fea0419f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:35:50.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:35:50.439+0000 7fea02ffd700 1 -- 172.21.15.160:0/2238518404 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea0419ff80 con 0x7fea04103380 2024-03-07T07:35:50.446 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:35:50.439+0000 7fea03fff700 1 --2- 172.21.15.160:0/2238518404 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea04103d20 0x7fea0419f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:35:50.446 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:35:50.439+0000 7fea02ffd700 1 --2- 172.21.15.160:0/2238518404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea04103380 0x7fea04198860 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe9f800ea20 tx=0x7fe9f800ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:35:50.446 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:35:50.439+0000 7fea00ff9700 1 -- 172.21.15.160:0/2238518404 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f800cbd0 con 0x7fea04103380 2024-03-07T07:35:50.446 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:35:50.439+0000 7fea09c2b700 1 -- 172.21.15.160:0/2238518404 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea041a0270 con 0x7fea04103380 2024-03-07T07:35:50.446 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:35:50.439+0000 7fea00ff9700 1 -- 172.21.15.160:0/2238518404 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe9f8004500 con 0x7fea04103380 2024-03-07T07:35:50.446 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:35:50.439+0000 7fea00ff9700 1 -- 172.21.15.160:0/2238518404 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f8005020 con 0x7fea04103380 2024-03-07T07:35:50.446 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:35:50.439+0000 7fea09c2b700 1 -- 172.21.15.160:0/2238518404 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea041a07b0 con 0x7fea04103380 2024-03-07T07:35:50.446 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:35:50.440+0000 7fea09c2b700 1 -- 172.21.15.160:0/2238518404 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea04068e10 con 0x7fea04103380 2024-03-07T07:35:50.446 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:35:50.441+0000 7fea00ff9700 1 -- 172.21.15.160:0/2238518404 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7fe9f80051c0 con 0x7fea04103380 2024-03-07T07:35:50.446 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:35:50.442+0000 7fea00ff9700 1 --2- 172.21.15.160:0/2238518404 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fe9ec0a1e10 0x7fe9ec0a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:35:50.446 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:35:50.442+0000 7fea037fe700 1 --2- 172.21.15.160:0/2238518404 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fe9ec0a1e10 0x7fe9ec0a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:35:50.446 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:35:50.442+0000 7fea00ff9700 1 -- 172.21.15.160:0/2238518404 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4562+0+0 (secure 0 0 0) 0x7fe9f80ccde0 con 0x7fea04103380 2024-03-07T07:35:50.446 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:35:50.442+0000 7fea037fe700 1 --2- 172.21.15.160:0/2238518404 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fe9ec0a1e10 0x7fe9ec0a42d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4007cf0 tx=0x7fe9f4007480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:35:50.447 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:35:50.445+0000 7fea00ff9700 1 -- 172.21.15.160:0/2238518404 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fe9f809b140 con 0x7fea04103380 2024-03-07T07:35:50.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:35:50.648+0000 7fea09c2b700 1 -- 172.21.15.160:0/2238518404 --> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fea04062590 con 0x7fe9ec0a1e10 2024-03-07T07:35:51.252 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:50 smithi160 ceph-mon[31963]: osdmap e40: 6 total, 6 up, 6 in 2024-03-07T07:35:51.252 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:50 smithi160 ceph-mon[31963]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-03-07T07:35:51.252 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:50 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-07T07:35:51.252 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:50 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-07T07:35:51.252 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:50 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:35:51.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:50 smithi178 ceph-mon[36650]: osdmap e40: 6 total, 6 up, 6 in 2024-03-07T07:35:51.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:50 smithi178 ceph-mon[36650]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-03-07T07:35:51.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:50 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-07T07:35:51.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:50 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-07T07:35:51.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:50 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:35:51.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:50 smithi177 ceph-mon[36578]: osdmap e40: 6 total, 6 up, 6 in 2024-03-07T07:35:51.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:50 smithi177 ceph-mon[36578]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-03-07T07:35:51.358 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:50 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-07T07:35:51.358 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:50 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-07T07:35:51.358 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:50 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:35:52.242 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:51 smithi160 ceph-mon[31963]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:35:52.243 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:51 smithi160 ceph-mon[31963]: osdmap e41: 6 total, 6 up, 6 in 2024-03-07T07:35:52.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:51 smithi178 ceph-mon[36650]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:35:52.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:51 smithi178 ceph-mon[36650]: osdmap e41: 6 total, 6 up, 6 in 2024-03-07T07:35:52.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:51 smithi177 ceph-mon[36578]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:35:52.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:51 smithi177 ceph-mon[36578]: osdmap e41: 6 total, 6 up, 6 in 2024-03-07T07:35:53.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:52 smithi178 ceph-mon[36650]: osdmap e42: 6 total, 6 up, 6 in 2024-03-07T07:35:53.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:52 smithi178 ceph-mon[36650]: pgmap v144: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-07T07:35:53.356 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:52 smithi177 ceph-mon[36578]: osdmap e42: 6 total, 6 up, 6 in 2024-03-07T07:35:53.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:52 smithi177 ceph-mon[36578]: pgmap v144: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-07T07:35:53.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:52 smithi160 ceph-mon[31963]: osdmap e42: 6 total, 6 up, 6 in 2024-03-07T07:35:53.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:52 smithi160 ceph-mon[31963]: pgmap v144: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-07T07:35:54.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:53 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/3937528173' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b1e1646e-f8a9-42b8-b62d-9d7b5d6a1ff4"}]: dispatch 2024-03-07T07:35:54.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:53 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/3937528173' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b1e1646e-f8a9-42b8-b62d-9d7b5d6a1ff4"}]': finished 2024-03-07T07:35:54.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:53 smithi160 ceph-mon[31963]: osdmap e43: 7 total, 6 up, 7 in 2024-03-07T07:35:54.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:53 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-07T07:35:54.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:53 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/3937528173' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b1e1646e-f8a9-42b8-b62d-9d7b5d6a1ff4"}]: dispatch 2024-03-07T07:35:54.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:53 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/3937528173' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b1e1646e-f8a9-42b8-b62d-9d7b5d6a1ff4"}]': finished 2024-03-07T07:35:54.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:53 smithi178 ceph-mon[36650]: osdmap e43: 7 total, 6 up, 7 in 2024-03-07T07:35:54.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:53 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-07T07:35:54.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:53 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/3937528173' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b1e1646e-f8a9-42b8-b62d-9d7b5d6a1ff4"}]: dispatch 2024-03-07T07:35:54.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:53 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/3937528173' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b1e1646e-f8a9-42b8-b62d-9d7b5d6a1ff4"}]': finished 2024-03-07T07:35:54.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:53 smithi177 ceph-mon[36578]: osdmap e43: 7 total, 6 up, 7 in 2024-03-07T07:35:54.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:53 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-07T07:35:55.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:54 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/4282565399' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-07T07:35:55.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:54 smithi178 ceph-mon[36650]: pgmap v146: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 134 KiB/s, 0 objects/s recovering 2024-03-07T07:35:55.356 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:54 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/4282565399' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-07T07:35:55.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:54 smithi177 ceph-mon[36578]: pgmap v146: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 134 KiB/s, 0 objects/s recovering 2024-03-07T07:35:55.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:54 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/4282565399' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-07T07:35:55.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:54 smithi160 ceph-mon[31963]: pgmap v146: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 134 KiB/s, 0 objects/s recovering 2024-03-07T07:35:57.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:57 smithi178 ceph-mon[36650]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-07T07:35:57.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:57 smithi177 ceph-mon[36578]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-07T07:35:57.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:57 smithi160 ceph-mon[31963]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-07T07:35:59.515 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:35:59 smithi160 ceph-mon[31963]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-07T07:35:59.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:35:59 smithi178 ceph-mon[36650]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-07T07:35:59.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:35:59 smithi177 ceph-mon[36578]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-07T07:36:01.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:01 smithi178 ceph-mon[36650]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-07T07:36:01.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:01 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-07T07:36:01.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:01 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:36:01.580 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:01 smithi160 ceph-mon[31963]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-07T07:36:01.581 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:01 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-07T07:36:01.581 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:01 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:36:01.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:01 smithi177 ceph-mon[36578]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-07T07:36:01.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:01 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-07T07:36:01.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:01 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:36:02.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:02 smithi178 ceph-mon[36650]: Deploying daemon osd.6 on smithi160 2024-03-07T07:36:02.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:02 smithi177 ceph-mon[36578]: Deploying daemon osd.6 on smithi160 2024-03-07T07:36:02.664 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:02 smithi160 ceph-mon[31963]: Deploying daemon osd.6 on smithi160 2024-03-07T07:36:03.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:03 smithi178 ceph-mon[36650]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-07T07:36:03.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:03 smithi177 ceph-mon[36578]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-07T07:36:03.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:03 smithi160 ceph-mon[31963]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-07T07:36:05.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:05 smithi178 ceph-mon[36650]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-07T07:36:05.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:05 smithi177 ceph-mon[36578]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-07T07:36:05.657 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:05 smithi160 ceph-mon[31963]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-07T07:36:07.547 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:07 smithi160 ceph-mon[31963]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-07T07:36:07.548 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:07 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:07.548 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:07 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:07.548 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:07 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:36:07.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:07 smithi178 ceph-mon[36650]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-07T07:36:07.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:07 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:07.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:07 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:07.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:07 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:36:07.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:07 smithi177 ceph-mon[36578]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-07T07:36:07.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:07 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:07.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:07 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:07.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:07 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:36:09.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:09 smithi160 ceph-mon[31963]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-07T07:36:09.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:09 smithi178 ceph-mon[36650]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-07T07:36:09.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:09 smithi177 ceph-mon[36578]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-07T07:36:10.655 INFO:teuthology.orchestra.run.smithi160.stdout:Created osd(s) 6 on host 'smithi160' 2024-03-07T07:36:10.655 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:36:10.654+0000 7fea00ff9700 1 -- 172.21.15.160:0/2238518404 <== mgr.14152 v2:172.21.15.160:6800/419630619 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fea04062590 con 0x7fe9ec0a1e10 2024-03-07T07:36:10.656 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:36:10.656+0000 7fea09c2b700 1 -- 172.21.15.160:0/2238518404 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fe9ec0a1e10 msgr2=0x7fe9ec0a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:36:10.656 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:36:10.656+0000 7fea09c2b700 1 --2- 172.21.15.160:0/2238518404 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fe9ec0a1e10 0x7fe9ec0a42d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4007cf0 tx=0x7fe9f4007480 comp rx=0 tx=0).stop 2024-03-07T07:36:10.656 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:36:10.656+0000 7fea09c2b700 1 -- 172.21.15.160:0/2238518404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea04103380 msgr2=0x7fea04198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:36:10.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:36:10.656+0000 7fea09c2b700 1 --2- 172.21.15.160:0/2238518404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea04103380 0x7fea04198860 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe9f800ea20 tx=0x7fe9f800ed30 comp rx=0 tx=0).stop 2024-03-07T07:36:10.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:36:10.656+0000 7fea09c2b700 1 -- 172.21.15.160:0/2238518404 shutdown_connections 2024-03-07T07:36:10.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:36:10.656+0000 7fea09c2b700 1 --2- 172.21.15.160:0/2238518404 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea04103d20 0x7fea0419f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:36:10.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:36:10.656+0000 7fea09c2b700 1 --2- 172.21.15.160:0/2238518404 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea04102180 0x7fea04198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:36:10.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:36:10.656+0000 7fea09c2b700 1 --2- 172.21.15.160:0/2238518404 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fe9ec0a1e10 0x7fe9ec0a42d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:36:10.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:36:10.656+0000 7fea09c2b700 1 --2- 172.21.15.160:0/2238518404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea04103380 0x7fea04198860 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:36:10.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:36:10.656+0000 7fea09c2b700 1 -- 172.21.15.160:0/2238518404 >> 172.21.15.160:0/2238518404 conn(0x7fea040fd760 msgr2=0x7fea04106680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:36:10.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:36:10.656+0000 7fea09c2b700 1 -- 172.21.15.160:0/2238518404 shutdown_connections 2024-03-07T07:36:10.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:36:10.656+0000 7fea09c2b700 1 -- 172.21.15.160:0/2238518404 wait complete. 2024-03-07T07:36:11.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:10 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:11.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:10 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:11.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:10 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:36:11.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:10 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:36:11.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:10 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:11.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:10 smithi177 ceph-mon[36578]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-07T07:36:11.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:10 smithi177 ceph-mon[36578]: from='osd.6 [v2:172.21.15.160:6818/3251327001,v1:172.21.15.160:6819/3251327001]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-07T07:36:11.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:10 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:11.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:10 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:11.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:10 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:36:11.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:10 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:11.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:10 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:11.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:10 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:36:11.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:10 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:36:11.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:10 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:11.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:10 smithi160 ceph-mon[31963]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-07T07:36:11.179 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:10 smithi160 ceph-mon[31963]: from='osd.6 [v2:172.21.15.160:6818/3251327001,v1:172.21.15.160:6819/3251327001]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-07T07:36:11.179 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:10 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:11.179 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:10 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:11.179 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:10 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:36:11.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:10 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:11.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:10 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:11.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:10 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:36:11.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:10 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:36:11.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:10 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:11.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:10 smithi178 ceph-mon[36650]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-07T07:36:11.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:10 smithi178 ceph-mon[36650]: from='osd.6 [v2:172.21.15.160:6818/3251327001,v1:172.21.15.160:6819/3251327001]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-07T07:36:11.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:10 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:11.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:10 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:11.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:10 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:36:11.365 DEBUG:teuthology.orchestra.run.smithi160:osd.6> sudo journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@osd.6.service 2024-03-07T07:36:11.368 INFO:tasks.cephadm:Deploying osd.7 on smithi177 with /dev/vg_nvme/lv_2... 2024-03-07T07:36:11.368 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-07T07:36:11.613 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.b/config 2024-03-07T07:36:11.848 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:11 smithi160 ceph-mon[31963]: from='osd.6 [v2:172.21.15.160:6818/3251327001,v1:172.21.15.160:6819/3251327001]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-07T07:36:11.848 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:11 smithi160 ceph-mon[31963]: osdmap e44: 7 total, 6 up, 7 in 2024-03-07T07:36:11.848 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:11 smithi160 ceph-mon[31963]: from='osd.6 [v2:172.21.15.160:6818/3251327001,v1:172.21.15.160:6819/3251327001]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-03-07T07:36:11.848 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:11 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-07T07:36:11.986 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:11 smithi177 ceph-mon[36578]: from='osd.6 [v2:172.21.15.160:6818/3251327001,v1:172.21.15.160:6819/3251327001]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-07T07:36:11.986 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:11 smithi177 ceph-mon[36578]: osdmap e44: 7 total, 6 up, 7 in 2024-03-07T07:36:11.986 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:11 smithi177 ceph-mon[36578]: from='osd.6 [v2:172.21.15.160:6818/3251327001,v1:172.21.15.160:6819/3251327001]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-03-07T07:36:11.986 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:11 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-07T07:36:12.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:11 smithi178 ceph-mon[36650]: from='osd.6 [v2:172.21.15.160:6818/3251327001,v1:172.21.15.160:6819/3251327001]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-07T07:36:12.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:11 smithi178 ceph-mon[36650]: osdmap e44: 7 total, 6 up, 7 in 2024-03-07T07:36:12.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:11 smithi178 ceph-mon[36650]: from='osd.6 [v2:172.21.15.160:6818/3251327001,v1:172.21.15.160:6819/3251327001]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-03-07T07:36:12.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:11 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-07T07:36:12.608 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:36:12 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:36:12.251+0000 7f0cc8c2c700 -1 osd.6 0 waiting for initial osdmap 2024-03-07T07:36:12.608 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:36:12 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:36:12.261+0000 7f0cc0a17700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-07T07:36:12.907 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:12 smithi160 ceph-mon[31963]: from='osd.6 [v2:172.21.15.160:6818/3251327001,v1:172.21.15.160:6819/3251327001]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-03-07T07:36:12.907 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:12 smithi160 ceph-mon[31963]: osdmap e45: 7 total, 6 up, 7 in 2024-03-07T07:36:12.907 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:12 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-07T07:36:12.907 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:12 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-07T07:36:12.907 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:12 smithi160 ceph-mon[31963]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-07T07:36:12.907 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:12 smithi160 ceph-mon[31963]: from='osd.6 [v2:172.21.15.160:6818/3251327001,v1:172.21.15.160:6819/3251327001]' entity='osd.6' 2024-03-07T07:36:13.088 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:12 smithi177 ceph-mon[36578]: from='osd.6 [v2:172.21.15.160:6818/3251327001,v1:172.21.15.160:6819/3251327001]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-03-07T07:36:13.088 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:12 smithi177 ceph-mon[36578]: osdmap e45: 7 total, 6 up, 7 in 2024-03-07T07:36:13.089 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:12 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-07T07:36:13.089 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:12 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-07T07:36:13.089 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:12 smithi177 ceph-mon[36578]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-07T07:36:13.089 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:12 smithi177 ceph-mon[36578]: from='osd.6 [v2:172.21.15.160:6818/3251327001,v1:172.21.15.160:6819/3251327001]' entity='osd.6' 2024-03-07T07:36:13.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:12 smithi178 ceph-mon[36650]: from='osd.6 [v2:172.21.15.160:6818/3251327001,v1:172.21.15.160:6819/3251327001]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-03-07T07:36:13.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:12 smithi178 ceph-mon[36650]: osdmap e45: 7 total, 6 up, 7 in 2024-03-07T07:36:13.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:12 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-07T07:36:13.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:12 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-07T07:36:13.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:12 smithi178 ceph-mon[36650]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-07T07:36:13.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:12 smithi178 ceph-mon[36650]: from='osd.6 [v2:172.21.15.160:6818/3251327001,v1:172.21.15.160:6819/3251327001]' entity='osd.6' 2024-03-07T07:36:13.788 INFO:teuthology.orchestra.run.smithi177.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-07T07:36:13.788 INFO:teuthology.orchestra.run.smithi177.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-07T07:36:13.788 INFO:teuthology.orchestra.run.smithi177.stderr: stderr: 10+0 records in 2024-03-07T07:36:13.788 INFO:teuthology.orchestra.run.smithi177.stderr:10+0 records out 2024-03-07T07:36:13.788 INFO:teuthology.orchestra.run.smithi177.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111115 s, 944 MB/s 2024-03-07T07:36:13.788 INFO:teuthology.orchestra.run.smithi177.stderr:--> Zapping successful for: 2024-03-07T07:36:14.085 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:13 smithi177 ceph-mon[36578]: purged_snaps scrub starts 2024-03-07T07:36:14.085 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:13 smithi177 ceph-mon[36578]: purged_snaps scrub ok 2024-03-07T07:36:14.086 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:13 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-07T07:36:14.086 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:13 smithi177 ceph-mon[36578]: osd.6 [v2:172.21.15.160:6818/3251327001,v1:172.21.15.160:6819/3251327001] boot 2024-03-07T07:36:14.086 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:13 smithi177 ceph-mon[36578]: osdmap e46: 7 total, 7 up, 7 in 2024-03-07T07:36:14.086 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:13 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-07T07:36:14.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:13 smithi160 ceph-mon[31963]: purged_snaps scrub starts 2024-03-07T07:36:14.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:13 smithi160 ceph-mon[31963]: purged_snaps scrub ok 2024-03-07T07:36:14.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:13 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-07T07:36:14.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:13 smithi160 ceph-mon[31963]: osd.6 [v2:172.21.15.160:6818/3251327001,v1:172.21.15.160:6819/3251327001] boot 2024-03-07T07:36:14.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:13 smithi160 ceph-mon[31963]: osdmap e46: 7 total, 7 up, 7 in 2024-03-07T07:36:14.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:13 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-07T07:36:14.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:13 smithi178 ceph-mon[36650]: purged_snaps scrub starts 2024-03-07T07:36:14.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:13 smithi178 ceph-mon[36650]: purged_snaps scrub ok 2024-03-07T07:36:14.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:13 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-07T07:36:14.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:13 smithi178 ceph-mon[36650]: osd.6 [v2:172.21.15.160:6818/3251327001,v1:172.21.15.160:6819/3251327001] boot 2024-03-07T07:36:14.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:13 smithi178 ceph-mon[36650]: osdmap e46: 7 total, 7 up, 7 in 2024-03-07T07:36:14.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:13 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-07T07:36:14.400 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi177:vg_nvme/lv_2 2024-03-07T07:36:14.642 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.b/config 2024-03-07T07:36:15.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:15 smithi178 ceph-mon[36650]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-07T07:36:15.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:15 smithi178 ceph-mon[36650]: osdmap e47: 7 total, 7 up, 7 in 2024-03-07T07:36:15.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:15 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:15.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:15 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:15.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:15 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:36:15.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:15 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:36:15.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:15 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:36:15.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:15 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:15.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:15 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:36:15.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:15 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:36:15.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:15 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:15.603 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:15 smithi177 ceph-mon[36578]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-07T07:36:15.603 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:15 smithi177 ceph-mon[36578]: osdmap e47: 7 total, 7 up, 7 in 2024-03-07T07:36:15.604 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:15 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:15.604 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:15 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:15.604 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:15 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:36:15.604 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:15 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:36:15.604 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:15 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:36:15.604 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:15 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:15.604 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:15 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:36:15.604 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:15 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:36:15.604 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:15 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:15.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:15 smithi160 ceph-mon[31963]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-07T07:36:15.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:15 smithi160 ceph-mon[31963]: osdmap e47: 7 total, 7 up, 7 in 2024-03-07T07:36:15.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:15 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:15.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:15 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:15.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:15 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:36:15.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:15 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:36:15.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:15 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:36:15.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:15 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:15.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:15 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:36:15.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:15 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:36:15.679 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:15 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:16.213 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:16.207+0000 7f0dd3b72700 1 -- 172.21.15.177:0/1410097799 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0dcc1022f0 msgr2=0x7f0dcc102750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:36:16.214 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:16.207+0000 7f0dd3b72700 1 --2- 172.21.15.177:0/1410097799 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0dcc1022f0 0x7f0dcc102750 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc0099c0 tx=0x7f0dbc009cd0 comp rx=0 tx=0).stop 2024-03-07T07:36:16.214 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:16.208+0000 7f0dd3b72700 1 -- 172.21.15.177:0/1410097799 shutdown_connections 2024-03-07T07:36:16.214 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:16.208+0000 7f0dd3b72700 1 --2- 172.21.15.177:0/1410097799 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0dcc102c90 0x7f0dcc109540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:36:16.214 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:16.208+0000 7f0dd3b72700 1 --2- 172.21.15.177:0/1410097799 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0dcc1010f0 0x7f0dcc101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:36:16.214 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:16.208+0000 7f0dd3b72700 1 --2- 172.21.15.177:0/1410097799 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0dcc1022f0 0x7f0dcc102750 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:36:16.214 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:16.208+0000 7f0dd3b72700 1 -- 172.21.15.177:0/1410097799 >> 172.21.15.177:0/1410097799 conn(0x7f0dcc0fc6b0 msgr2=0x7f0dcc0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:36:16.214 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:16.208+0000 7f0dd3b72700 1 -- 172.21.15.177:0/1410097799 shutdown_connections 2024-03-07T07:36:16.214 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:16.208+0000 7f0dd3b72700 1 -- 172.21.15.177:0/1410097799 wait complete. 2024-03-07T07:36:16.214 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:16.208+0000 7f0dd3b72700 1 Processor -- start 2024-03-07T07:36:16.214 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:16.208+0000 7f0dd3b72700 1 -- start start 2024-03-07T07:36:16.215 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:16.208+0000 7f0dd3b72700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0dcc1010f0 0x7f0dcc10bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:36:16.215 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:16.208+0000 7f0dd3b72700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0dcc1022f0 0x7f0dcc10a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:36:16.215 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:16.208+0000 7f0dd3b72700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0dcc102c90 0x7f0dcc10a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:36:16.215 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:16.208+0000 7f0dd3b72700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dcc10dd90 con 0x7f0dcc1010f0 2024-03-07T07:36:16.215 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:16.208+0000 7f0dd3b72700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dcc10dc10 con 0x7f0dcc102c90 2024-03-07T07:36:16.215 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:16.208+0000 7f0dd3b72700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dcc10df10 con 0x7f0dcc1022f0 2024-03-07T07:36:16.215 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:16.209+0000 7f0dd210f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0dcc102c90 0x7f0dcc10a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:36:16.215 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:16.209+0000 7f0dd210f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0dcc102c90 0x7f0dcc10a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:55712/0 (socket says 172.21.15.177:55712) 2024-03-07T07:36:16.215 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:16.209+0000 7f0dd110d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0dcc1022f0 0x7f0dcc10a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:36:16.215 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:16.209+0000 7f0dd210f700 1 -- 172.21.15.177:0/3278215186 learned_addr learned my addr 172.21.15.177:0/3278215186 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-03-07T07:36:16.215 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:16.209+0000 7f0dd190e700 1 --2- 172.21.15.177:0/3278215186 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0dcc1010f0 0x7f0dcc10bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:36:16.215 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:16.209+0000 7f0dd210f700 1 -- 172.21.15.177:0/3278215186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0dcc1022f0 msgr2=0x7f0dcc10a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:36:16.215 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:16.209+0000 7f0dd210f700 1 --2- 172.21.15.177:0/3278215186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0dcc1022f0 0x7f0dcc10a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:36:16.216 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:16.209+0000 7f0dd210f700 1 -- 172.21.15.177:0/3278215186 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0dcc1010f0 msgr2=0x7f0dcc10bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:36:16.216 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:16.209+0000 7f0dd210f700 1 --2- 172.21.15.177:0/3278215186 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0dcc1010f0 0x7f0dcc10bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:36:16.216 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:16.209+0000 7f0dd210f700 1 -- 172.21.15.177:0/3278215186 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dcc10af00 con 0x7f0dcc102c90 2024-03-07T07:36:16.216 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:16.209+0000 7f0dd210f700 1 --2- 172.21.15.177:0/3278215186 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0dcc102c90 0x7f0dcc10a650 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4007ee0 tx=0x7f0dc400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:36:16.216 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:16.209+0000 7f0dc2ffd700 1 -- 172.21.15.177:0/3278215186 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4004bb0 con 0x7f0dcc102c90 2024-03-07T07:36:16.216 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:16.209+0000 7f0dc2ffd700 1 -- 172.21.15.177:0/3278215186 <== mon.2 v2:172.21.15.177:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0dc4004d50 con 0x7f0dcc102c90 2024-03-07T07:36:16.216 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:16.209+0000 7f0dd3b72700 1 -- 172.21.15.177:0/3278215186 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dcc1a24d0 con 0x7f0dcc102c90 2024-03-07T07:36:16.216 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:16.209+0000 7f0dc2ffd700 1 -- 172.21.15.177:0/3278215186 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4005960 con 0x7f0dcc102c90 2024-03-07T07:36:16.216 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:16.209+0000 7f0dd3b72700 1 -- 172.21.15.177:0/3278215186 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dcc1a2990 con 0x7f0dcc102c90 2024-03-07T07:36:16.216 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:16.210+0000 7f0dd3b72700 1 -- 172.21.15.177:0/3278215186 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dcc068e10 con 0x7f0dcc102c90 2024-03-07T07:36:16.216 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:16.212+0000 7f0dc2ffd700 1 -- 172.21.15.177:0/3278215186 <== mon.2 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f0dc401a020 con 0x7f0dcc102c90 2024-03-07T07:36:16.217 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:16.212+0000 7f0dc2ffd700 1 --2- 172.21.15.177:0/3278215186 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f0db80a1e10 0x7f0db80a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:36:16.217 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:16.213+0000 7f0dc2ffd700 1 -- 172.21.15.177:0/3278215186 <== mon.2 v2:172.21.15.177:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f0dc40cce80 con 0x7f0dcc102c90 2024-03-07T07:36:16.217 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:16.213+0000 7f0dd190e700 1 --2- 172.21.15.177:0/3278215186 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f0db80a1e10 0x7f0db80a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:36:16.217 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:16.213+0000 7f0dd190e700 1 --2- 172.21.15.177:0/3278215186 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f0db80a1e10 0x7f0db80a42d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc102150 tx=0x7f0dc800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:36:16.217 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:16.217+0000 7f0dc2ffd700 1 -- 172.21.15.177:0/3278215186 <== mon.2 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f0dc4010070 con 0x7f0dcc102c90 2024-03-07T07:36:16.420 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:16.420+0000 7f0dd3b72700 1 -- 172.21.15.177:0/3278215186 --> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f0dcc062590 con 0x7f0db80a1e10 2024-03-07T07:36:16.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:16 smithi178 ceph-mon[36650]: Detected new or changed devices on smithi160 2024-03-07T07:36:16.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:16 smithi178 ceph-mon[36650]: Adjusting osd_memory_target on smithi160 to 5696M 2024-03-07T07:36:16.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:16 smithi177 ceph-mon[36578]: Detected new or changed devices on smithi160 2024-03-07T07:36:16.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:16 smithi177 ceph-mon[36578]: Adjusting osd_memory_target on smithi160 to 5696M 2024-03-07T07:36:16.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:16 smithi160 ceph-mon[31963]: Detected new or changed devices on smithi160 2024-03-07T07:36:16.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:16 smithi160 ceph-mon[31963]: Adjusting osd_memory_target on smithi160 to 5696M 2024-03-07T07:36:17.308 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:17 smithi177 ceph-mon[36578]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-07T07:36:17.309 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:17 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-07T07:36:17.309 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:17 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-07T07:36:17.309 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:17 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:36:17.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:17 smithi178 ceph-mon[36650]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-07T07:36:17.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:17 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-07T07:36:17.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:17 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-07T07:36:17.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:17 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:36:17.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:17 smithi160 ceph-mon[31963]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-07T07:36:17.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:17 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-07T07:36:17.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:17 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-07T07:36:17.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:17 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:36:18.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:18 smithi177 ceph-mon[36578]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:36:18.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:18 smithi178 ceph-mon[36650]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:36:18.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:18 smithi160 ceph-mon[31963]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:36:19.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:19 smithi178 ceph-mon[36650]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-07T07:36:19.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:19 smithi178 ceph-mon[36650]: from='client.? 172.21.15.177:0/2566037498' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e66b400b-9747-4f6d-b05b-db22468e2750"}]: dispatch 2024-03-07T07:36:19.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:19 smithi178 ceph-mon[36650]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e66b400b-9747-4f6d-b05b-db22468e2750"}]: dispatch 2024-03-07T07:36:19.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:19 smithi177 ceph-mon[36578]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-07T07:36:19.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:19 smithi177 ceph-mon[36578]: from='client.? 172.21.15.177:0/2566037498' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e66b400b-9747-4f6d-b05b-db22468e2750"}]: dispatch 2024-03-07T07:36:19.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:19 smithi177 ceph-mon[36578]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e66b400b-9747-4f6d-b05b-db22468e2750"}]: dispatch 2024-03-07T07:36:19.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:19 smithi160 ceph-mon[31963]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-07T07:36:19.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:19 smithi160 ceph-mon[31963]: from='client.? 172.21.15.177:0/2566037498' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e66b400b-9747-4f6d-b05b-db22468e2750"}]: dispatch 2024-03-07T07:36:19.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:19 smithi160 ceph-mon[31963]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e66b400b-9747-4f6d-b05b-db22468e2750"}]: dispatch 2024-03-07T07:36:20.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:20 smithi178 ceph-mon[36650]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e66b400b-9747-4f6d-b05b-db22468e2750"}]': finished 2024-03-07T07:36:20.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:20 smithi178 ceph-mon[36650]: osdmap e48: 8 total, 7 up, 8 in 2024-03-07T07:36:20.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:20 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-07T07:36:20.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:20 smithi178 ceph-mon[36650]: from='client.? 172.21.15.177:0/1638027113' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-07T07:36:20.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:20 smithi177 ceph-mon[36578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e66b400b-9747-4f6d-b05b-db22468e2750"}]': finished 2024-03-07T07:36:20.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:20 smithi177 ceph-mon[36578]: osdmap e48: 8 total, 7 up, 8 in 2024-03-07T07:36:20.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:20 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-07T07:36:20.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:20 smithi177 ceph-mon[36578]: from='client.? 172.21.15.177:0/1638027113' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-07T07:36:20.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:20 smithi160 ceph-mon[31963]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e66b400b-9747-4f6d-b05b-db22468e2750"}]': finished 2024-03-07T07:36:20.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:20 smithi160 ceph-mon[31963]: osdmap e48: 8 total, 7 up, 8 in 2024-03-07T07:36:20.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:20 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-07T07:36:20.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:20 smithi160 ceph-mon[31963]: from='client.? 172.21.15.177:0/1638027113' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-07T07:36:21.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:21 smithi178 ceph-mon[36650]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-07T07:36:21.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:21 smithi177 ceph-mon[36578]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-07T07:36:21.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:21 smithi160 ceph-mon[31963]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-07T07:36:23.522 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:23 smithi177 ceph-mon[36578]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-07T07:36:23.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:23 smithi178 ceph-mon[36650]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-07T07:36:23.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:23 smithi160 ceph-mon[31963]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-07T07:36:25.431 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:25 smithi177 ceph-mon[36578]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-07T07:36:25.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:25 smithi178 ceph-mon[36650]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-07T07:36:25.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:25 smithi160 ceph-mon[31963]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-07T07:36:27.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:27 smithi178 ceph-mon[36650]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-07T07:36:27.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:27 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-07T07:36:27.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:27 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:36:27.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:27 smithi177 ceph-mon[36578]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-07T07:36:27.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:27 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-07T07:36:27.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:27 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:36:27.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:27 smithi160 ceph-mon[31963]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-07T07:36:27.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:27 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-07T07:36:27.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:27 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:36:28.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:28 smithi177 ceph-mon[36578]: Deploying daemon osd.7 on smithi177 2024-03-07T07:36:28.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:28 smithi178 ceph-mon[36650]: Deploying daemon osd.7 on smithi177 2024-03-07T07:36:28.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:28 smithi160 ceph-mon[31963]: Deploying daemon osd.7 on smithi177 2024-03-07T07:36:29.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:29 smithi178 ceph-mon[36650]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-07T07:36:29.579 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:29 smithi160 ceph-mon[31963]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-07T07:36:29.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:29 smithi177 ceph-mon[36578]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-07T07:36:31.326 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:31 smithi177 ceph-mon[36578]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-07T07:36:31.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:31 smithi178 ceph-mon[36650]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-07T07:36:31.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:31 smithi160 ceph-mon[31963]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-07T07:36:33.180 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:32 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:33.180 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:32 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:33.180 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:32 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:36:33.180 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:32 smithi160 ceph-mon[31963]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-07T07:36:33.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:32 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:33.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:32 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:33.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:32 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:36:33.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:32 smithi178 ceph-mon[36650]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-07T07:36:33.307 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:32 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:33.307 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:32 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:33.307 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:32 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:36:33.307 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:32 smithi177 ceph-mon[36578]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-07T07:36:35.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:35 smithi178 ceph-mon[36650]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-07T07:36:35.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:35 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:35.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:35 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:35.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:35 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:36:35.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:35 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:36:35.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:35 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:35.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:35 smithi177 ceph-mon[36578]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-07T07:36:35.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:35 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:35.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:35 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:35.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:35 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:36:35.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:35 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:36:35.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:35 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:35.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:35 smithi160 ceph-mon[31963]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-07T07:36:35.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:35 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:35.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:35 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:35.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:35 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:36:35.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:35 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:36:35.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:35 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:35.685 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:35.685+0000 7f0dc2ffd700 1 -- 172.21.15.177:0/3278215186 <== mgr.14152 v2:172.21.15.160:6800/419630619 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0dcc062590 con 0x7f0db80a1e10 2024-03-07T07:36:35.686 INFO:teuthology.orchestra.run.smithi177.stdout:Created osd(s) 7 on host 'smithi177' 2024-03-07T07:36:35.688 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:35.688+0000 7f0dd3b72700 1 -- 172.21.15.177:0/3278215186 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f0db80a1e10 msgr2=0x7f0db80a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:36:35.688 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:35.688+0000 7f0dd3b72700 1 --2- 172.21.15.177:0/3278215186 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f0db80a1e10 0x7f0db80a42d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc102150 tx=0x7f0dc800a380 comp rx=0 tx=0).stop 2024-03-07T07:36:35.688 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:35.688+0000 7f0dd3b72700 1 -- 172.21.15.177:0/3278215186 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0dcc102c90 msgr2=0x7f0dcc10a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:36:35.688 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:35.688+0000 7f0dd3b72700 1 --2- 172.21.15.177:0/3278215186 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0dcc102c90 0x7f0dcc10a650 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4007ee0 tx=0x7f0dc400d3b0 comp rx=0 tx=0).stop 2024-03-07T07:36:35.689 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:35.688+0000 7f0dd3b72700 1 -- 172.21.15.177:0/3278215186 shutdown_connections 2024-03-07T07:36:35.689 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:35.688+0000 7f0dd3b72700 1 --2- 172.21.15.177:0/3278215186 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0dcc102c90 0x7f0dcc10a650 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:36:35.689 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:35.688+0000 7f0dd3b72700 1 --2- 172.21.15.177:0/3278215186 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0dcc1010f0 0x7f0dcc10bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:36:35.689 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:35.688+0000 7f0dd3b72700 1 --2- 172.21.15.177:0/3278215186 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f0db80a1e10 0x7f0db80a42d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:36:35.689 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:35.688+0000 7f0dd3b72700 1 --2- 172.21.15.177:0/3278215186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0dcc1022f0 0x7f0dcc10a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:36:35.689 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:35.688+0000 7f0dd3b72700 1 -- 172.21.15.177:0/3278215186 >> 172.21.15.177:0/3278215186 conn(0x7f0dcc0fc6b0 msgr2=0x7f0dcc1055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:36:35.689 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:35.688+0000 7f0dd3b72700 1 -- 172.21.15.177:0/3278215186 shutdown_connections 2024-03-07T07:36:35.689 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:36:35.688+0000 7f0dd3b72700 1 -- 172.21.15.177:0/3278215186 wait complete. 2024-03-07T07:36:36.372 DEBUG:teuthology.orchestra.run.smithi177:osd.7> sudo journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@osd.7.service 2024-03-07T07:36:36.375 INFO:tasks.cephadm:Deploying osd.8 on smithi178 with /dev/vg_nvme/lv_2... 2024-03-07T07:36:36.375 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-07T07:36:36.620 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.c/config 2024-03-07T07:36:36.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:36 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:36.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:36 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:36.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:36 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:36:36.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:36 smithi160 ceph-mon[31963]: from='osd.7 [v2:172.21.15.177:6816/1175500850,v1:172.21.15.177:6817/1175500850]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-07T07:36:36.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:36 smithi160 ceph-mon[31963]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-07T07:36:36.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:36 smithi160 ceph-mon[31963]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-07T07:36:36.931 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:36 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:36.932 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:36 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:36.932 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:36 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:36:36.932 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:36 smithi177 ceph-mon[36578]: from='osd.7 [v2:172.21.15.177:6816/1175500850,v1:172.21.15.177:6817/1175500850]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-07T07:36:36.932 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:36 smithi177 ceph-mon[36578]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-07T07:36:36.932 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:36 smithi177 ceph-mon[36578]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-07T07:36:37.004 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:36 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:37.004 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:36 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:37.004 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:36 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:36:37.004 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:36 smithi178 ceph-mon[36650]: from='osd.7 [v2:172.21.15.177:6816/1175500850,v1:172.21.15.177:6817/1175500850]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-07T07:36:37.004 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:36 smithi178 ceph-mon[36650]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-07T07:36:37.004 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:36 smithi178 ceph-mon[36650]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-07T07:36:37.993 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:37 smithi178 ceph-mon[36650]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-07T07:36:37.993 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:37 smithi178 ceph-mon[36650]: from='osd.7 [v2:172.21.15.177:6816/1175500850,v1:172.21.15.177:6817/1175500850]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-03-07T07:36:37.993 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:37 smithi178 ceph-mon[36650]: osdmap e49: 8 total, 7 up, 8 in 2024-03-07T07:36:37.993 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:37 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-07T07:36:37.993 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:37 smithi178 ceph-mon[36650]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-03-07T07:36:37.998 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:37 smithi177 ceph-mon[36578]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-07T07:36:37.998 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:37 smithi177 ceph-mon[36578]: from='osd.7 [v2:172.21.15.177:6816/1175500850,v1:172.21.15.177:6817/1175500850]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-03-07T07:36:37.999 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:37 smithi177 ceph-mon[36578]: osdmap e49: 8 total, 7 up, 8 in 2024-03-07T07:36:37.999 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:37 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-07T07:36:37.999 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:37 smithi177 ceph-mon[36578]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-03-07T07:36:38.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:37 smithi160 ceph-mon[31963]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-07T07:36:38.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:37 smithi160 ceph-mon[31963]: from='osd.7 [v2:172.21.15.177:6816/1175500850,v1:172.21.15.177:6817/1175500850]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-03-07T07:36:38.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:37 smithi160 ceph-mon[31963]: osdmap e49: 8 total, 7 up, 8 in 2024-03-07T07:36:38.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:37 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-07T07:36:38.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:37 smithi160 ceph-mon[31963]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-03-07T07:36:38.357 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:36:38 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:36:38.128+0000 7f974f04d700 -1 osd.7 0 waiting for initial osdmap 2024-03-07T07:36:38.357 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:36:38 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:36:38.139+0000 7f9746e38700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-07T07:36:38.866 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-07T07:36:38.867 INFO:teuthology.orchestra.run.smithi178.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-07T07:36:38.867 INFO:teuthology.orchestra.run.smithi178.stderr: stderr: 10+0 records in 2024-03-07T07:36:38.867 INFO:teuthology.orchestra.run.smithi178.stderr:10+0 records out 2024-03-07T07:36:38.867 INFO:teuthology.orchestra.run.smithi178.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00842828 s, 1.2 GB/s 2024-03-07T07:36:38.867 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping successful for: 2024-03-07T07:36:39.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:38 smithi178 ceph-mon[36650]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2024-03-07T07:36:39.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:38 smithi178 ceph-mon[36650]: osdmap e50: 8 total, 7 up, 8 in 2024-03-07T07:36:39.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:38 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-07T07:36:39.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:38 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-07T07:36:39.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:38 smithi178 ceph-mon[36650]: from='osd.7 ' entity='osd.7' 2024-03-07T07:36:39.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:38 smithi178 ceph-mon[36650]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-07T07:36:39.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:38 smithi178 ceph-mon[36650]: osd.7 [v2:172.21.15.177:6816/1175500850,v1:172.21.15.177:6817/1175500850] boot 2024-03-07T07:36:39.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:38 smithi178 ceph-mon[36650]: osdmap e51: 8 total, 8 up, 8 in 2024-03-07T07:36:39.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:38 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-07T07:36:39.068 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:38 smithi177 ceph-mon[36578]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2024-03-07T07:36:39.068 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:38 smithi177 ceph-mon[36578]: osdmap e50: 8 total, 7 up, 8 in 2024-03-07T07:36:39.068 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:38 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-07T07:36:39.068 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:38 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-07T07:36:39.068 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:38 smithi177 ceph-mon[36578]: from='osd.7 ' entity='osd.7' 2024-03-07T07:36:39.068 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:38 smithi177 ceph-mon[36578]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-07T07:36:39.068 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:38 smithi177 ceph-mon[36578]: osd.7 [v2:172.21.15.177:6816/1175500850,v1:172.21.15.177:6817/1175500850] boot 2024-03-07T07:36:39.068 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:38 smithi177 ceph-mon[36578]: osdmap e51: 8 total, 8 up, 8 in 2024-03-07T07:36:39.068 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:38 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-07T07:36:39.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:38 smithi160 ceph-mon[31963]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2024-03-07T07:36:39.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:38 smithi160 ceph-mon[31963]: osdmap e50: 8 total, 7 up, 8 in 2024-03-07T07:36:39.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:38 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-07T07:36:39.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:38 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-07T07:36:39.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:38 smithi160 ceph-mon[31963]: from='osd.7 ' entity='osd.7' 2024-03-07T07:36:39.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:38 smithi160 ceph-mon[31963]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-07T07:36:39.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:38 smithi160 ceph-mon[31963]: osd.7 [v2:172.21.15.177:6816/1175500850,v1:172.21.15.177:6817/1175500850] boot 2024-03-07T07:36:39.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:38 smithi160 ceph-mon[31963]: osdmap e51: 8 total, 8 up, 8 in 2024-03-07T07:36:39.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:38 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-07T07:36:39.586 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi178:vg_nvme/lv_2 2024-03-07T07:36:39.836 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.c/config 2024-03-07T07:36:39.865 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:39 smithi178 ceph-mon[36650]: purged_snaps scrub starts 2024-03-07T07:36:39.865 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:39 smithi178 ceph-mon[36650]: purged_snaps scrub ok 2024-03-07T07:36:39.865 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:39 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:39.865 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:39 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:39.865 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:39 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:36:39.865 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:39 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:36:39.865 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:39 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:36:39.865 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:39 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:39.865 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:39 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:36:39.865 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:39 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:36:39.865 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:39 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:39.865 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:39 smithi178 ceph-mon[36650]: osdmap e52: 8 total, 8 up, 8 in 2024-03-07T07:36:40.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:39 smithi177 ceph-mon[36578]: purged_snaps scrub starts 2024-03-07T07:36:40.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:39 smithi177 ceph-mon[36578]: purged_snaps scrub ok 2024-03-07T07:36:40.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:39 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:40.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:39 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:40.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:39 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:36:40.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:39 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:36:40.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:39 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:36:40.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:39 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:40.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:39 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:36:40.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:39 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:36:40.108 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:39 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:40.108 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:39 smithi177 ceph-mon[36578]: osdmap e52: 8 total, 8 up, 8 in 2024-03-07T07:36:40.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:39 smithi160 ceph-mon[31963]: purged_snaps scrub starts 2024-03-07T07:36:40.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:39 smithi160 ceph-mon[31963]: purged_snaps scrub ok 2024-03-07T07:36:40.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:39 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:40.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:39 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:40.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:39 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:36:40.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:39 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:36:40.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:39 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:36:40.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:39 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:40.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:39 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:36:40.179 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:39 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:36:40.179 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:39 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:40.179 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:39 smithi160 ceph-mon[31963]: osdmap e52: 8 total, 8 up, 8 in 2024-03-07T07:36:40.958 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:40 smithi178 ceph-mon[36650]: Detected new or changed devices on smithi177 2024-03-07T07:36:40.958 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:40 smithi178 ceph-mon[36650]: Adjusting osd_memory_target on smithi177 to 5693M 2024-03-07T07:36:40.958 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:40 smithi178 ceph-mon[36650]: pgmap v178: 1 pgs: 1 remapped; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-03-07T07:36:40.958 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:40 smithi178 ceph-mon[36650]: osdmap e53: 8 total, 8 up, 8 in 2024-03-07T07:36:41.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:40 smithi177 ceph-mon[36578]: Detected new or changed devices on smithi177 2024-03-07T07:36:41.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:40 smithi177 ceph-mon[36578]: Adjusting osd_memory_target on smithi177 to 5693M 2024-03-07T07:36:41.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:40 smithi177 ceph-mon[36578]: pgmap v178: 1 pgs: 1 remapped; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-03-07T07:36:41.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:40 smithi177 ceph-mon[36578]: osdmap e53: 8 total, 8 up, 8 in 2024-03-07T07:36:41.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:40 smithi160 ceph-mon[31963]: Detected new or changed devices on smithi177 2024-03-07T07:36:41.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:40 smithi160 ceph-mon[31963]: Adjusting osd_memory_target on smithi177 to 5693M 2024-03-07T07:36:41.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:40 smithi160 ceph-mon[31963]: pgmap v178: 1 pgs: 1 remapped; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-03-07T07:36:41.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:40 smithi160 ceph-mon[31963]: osdmap e53: 8 total, 8 up, 8 in 2024-03-07T07:36:41.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:36:41.489+0000 7fb6bb55e700 1 -- 172.21.15.178:0/4049811607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6b4102c20 msgr2=0x7fb6b41094d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:36:41.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:36:41.489+0000 7fb6bb55e700 1 --2- 172.21.15.178:0/4049811607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6b4102c20 0x7fb6b41094d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac00b0a0 tx=0x7fb6ac00b3b0 comp rx=0 tx=0).stop 2024-03-07T07:36:41.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:36:41.489+0000 7fb6bb55e700 1 -- 172.21.15.178:0/4049811607 shutdown_connections 2024-03-07T07:36:41.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:36:41.489+0000 7fb6bb55e700 1 --2- 172.21.15.178:0/4049811607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6b4102c20 0x7fb6b41094d0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:36:41.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:36:41.489+0000 7fb6bb55e700 1 --2- 172.21.15.178:0/4049811607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb6b4101080 0x7fb6b41014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:36:41.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:36:41.489+0000 7fb6bb55e700 1 --2- 172.21.15.178:0/4049811607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6b4102280 0x7fb6b41026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:36:41.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:36:41.489+0000 7fb6bb55e700 1 -- 172.21.15.178:0/4049811607 >> 172.21.15.178:0/4049811607 conn(0x7fb6b40fc640 msgr2=0x7fb6b40fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:36:41.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:36:41.489+0000 7fb6bb55e700 1 -- 172.21.15.178:0/4049811607 shutdown_connections 2024-03-07T07:36:41.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:36:41.489+0000 7fb6bb55e700 1 -- 172.21.15.178:0/4049811607 wait complete. 2024-03-07T07:36:41.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:36:41.490+0000 7fb6bb55e700 1 Processor -- start 2024-03-07T07:36:41.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:36:41.490+0000 7fb6bb55e700 1 -- start start 2024-03-07T07:36:41.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:36:41.490+0000 7fb6bb55e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6b4101080 0x7fb6b406b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:36:41.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:36:41.490+0000 7fb6bb55e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb6b4102280 0x7fb6b419d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:36:41.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:36:41.490+0000 7fb6bb55e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6b4102c20 0x7fb6b419f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:36:41.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:36:41.490+0000 7fb6bb55e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b410cd60 con 0x7fb6b4101080 2024-03-07T07:36:41.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:36:41.490+0000 7fb6bb55e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b410cbe0 con 0x7fb6b4102280 2024-03-07T07:36:41.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:36:41.490+0000 7fb6bb55e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b410cee0 con 0x7fb6b4102c20 2024-03-07T07:36:41.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:36:41.490+0000 7fb6b9afb700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6b4102c20 0x7fb6b419f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:36:41.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:36:41.490+0000 7fb6b9afb700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6b4102c20 0x7fb6b419f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:54600/0 (socket says 172.21.15.178:54600) 2024-03-07T07:36:41.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:36:41.490+0000 7fb6b9afb700 1 -- 172.21.15.178:0/1771162349 learned_addr learned my addr 172.21.15.178:0/1771162349 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-03-07T07:36:41.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:36:41.490+0000 7fb6b92fa700 1 --2- 172.21.15.178:0/1771162349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6b4101080 0x7fb6b406b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:36:41.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:36:41.490+0000 7fb6b8af9700 1 --2- 172.21.15.178:0/1771162349 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb6b4102280 0x7fb6b419d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:36:41.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:36:41.490+0000 7fb6b9afb700 1 -- 172.21.15.178:0/1771162349 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb6b4102280 msgr2=0x7fb6b419d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:36:41.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:36:41.490+0000 7fb6b9afb700 1 --2- 172.21.15.178:0/1771162349 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb6b4102280 0x7fb6b419d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:36:41.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:36:41.490+0000 7fb6b9afb700 1 -- 172.21.15.178:0/1771162349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6b4101080 msgr2=0x7fb6b406b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:36:41.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:36:41.490+0000 7fb6b9afb700 1 --2- 172.21.15.178:0/1771162349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6b4101080 0x7fb6b406b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:36:41.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:36:41.490+0000 7fb6b9afb700 1 -- 172.21.15.178:0/1771162349 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b4196f70 con 0x7fb6b4102c20 2024-03-07T07:36:41.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:36:41.490+0000 7fb6b9afb700 1 --2- 172.21.15.178:0/1771162349 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6b4102c20 0x7fb6b419f7e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac00b070 tx=0x7fb6ac00be60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:36:41.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:36:41.490+0000 7fb6aa7fc700 1 -- 172.21.15.178:0/1771162349 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac021020 con 0x7fb6b4102c20 2024-03-07T07:36:41.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:36:41.491+0000 7fb6bb55e700 1 -- 172.21.15.178:0/1771162349 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6b4197200 con 0x7fb6b4102c20 2024-03-07T07:36:41.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:36:41.491+0000 7fb6aa7fc700 1 -- 172.21.15.178:0/1771162349 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb6ac005520 con 0x7fb6b4102c20 2024-03-07T07:36:41.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:36:41.491+0000 7fb6aa7fc700 1 -- 172.21.15.178:0/1771162349 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac01ba00 con 0x7fb6b4102c20 2024-03-07T07:36:41.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:36:41.491+0000 7fb6bb55e700 1 -- 172.21.15.178:0/1771162349 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6b41977c0 con 0x7fb6b4102c20 2024-03-07T07:36:41.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:36:41.491+0000 7fb6bb55e700 1 -- 172.21.15.178:0/1771162349 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6b4068e10 con 0x7fb6b4102c20 2024-03-07T07:36:41.495 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:36:41.493+0000 7fb6aa7fc700 1 -- 172.21.15.178:0/1771162349 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7fb6ac024020 con 0x7fb6b4102c20 2024-03-07T07:36:41.495 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:36:41.493+0000 7fb6aa7fc700 1 --2- 172.21.15.178:0/1771162349 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fb6a00a1e10 0x7fb6a00a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:36:41.495 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:36:41.493+0000 7fb6aa7fc700 1 -- 172.21.15.178:0/1771162349 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5426+0+0 (secure 0 0 0) 0x7fb6ac00f020 con 0x7fb6b4102c20 2024-03-07T07:36:41.495 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:36:41.494+0000 7fb6b92fa700 1 --2- 172.21.15.178:0/1771162349 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fb6a00a1e10 0x7fb6a00a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:36:41.495 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:36:41.494+0000 7fb6b92fa700 1 --2- 172.21.15.178:0/1771162349 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fb6a00a1e10 0x7fb6a00a42d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb6b41020e0 tx=0x7fb6b0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:36:41.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:36:41.497+0000 7fb6aa7fc700 1 -- 172.21.15.178:0/1771162349 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb6ac01a390 con 0x7fb6b4102c20 2024-03-07T07:36:41.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:36:41.702+0000 7fb6bb55e700 1 -- 172.21.15.178:0/1771162349 --> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fb6b4107f30 con 0x7fb6a00a1e10 2024-03-07T07:36:43.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:42 smithi178 ceph-mon[36650]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:36:43.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:42 smithi178 ceph-mon[36650]: osdmap e54: 8 total, 8 up, 8 in 2024-03-07T07:36:43.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:42 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-07T07:36:43.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:42 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-07T07:36:43.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:42 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:36:43.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:42 smithi178 ceph-mon[36650]: pgmap v181: 1 pgs: 1 remapped; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-03-07T07:36:43.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:42 smithi177 ceph-mon[36578]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:36:43.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:42 smithi177 ceph-mon[36578]: osdmap e54: 8 total, 8 up, 8 in 2024-03-07T07:36:43.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:42 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-07T07:36:43.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:42 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-07T07:36:43.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:42 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:36:43.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:42 smithi177 ceph-mon[36578]: pgmap v181: 1 pgs: 1 remapped; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-03-07T07:36:43.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:42 smithi160 ceph-mon[31963]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:36:43.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:42 smithi160 ceph-mon[31963]: osdmap e54: 8 total, 8 up, 8 in 2024-03-07T07:36:43.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:42 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-07T07:36:43.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:42 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-07T07:36:43.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:42 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:36:43.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:42 smithi160 ceph-mon[31963]: pgmap v181: 1 pgs: 1 remapped; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-03-07T07:36:44.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:43 smithi178 ceph-mon[36650]: osdmap e55: 8 total, 8 up, 8 in 2024-03-07T07:36:44.106 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:43 smithi177 ceph-mon[36578]: osdmap e55: 8 total, 8 up, 8 in 2024-03-07T07:36:44.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:43 smithi160 ceph-mon[31963]: osdmap e55: 8 total, 8 up, 8 in 2024-03-07T07:36:45.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:44 smithi178 ceph-mon[36650]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-07T07:36:45.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:44 smithi178 ceph-mon[36650]: from='client.? 172.21.15.178:0/1482114435' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ee0f2c4-fca4-49fc-a98b-07aa0fcba0e8"}]: dispatch 2024-03-07T07:36:45.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:44 smithi178 ceph-mon[36650]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ee0f2c4-fca4-49fc-a98b-07aa0fcba0e8"}]: dispatch 2024-03-07T07:36:45.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:44 smithi178 ceph-mon[36650]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3ee0f2c4-fca4-49fc-a98b-07aa0fcba0e8"}]': finished 2024-03-07T07:36:45.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:44 smithi178 ceph-mon[36650]: osdmap e56: 9 total, 8 up, 9 in 2024-03-07T07:36:45.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:44 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-07T07:36:45.106 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:44 smithi177 ceph-mon[36578]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-07T07:36:45.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:44 smithi177 ceph-mon[36578]: from='client.? 172.21.15.178:0/1482114435' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ee0f2c4-fca4-49fc-a98b-07aa0fcba0e8"}]: dispatch 2024-03-07T07:36:45.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:44 smithi177 ceph-mon[36578]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ee0f2c4-fca4-49fc-a98b-07aa0fcba0e8"}]: dispatch 2024-03-07T07:36:45.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:44 smithi177 ceph-mon[36578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3ee0f2c4-fca4-49fc-a98b-07aa0fcba0e8"}]': finished 2024-03-07T07:36:45.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:44 smithi177 ceph-mon[36578]: osdmap e56: 9 total, 8 up, 9 in 2024-03-07T07:36:45.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:44 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-07T07:36:45.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:44 smithi160 ceph-mon[31963]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-07T07:36:45.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:44 smithi160 ceph-mon[31963]: from='client.? 172.21.15.178:0/1482114435' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ee0f2c4-fca4-49fc-a98b-07aa0fcba0e8"}]: dispatch 2024-03-07T07:36:45.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:44 smithi160 ceph-mon[31963]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ee0f2c4-fca4-49fc-a98b-07aa0fcba0e8"}]: dispatch 2024-03-07T07:36:45.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:44 smithi160 ceph-mon[31963]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3ee0f2c4-fca4-49fc-a98b-07aa0fcba0e8"}]': finished 2024-03-07T07:36:45.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:44 smithi160 ceph-mon[31963]: osdmap e56: 9 total, 8 up, 9 in 2024-03-07T07:36:45.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:44 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-07T07:36:46.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:45 smithi178 ceph-mon[36650]: from='client.? 172.21.15.178:0/1918977688' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-07T07:36:46.106 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:45 smithi177 ceph-mon[36578]: from='client.? 172.21.15.178:0/1918977688' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-07T07:36:46.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:45 smithi160 ceph-mon[31963]: from='client.? 172.21.15.178:0/1918977688' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-07T07:36:47.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:46 smithi178 ceph-mon[36650]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-07T07:36:47.106 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:46 smithi177 ceph-mon[36578]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-07T07:36:47.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:46 smithi160 ceph-mon[31963]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-07T07:36:49.345 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:49 smithi178 ceph-mon[36650]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-07T07:36:49.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:49 smithi177 ceph-mon[36578]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-07T07:36:49.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:49 smithi160 ceph-mon[31963]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-07T07:36:51.491 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:51 smithi178 ceph-mon[36650]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-07T07:36:51.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:51 smithi177 ceph-mon[36578]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-07T07:36:51.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:51 smithi160 ceph-mon[31963]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-07T07:36:52.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:52 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-07T07:36:52.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:52 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:36:52.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:52 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-07T07:36:52.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:52 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:36:52.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:52 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-07T07:36:52.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:52 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:36:53.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:53 smithi178 ceph-mon[36650]: Deploying daemon osd.8 on smithi178 2024-03-07T07:36:53.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:53 smithi178 ceph-mon[36650]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-07T07:36:53.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:53 smithi177 ceph-mon[36578]: Deploying daemon osd.8 on smithi178 2024-03-07T07:36:53.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:53 smithi177 ceph-mon[36578]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-07T07:36:53.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:53 smithi160 ceph-mon[31963]: Deploying daemon osd.8 on smithi178 2024-03-07T07:36:53.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:53 smithi160 ceph-mon[31963]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-07T07:36:55.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:55 smithi178 ceph-mon[36650]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-07T07:36:55.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:55 smithi177 ceph-mon[36578]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-07T07:36:55.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:55 smithi160 ceph-mon[31963]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-07T07:36:57.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:57 smithi178 ceph-mon[36650]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-07T07:36:57.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:57 smithi177 ceph-mon[36578]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-07T07:36:57.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:57 smithi160 ceph-mon[31963]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-07T07:36:58.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:58 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:58.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:58 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:58.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:58 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:36:58.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:36:58 smithi178 ceph-mon[36650]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-07T07:36:58.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:58 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:58.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:58 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:58.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:58 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:36:58.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:36:58 smithi160 ceph-mon[31963]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-07T07:36:58.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:58 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:58.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:58 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:36:58.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:58 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:36:58.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:36:58 smithi177 ceph-mon[36578]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-07T07:37:00.669 INFO:teuthology.orchestra.run.smithi178.stdout:Created osd(s) 8 on host 'smithi178' 2024-03-07T07:37:00.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:00.668+0000 7fb6aa7fc700 1 -- 172.21.15.178:0/1771162349 <== mgr.14152 v2:172.21.15.160:6800/419630619 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb6b4107f30 con 0x7fb6a00a1e10 2024-03-07T07:37:00.671 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:00.671+0000 7fb6bb55e700 1 -- 172.21.15.178:0/1771162349 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fb6a00a1e10 msgr2=0x7fb6a00a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:37:00.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:00.671+0000 7fb6bb55e700 1 --2- 172.21.15.178:0/1771162349 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fb6a00a1e10 0x7fb6a00a42d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb6b41020e0 tx=0x7fb6b0009190 comp rx=0 tx=0).stop 2024-03-07T07:37:00.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:00.671+0000 7fb6bb55e700 1 -- 172.21.15.178:0/1771162349 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6b4102c20 msgr2=0x7fb6b419f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:37:00.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:00.671+0000 7fb6bb55e700 1 --2- 172.21.15.178:0/1771162349 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6b4102c20 0x7fb6b419f7e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac00b070 tx=0x7fb6ac00be60 comp rx=0 tx=0).stop 2024-03-07T07:37:00.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:00.671+0000 7fb6bb55e700 1 -- 172.21.15.178:0/1771162349 shutdown_connections 2024-03-07T07:37:00.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:00.671+0000 7fb6bb55e700 1 --2- 172.21.15.178:0/1771162349 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fb6a00a1e10 0x7fb6a00a42d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:37:00.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:00.671+0000 7fb6bb55e700 1 --2- 172.21.15.178:0/1771162349 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6b4102c20 0x7fb6b419f7e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:37:00.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:00.671+0000 7fb6bb55e700 1 --2- 172.21.15.178:0/1771162349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6b4101080 0x7fb6b406b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:37:00.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:00.671+0000 7fb6bb55e700 1 --2- 172.21.15.178:0/1771162349 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb6b4102280 0x7fb6b419d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:37:00.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:00.671+0000 7fb6bb55e700 1 -- 172.21.15.178:0/1771162349 >> 172.21.15.178:0/1771162349 conn(0x7fb6b40fc640 msgr2=0x7fb6b4105580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:37:00.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:00.671+0000 7fb6bb55e700 1 -- 172.21.15.178:0/1771162349 shutdown_connections 2024-03-07T07:37:00.673 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:00.671+0000 7fb6bb55e700 1 -- 172.21.15.178:0/1771162349 wait complete. 2024-03-07T07:37:01.399 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:01 smithi178 ceph-mon[36650]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-07T07:37:01.399 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:01 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:01.399 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:01 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:01.399 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:01 smithi178 ceph-mon[36650]: from='osd.8 [v2:172.21.15.178:6816/688015531,v1:172.21.15.178:6817/688015531]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-07T07:37:01.400 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:01 smithi178 ceph-mon[36650]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-07T07:37:01.425 DEBUG:teuthology.orchestra.run.smithi178:osd.8> sudo journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@osd.8.service 2024-03-07T07:37:01.428 INFO:tasks.cephadm:Deploying osd.9 on smithi160 with /dev/vg_nvme/lv_1... 2024-03-07T07:37:01.428 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-07T07:37:01.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:01 smithi177 ceph-mon[36578]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-07T07:37:01.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:01 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:01.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:01 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:01.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:01 smithi177 ceph-mon[36578]: from='osd.8 [v2:172.21.15.178:6816/688015531,v1:172.21.15.178:6817/688015531]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-07T07:37:01.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:01 smithi177 ceph-mon[36578]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-07T07:37:01.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:01 smithi160 ceph-mon[31963]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-07T07:37:01.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:01 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:01.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:01 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:01.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:01 smithi160 ceph-mon[31963]: from='osd.8 [v2:172.21.15.178:6816/688015531,v1:172.21.15.178:6817/688015531]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-07T07:37:01.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:01 smithi160 ceph-mon[31963]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-07T07:37:01.685 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:37:02.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:02 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:02.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:02 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:02.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:02 smithi178 ceph-mon[36650]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-07T07:37:02.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:02 smithi178 ceph-mon[36650]: osdmap e57: 9 total, 8 up, 9 in 2024-03-07T07:37:02.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:02 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-07T07:37:02.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:02 smithi178 ceph-mon[36650]: from='osd.8 [v2:172.21.15.178:6816/688015531,v1:172.21.15.178:6817/688015531]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-03-07T07:37:02.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:02 smithi178 ceph-mon[36650]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-03-07T07:37:02.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:02 smithi178 ceph-mon[36650]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-07T07:37:02.807 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:37:02 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:37:02.664+0000 7fb4ce315700 -1 osd.8 0 waiting for initial osdmap 2024-03-07T07:37:02.807 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:37:02 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:37:02.677+0000 7fb4ca108700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-07T07:37:02.834 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:02 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:02.834 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:02 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:02.834 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:02 smithi160 ceph-mon[31963]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-07T07:37:02.834 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:02 smithi160 ceph-mon[31963]: osdmap e57: 9 total, 8 up, 9 in 2024-03-07T07:37:02.834 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:02 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-07T07:37:02.834 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:02 smithi160 ceph-mon[31963]: from='osd.8 [v2:172.21.15.178:6816/688015531,v1:172.21.15.178:6817/688015531]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-03-07T07:37:02.834 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:02 smithi160 ceph-mon[31963]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-03-07T07:37:02.834 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:02 smithi160 ceph-mon[31963]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-07T07:37:02.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:02 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:02.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:02 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:02.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:02 smithi177 ceph-mon[36578]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-07T07:37:02.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:02 smithi177 ceph-mon[36578]: osdmap e57: 9 total, 8 up, 9 in 2024-03-07T07:37:02.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:02 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-07T07:37:02.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:02 smithi177 ceph-mon[36578]: from='osd.8 [v2:172.21.15.178:6816/688015531,v1:172.21.15.178:6817/688015531]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-03-07T07:37:02.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:02 smithi177 ceph-mon[36578]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-03-07T07:37:02.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:02 smithi177 ceph-mon[36578]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-07T07:37:03.737 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-07T07:37:03.737 INFO:teuthology.orchestra.run.smithi160.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-07T07:37:03.737 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 10+0 records in 2024-03-07T07:37:03.737 INFO:teuthology.orchestra.run.smithi160.stderr:10+0 records out 2024-03-07T07:37:03.737 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0117003 s, 896 MB/s 2024-03-07T07:37:03.738 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping successful for: 2024-03-07T07:37:03.738 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:03 smithi160 ceph-mon[31963]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-03-07T07:37:03.738 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:03 smithi160 ceph-mon[31963]: osdmap e58: 9 total, 8 up, 9 in 2024-03-07T07:37:03.738 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:03 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-07T07:37:03.738 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:03 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-07T07:37:03.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:03 smithi178 ceph-mon[36650]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-03-07T07:37:03.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:03 smithi178 ceph-mon[36650]: osdmap e58: 9 total, 8 up, 9 in 2024-03-07T07:37:03.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:03 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-07T07:37:03.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:03 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-07T07:37:04.106 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:03 smithi177 ceph-mon[36578]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-03-07T07:37:04.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:03 smithi177 ceph-mon[36578]: osdmap e58: 9 total, 8 up, 9 in 2024-03-07T07:37:04.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:03 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-07T07:37:04.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:03 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-07T07:37:04.304 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi160:vg_nvme/lv_1 2024-03-07T07:37:04.548 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:37:04.906 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:04 smithi178 ceph-mon[36650]: purged_snaps scrub starts 2024-03-07T07:37:04.906 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:04 smithi178 ceph-mon[36650]: purged_snaps scrub ok 2024-03-07T07:37:04.906 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:04 smithi178 ceph-mon[36650]: osd.8 [v2:172.21.15.178:6816/688015531,v1:172.21.15.178:6817/688015531] boot 2024-03-07T07:37:04.906 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:04 smithi178 ceph-mon[36650]: osdmap e59: 9 total, 9 up, 9 in 2024-03-07T07:37:04.906 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:04 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-07T07:37:04.906 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:04 smithi178 ceph-mon[36650]: pgmap v197: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-07T07:37:04.906 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:04 smithi178 ceph-mon[36650]: osdmap e60: 9 total, 9 up, 9 in 2024-03-07T07:37:04.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:04 smithi160 ceph-mon[31963]: purged_snaps scrub starts 2024-03-07T07:37:04.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:04 smithi160 ceph-mon[31963]: purged_snaps scrub ok 2024-03-07T07:37:04.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:04 smithi160 ceph-mon[31963]: osd.8 [v2:172.21.15.178:6816/688015531,v1:172.21.15.178:6817/688015531] boot 2024-03-07T07:37:04.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:04 smithi160 ceph-mon[31963]: osdmap e59: 9 total, 9 up, 9 in 2024-03-07T07:37:04.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:04 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-07T07:37:04.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:04 smithi160 ceph-mon[31963]: pgmap v197: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-07T07:37:04.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:04 smithi160 ceph-mon[31963]: osdmap e60: 9 total, 9 up, 9 in 2024-03-07T07:37:05.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:04 smithi177 ceph-mon[36578]: purged_snaps scrub starts 2024-03-07T07:37:05.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:04 smithi177 ceph-mon[36578]: purged_snaps scrub ok 2024-03-07T07:37:05.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:04 smithi177 ceph-mon[36578]: osd.8 [v2:172.21.15.178:6816/688015531,v1:172.21.15.178:6817/688015531] boot 2024-03-07T07:37:05.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:04 smithi177 ceph-mon[36578]: osdmap e59: 9 total, 9 up, 9 in 2024-03-07T07:37:05.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:04 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-07T07:37:05.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:04 smithi177 ceph-mon[36578]: pgmap v197: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-07T07:37:05.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:04 smithi177 ceph-mon[36578]: osdmap e60: 9 total, 9 up, 9 in 2024-03-07T07:37:06.239 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:05 smithi160 ceph-mon[31963]: Detected new or changed devices on smithi178 2024-03-07T07:37:06.239 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:05 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:06.239 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:05 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:06.239 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:05 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:37:06.240 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:05 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:37:06.240 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:05 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:37:06.240 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:05 smithi160 ceph-mon[31963]: Adjusting osd_memory_target on smithi178 to 5693M 2024-03-07T07:37:06.240 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:05 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:06.240 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:05 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:37:06.240 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:05 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:37:06.240 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:05 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:06.240 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:05 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:37:06.240 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:05 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:37:06.240 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:05 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:37:06.241 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:05 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:06.241 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:05 smithi160 ceph-mon[31963]: osdmap e61: 9 total, 9 up, 9 in 2024-03-07T07:37:06.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:06.242+0000 7fc835e28700 1 -- 172.21.15.160:0/1707418843 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc830102280 msgr2=0x7fc8301026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:37:06.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:06.242+0000 7fc835e28700 1 --2- 172.21.15.160:0/1707418843 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc830102280 0x7fc8301026e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc8180099c0 tx=0x7fc818009cd0 comp rx=0 tx=0).stop 2024-03-07T07:37:06.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:06.242+0000 7fc835e28700 1 -- 172.21.15.160:0/1707418843 shutdown_connections 2024-03-07T07:37:06.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:06.242+0000 7fc835e28700 1 --2- 172.21.15.160:0/1707418843 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc830102c20 0x7fc8301094d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:37:06.243 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:06.242+0000 7fc835e28700 1 --2- 172.21.15.160:0/1707418843 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc830101080 0x7fc8301014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:37:06.244 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:06.242+0000 7fc835e28700 1 --2- 172.21.15.160:0/1707418843 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc830102280 0x7fc8301026e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:37:06.244 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:06.242+0000 7fc835e28700 1 -- 172.21.15.160:0/1707418843 >> 172.21.15.160:0/1707418843 conn(0x7fc8300fc620 msgr2=0x7fc8300fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:37:06.244 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:06.242+0000 7fc835e28700 1 -- 172.21.15.160:0/1707418843 shutdown_connections 2024-03-07T07:37:06.244 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:06.242+0000 7fc835e28700 1 -- 172.21.15.160:0/1707418843 wait complete. 2024-03-07T07:37:06.244 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:06.242+0000 7fc835e28700 1 Processor -- start 2024-03-07T07:37:06.244 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:06.242+0000 7fc835e28700 1 -- start start 2024-03-07T07:37:06.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:06.243+0000 7fc835e28700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc830101080 0x7fc830197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:37:06.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:06.243+0000 7fc835e28700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc830102280 0x7fc830197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:37:06.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:06.243+0000 7fc835e28700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc830102c20 0x7fc83019e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:37:06.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:06.243+0000 7fc835e28700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc83010ccc0 con 0x7fc830101080 2024-03-07T07:37:06.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:06.243+0000 7fc835e28700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc83010cb40 con 0x7fc830102c20 2024-03-07T07:37:06.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:06.243+0000 7fc835e28700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc83010ce40 con 0x7fc830102280 2024-03-07T07:37:06.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:06.243+0000 7fc82f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc830101080 0x7fc830197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:37:06.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:06.243+0000 7fc82f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc830101080 0x7fc830197310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:52304/0 (socket says 172.21.15.160:52304) 2024-03-07T07:37:06.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:06.243+0000 7fc82f7fe700 1 -- 172.21.15.160:0/3597523255 learned_addr learned my addr 172.21.15.160:0/3597523255 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:37:06.248 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:06.243+0000 7fc826ffd700 1 --2- 172.21.15.160:0/3597523255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc830102280 0x7fc830197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:37:06.248 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:06.243+0000 7fc82f7fe700 1 -- 172.21.15.160:0/3597523255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc830102280 msgr2=0x7fc830197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:37:06.248 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:06.243+0000 7fc82f7fe700 1 --2- 172.21.15.160:0/3597523255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc830102280 0x7fc830197850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:37:06.248 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:06.243+0000 7fc82f7fe700 1 -- 172.21.15.160:0/3597523255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc830102c20 msgr2=0x7fc83019e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:37:06.248 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:06.243+0000 7fc82ffff700 1 --2- 172.21.15.160:0/3597523255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc830102c20 0x7fc83019e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:37:06.248 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:06.243+0000 7fc82f7fe700 1 --2- 172.21.15.160:0/3597523255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc830102c20 0x7fc83019e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:37:06.248 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:06.243+0000 7fc82f7fe700 1 -- 172.21.15.160:0/3597523255 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc83019ef70 con 0x7fc830101080 2024-03-07T07:37:06.248 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:06.243+0000 7fc82f7fe700 1 --2- 172.21.15.160:0/3597523255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc830101080 0x7fc830197310 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc82000c9a0 tx=0x7fc82000cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:37:06.248 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:06.244+0000 7fc82d7fa700 1 -- 172.21.15.160:0/3597523255 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc820004220 con 0x7fc830101080 2024-03-07T07:37:06.248 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:06.244+0000 7fc835e28700 1 -- 172.21.15.160:0/3597523255 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc83019f260 con 0x7fc830101080 2024-03-07T07:37:06.248 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:06.244+0000 7fc82d7fa700 1 -- 172.21.15.160:0/3597523255 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc820007d40 con 0x7fc830101080 2024-03-07T07:37:06.248 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:06.244+0000 7fc82d7fa700 1 -- 172.21.15.160:0/3597523255 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc82000f450 con 0x7fc830101080 2024-03-07T07:37:06.248 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:06.244+0000 7fc835e28700 1 -- 172.21.15.160:0/3597523255 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc83019f870 con 0x7fc830101080 2024-03-07T07:37:06.248 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:06.245+0000 7fc835e28700 1 -- 172.21.15.160:0/3597523255 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc830068e10 con 0x7fc830101080 2024-03-07T07:37:06.249 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:06.246+0000 7fc82d7fa700 1 -- 172.21.15.160:0/3597523255 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7fc820007860 con 0x7fc830101080 2024-03-07T07:37:06.249 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:06.246+0000 7fc82d7fa700 1 --2- 172.21.15.160:0/3597523255 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fc81c0a1e10 0x7fc81c0a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:37:06.249 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:06.246+0000 7fc826ffd700 1 --2- 172.21.15.160:0/3597523255 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fc81c0a1e10 0x7fc81c0a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:37:06.249 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:06.247+0000 7fc82d7fa700 1 -- 172.21.15.160:0/3597523255 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5858+0+0 (secure 0 0 0) 0x7fc8200cd3a0 con 0x7fc830101080 2024-03-07T07:37:06.249 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:06.247+0000 7fc826ffd700 1 --2- 172.21.15.160:0/3597523255 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fc81c0a1e10 0x7fc81c0a42d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc818009fd0 tx=0x7fc818000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:37:06.250 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:06.250+0000 7fc82d7fa700 1 -- 172.21.15.160:0/3597523255 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc820014030 con 0x7fc830101080 2024-03-07T07:37:06.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:05 smithi178 ceph-mon[36650]: Detected new or changed devices on smithi178 2024-03-07T07:37:06.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:05 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:06.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:05 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:06.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:05 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:37:06.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:05 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:37:06.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:05 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:37:06.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:05 smithi178 ceph-mon[36650]: Adjusting osd_memory_target on smithi178 to 5693M 2024-03-07T07:37:06.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:05 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:06.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:05 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:37:06.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:05 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:37:06.308 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:05 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:06.308 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:05 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:37:06.308 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:05 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:37:06.308 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:05 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:37:06.308 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:05 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:06.308 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:05 smithi178 ceph-mon[36650]: osdmap e61: 9 total, 9 up, 9 in 2024-03-07T07:37:06.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:05 smithi177 ceph-mon[36578]: Detected new or changed devices on smithi178 2024-03-07T07:37:06.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:05 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:06.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:05 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:06.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:05 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:37:06.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:05 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:37:06.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:05 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:37:06.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:05 smithi177 ceph-mon[36578]: Adjusting osd_memory_target on smithi178 to 5693M 2024-03-07T07:37:06.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:05 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:06.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:05 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:37:06.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:05 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:37:06.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:05 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:06.358 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:05 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:37:06.358 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:05 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:37:06.358 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:05 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:37:06.358 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:05 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:06.358 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:05 smithi177 ceph-mon[36578]: osdmap e61: 9 total, 9 up, 9 in 2024-03-07T07:37:06.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:06.455+0000 7fc835e28700 1 -- 172.21.15.160:0/3597523255 --> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc830062590 con 0x7fc81c0a1e10 2024-03-07T07:37:07.293 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:06 smithi160 ceph-mon[31963]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-07T07:37:07.293 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:06 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-07T07:37:07.293 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:06 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-07T07:37:07.293 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:06 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:37:07.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:06 smithi178 ceph-mon[36650]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-07T07:37:07.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:06 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-07T07:37:07.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:06 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-07T07:37:07.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:06 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:37:07.356 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:06 smithi177 ceph-mon[36578]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-07T07:37:07.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:06 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-07T07:37:07.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:06 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-07T07:37:07.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:06 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:37:08.251 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:07 smithi160 ceph-mon[31963]: from='client.14409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:37:09.216 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:07 smithi160 ceph-mon[31963]: osdmap e62: 9 total, 9 up, 9 in 2024-03-07T07:37:09.217 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:07 smithi177 ceph-mon[36578]: from='client.14409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:37:09.217 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:07 smithi177 ceph-mon[36578]: osdmap e62: 9 total, 9 up, 9 in 2024-03-07T07:37:09.217 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:07 smithi178 ceph-mon[36650]: from='client.14409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:37:09.218 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:07 smithi178 ceph-mon[36650]: osdmap e62: 9 total, 9 up, 9 in 2024-03-07T07:37:09.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:08 smithi178 ceph-mon[36650]: osdmap e63: 9 total, 9 up, 9 in 2024-03-07T07:37:09.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:08 smithi178 ceph-mon[36650]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-07T07:37:09.358 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:08 smithi177 ceph-mon[36578]: osdmap e63: 9 total, 9 up, 9 in 2024-03-07T07:37:09.358 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:08 smithi177 ceph-mon[36578]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-07T07:37:09.429 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:08 smithi160 ceph-mon[31963]: osdmap e63: 9 total, 9 up, 9 in 2024-03-07T07:37:09.429 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:08 smithi160 ceph-mon[31963]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-07T07:37:10.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:09 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/3100777257' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "856039d7-f7ee-4dbd-b019-0073e705ce0f"}]: dispatch 2024-03-07T07:37:10.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:09 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/3100777257' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "856039d7-f7ee-4dbd-b019-0073e705ce0f"}]': finished 2024-03-07T07:37:10.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:09 smithi178 ceph-mon[36650]: osdmap e64: 10 total, 9 up, 10 in 2024-03-07T07:37:10.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:09 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-07T07:37:10.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:09 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/1852236357' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-07T07:37:10.356 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:09 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/3100777257' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "856039d7-f7ee-4dbd-b019-0073e705ce0f"}]: dispatch 2024-03-07T07:37:10.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:09 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/3100777257' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "856039d7-f7ee-4dbd-b019-0073e705ce0f"}]': finished 2024-03-07T07:37:10.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:09 smithi177 ceph-mon[36578]: osdmap e64: 10 total, 9 up, 10 in 2024-03-07T07:37:10.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:09 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-07T07:37:10.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:09 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/1852236357' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-07T07:37:10.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:09 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/3100777257' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "856039d7-f7ee-4dbd-b019-0073e705ce0f"}]: dispatch 2024-03-07T07:37:10.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:09 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/3100777257' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "856039d7-f7ee-4dbd-b019-0073e705ce0f"}]': finished 2024-03-07T07:37:10.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:09 smithi160 ceph-mon[31963]: osdmap e64: 10 total, 9 up, 10 in 2024-03-07T07:37:10.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:09 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-07T07:37:10.429 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:09 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/1852236357' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-07T07:37:11.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:10 smithi178 ceph-mon[36650]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-03-07T07:37:11.356 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:10 smithi177 ceph-mon[36578]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-03-07T07:37:11.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:10 smithi160 ceph-mon[31963]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-03-07T07:37:13.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:13 smithi160 ceph-mon[31963]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-07T07:37:13.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:13 smithi178 ceph-mon[36650]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-07T07:37:13.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:13 smithi177 ceph-mon[36578]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-07T07:37:15.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:15 smithi178 ceph-mon[36650]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-03-07T07:37:15.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:15 smithi177 ceph-mon[36578]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-03-07T07:37:15.657 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:15 smithi160 ceph-mon[31963]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-03-07T07:37:17.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:17 smithi177 ceph-mon[36578]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-03-07T07:37:17.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:17 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-07T07:37:17.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:17 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:37:17.636 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:17 smithi160 ceph-mon[31963]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-03-07T07:37:17.636 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:17 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-07T07:37:17.637 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:17 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:37:17.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:17 smithi178 ceph-mon[36650]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-03-07T07:37:17.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:17 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-07T07:37:17.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:17 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:37:18.565 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:18 smithi160 ceph-mon[31963]: Deploying daemon osd.9 on smithi160 2024-03-07T07:37:18.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:18 smithi177 ceph-mon[36578]: Deploying daemon osd.9 on smithi160 2024-03-07T07:37:18.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:18 smithi178 ceph-mon[36650]: Deploying daemon osd.9 on smithi160 2024-03-07T07:37:19.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:19 smithi177 ceph-mon[36578]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-07T07:37:19.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:19 smithi160 ceph-mon[31963]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-07T07:37:19.707 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:19 smithi178 ceph-mon[36650]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-07T07:37:21.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:21 smithi177 ceph-mon[36578]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-07T07:37:21.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:21 smithi160 ceph-mon[31963]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-07T07:37:21.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:21 smithi178 ceph-mon[36650]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-07T07:37:23.382 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:23 smithi160 ceph-mon[31963]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-07T07:37:23.383 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:23 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:23.383 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:23 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:23.383 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:23 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:37:23.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:23 smithi177 ceph-mon[36578]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-07T07:37:23.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:23 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:23.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:23 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:23.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:23 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:37:23.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:23 smithi178 ceph-mon[36650]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-07T07:37:23.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:23 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:23.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:23 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:23.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:23 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:37:25.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:25 smithi177 ceph-mon[36578]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-07T07:37:25.641 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:25 smithi160 ceph-mon[31963]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-07T07:37:25.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:25 smithi178 ceph-mon[36650]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-07T07:37:26.917 INFO:teuthology.orchestra.run.smithi160.stdout:Created osd(s) 9 on host 'smithi160' 2024-03-07T07:37:26.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:26.917+0000 7fc82d7fa700 1 -- 172.21.15.160:0/3597523255 <== mgr.14152 v2:172.21.15.160:6800/419630619 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc830062590 con 0x7fc81c0a1e10 2024-03-07T07:37:26.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:26.920+0000 7fc835e28700 1 -- 172.21.15.160:0/3597523255 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fc81c0a1e10 msgr2=0x7fc81c0a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:37:26.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:26.920+0000 7fc835e28700 1 --2- 172.21.15.160:0/3597523255 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fc81c0a1e10 0x7fc81c0a42d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc818009fd0 tx=0x7fc818000bc0 comp rx=0 tx=0).stop 2024-03-07T07:37:26.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:26.920+0000 7fc835e28700 1 -- 172.21.15.160:0/3597523255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc830101080 msgr2=0x7fc830197310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:37:26.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:26.920+0000 7fc835e28700 1 --2- 172.21.15.160:0/3597523255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc830101080 0x7fc830197310 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc82000c9a0 tx=0x7fc82000cd60 comp rx=0 tx=0).stop 2024-03-07T07:37:26.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:26.920+0000 7fc835e28700 1 -- 172.21.15.160:0/3597523255 shutdown_connections 2024-03-07T07:37:26.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:26.920+0000 7fc835e28700 1 --2- 172.21.15.160:0/3597523255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc830102c20 0x7fc83019e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:37:26.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:26.920+0000 7fc835e28700 1 --2- 172.21.15.160:0/3597523255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc830101080 0x7fc830197310 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:37:26.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:26.920+0000 7fc835e28700 1 --2- 172.21.15.160:0/3597523255 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fc81c0a1e10 0x7fc81c0a42d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:37:26.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:26.920+0000 7fc835e28700 1 --2- 172.21.15.160:0/3597523255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc830102280 0x7fc830197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:37:26.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:26.920+0000 7fc835e28700 1 -- 172.21.15.160:0/3597523255 >> 172.21.15.160:0/3597523255 conn(0x7fc8300fc620 msgr2=0x7fc8301055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:37:26.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:26.920+0000 7fc835e28700 1 -- 172.21.15.160:0/3597523255 shutdown_connections 2024-03-07T07:37:26.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:37:26.920+0000 7fc835e28700 1 -- 172.21.15.160:0/3597523255 wait complete. 2024-03-07T07:37:27.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:26 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:27.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:26 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:27.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:26 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:37:27.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:26 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:37:27.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:26 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:27.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:26 smithi177 ceph-mon[36578]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-07T07:37:27.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:26 smithi177 ceph-mon[36578]: from='osd.9 [v2:172.21.15.160:6826/3265797851,v1:172.21.15.160:6827/3265797851]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-07T07:37:27.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:26 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:27.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:26 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:27.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:26 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:37:27.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:26 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:37:27.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:26 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:27.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:26 smithi160 ceph-mon[31963]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-07T07:37:27.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:26 smithi160 ceph-mon[31963]: from='osd.9 [v2:172.21.15.160:6826/3265797851,v1:172.21.15.160:6827/3265797851]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-07T07:37:27.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:26 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:27.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:26 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:27.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:26 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:37:27.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:26 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:37:27.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:26 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:27.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:26 smithi178 ceph-mon[36650]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-07T07:37:27.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:26 smithi178 ceph-mon[36650]: from='osd.9 [v2:172.21.15.160:6826/3265797851,v1:172.21.15.160:6827/3265797851]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-07T07:37:27.629 DEBUG:teuthology.orchestra.run.smithi160:osd.9> sudo journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@osd.9.service 2024-03-07T07:37:27.631 INFO:tasks.cephadm:Deploying osd.10 on smithi177 with /dev/vg_nvme/lv_1... 2024-03-07T07:37:27.631 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-07T07:37:27.877 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.b/config 2024-03-07T07:37:27.998 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:27 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:27.999 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:27 smithi177 ceph-mon[36578]: from='osd.9 [v2:172.21.15.160:6826/3265797851,v1:172.21.15.160:6827/3265797851]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-07T07:37:27.999 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:27 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:27.999 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:27 smithi177 ceph-mon[36578]: osdmap e65: 10 total, 9 up, 10 in 2024-03-07T07:37:27.999 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:27 smithi177 ceph-mon[36578]: from='osd.9 [v2:172.21.15.160:6826/3265797851,v1:172.21.15.160:6827/3265797851]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-03-07T07:37:27.999 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:27 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-07T07:37:27.999 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:27 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:37:28.115 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:27 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:28.115 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:27 smithi160 ceph-mon[31963]: from='osd.9 [v2:172.21.15.160:6826/3265797851,v1:172.21.15.160:6827/3265797851]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-07T07:37:28.115 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:27 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:28.115 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:27 smithi160 ceph-mon[31963]: osdmap e65: 10 total, 9 up, 10 in 2024-03-07T07:37:28.115 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:27 smithi160 ceph-mon[31963]: from='osd.9 [v2:172.21.15.160:6826/3265797851,v1:172.21.15.160:6827/3265797851]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-03-07T07:37:28.115 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:27 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-07T07:37:28.115 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:27 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:37:28.116 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:37:27 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:37:27.873+0000 7ff6003f1700 -1 osd.9 0 waiting for initial osdmap 2024-03-07T07:37:28.116 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:37:27 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:37:27.883+0000 7ff5fc1e4700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-07T07:37:28.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:27 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:28.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:27 smithi178 ceph-mon[36650]: from='osd.9 [v2:172.21.15.160:6826/3265797851,v1:172.21.15.160:6827/3265797851]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-07T07:37:28.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:27 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:28.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:27 smithi178 ceph-mon[36650]: osdmap e65: 10 total, 9 up, 10 in 2024-03-07T07:37:28.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:27 smithi178 ceph-mon[36650]: from='osd.9 [v2:172.21.15.160:6826/3265797851,v1:172.21.15.160:6827/3265797851]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-03-07T07:37:28.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:27 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-07T07:37:28.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:27 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:37:28.956 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:28 smithi177 ceph-mon[36578]: from='osd.9 [v2:172.21.15.160:6826/3265797851,v1:172.21.15.160:6827/3265797851]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-03-07T07:37:28.956 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:28 smithi177 ceph-mon[36578]: osdmap e66: 10 total, 9 up, 10 in 2024-03-07T07:37:28.957 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:28 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-07T07:37:28.957 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:28 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-07T07:37:28.957 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:28 smithi177 ceph-mon[36578]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-07T07:37:29.124 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:28 smithi160 ceph-mon[31963]: from='osd.9 [v2:172.21.15.160:6826/3265797851,v1:172.21.15.160:6827/3265797851]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-03-07T07:37:29.124 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:28 smithi160 ceph-mon[31963]: osdmap e66: 10 total, 9 up, 10 in 2024-03-07T07:37:29.124 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:28 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-07T07:37:29.124 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:28 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-07T07:37:29.124 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:28 smithi160 ceph-mon[31963]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-07T07:37:29.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:28 smithi178 ceph-mon[36650]: from='osd.9 [v2:172.21.15.160:6826/3265797851,v1:172.21.15.160:6827/3265797851]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-03-07T07:37:29.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:28 smithi178 ceph-mon[36650]: osdmap e66: 10 total, 9 up, 10 in 2024-03-07T07:37:29.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:28 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-07T07:37:29.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:28 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-07T07:37:29.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:28 smithi178 ceph-mon[36650]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-07T07:37:29.828 INFO:teuthology.orchestra.run.smithi177.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-07T07:37:29.828 INFO:teuthology.orchestra.run.smithi177.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-07T07:37:29.828 INFO:teuthology.orchestra.run.smithi177.stderr: stderr: 10+0 records in 2024-03-07T07:37:29.828 INFO:teuthology.orchestra.run.smithi177.stderr:10+0 records out 2024-03-07T07:37:29.828 INFO:teuthology.orchestra.run.smithi177.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0131939 s, 795 MB/s 2024-03-07T07:37:29.828 INFO:teuthology.orchestra.run.smithi177.stderr:--> Zapping successful for: 2024-03-07T07:37:30.147 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:29 smithi177 ceph-mon[36578]: purged_snaps scrub starts 2024-03-07T07:37:30.147 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:29 smithi177 ceph-mon[36578]: purged_snaps scrub ok 2024-03-07T07:37:30.147 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:29 smithi177 ceph-mon[36578]: osd.9 [v2:172.21.15.160:6826/3265797851,v1:172.21.15.160:6827/3265797851] boot 2024-03-07T07:37:30.147 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:29 smithi177 ceph-mon[36578]: osdmap e67: 10 total, 10 up, 10 in 2024-03-07T07:37:30.147 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:29 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-07T07:37:30.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:29 smithi160 ceph-mon[31963]: purged_snaps scrub starts 2024-03-07T07:37:30.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:29 smithi160 ceph-mon[31963]: purged_snaps scrub ok 2024-03-07T07:37:30.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:29 smithi160 ceph-mon[31963]: osd.9 [v2:172.21.15.160:6826/3265797851,v1:172.21.15.160:6827/3265797851] boot 2024-03-07T07:37:30.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:29 smithi160 ceph-mon[31963]: osdmap e67: 10 total, 10 up, 10 in 2024-03-07T07:37:30.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:29 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-07T07:37:30.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:29 smithi178 ceph-mon[36650]: purged_snaps scrub starts 2024-03-07T07:37:30.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:29 smithi178 ceph-mon[36650]: purged_snaps scrub ok 2024-03-07T07:37:30.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:29 smithi178 ceph-mon[36650]: osd.9 [v2:172.21.15.160:6826/3265797851,v1:172.21.15.160:6827/3265797851] boot 2024-03-07T07:37:30.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:29 smithi178 ceph-mon[36650]: osdmap e67: 10 total, 10 up, 10 in 2024-03-07T07:37:30.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:29 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-07T07:37:30.468 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi177:vg_nvme/lv_1 2024-03-07T07:37:30.718 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.b/config 2024-03-07T07:37:31.077 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:30 smithi177 ceph-mon[36578]: osdmap e68: 10 total, 10 up, 10 in 2024-03-07T07:37:31.077 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:30 smithi177 ceph-mon[36578]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-07T07:37:31.077 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:30 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:31.077 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:30 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:31.077 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:30 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:37:31.077 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:30 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:37:31.077 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:30 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:37:31.077 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:30 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:37:31.077 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:30 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:31.077 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:30 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:37:31.078 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:30 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:37:31.078 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:30 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:31.179 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:30 smithi160 ceph-mon[31963]: osdmap e68: 10 total, 10 up, 10 in 2024-03-07T07:37:31.179 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:30 smithi160 ceph-mon[31963]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-07T07:37:31.179 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:30 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:31.179 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:30 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:31.180 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:30 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:37:31.180 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:30 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:37:31.180 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:30 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:37:31.180 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:30 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:37:31.180 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:30 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:31.180 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:30 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:37:31.180 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:30 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:37:31.180 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:30 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:31.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:30 smithi178 ceph-mon[36650]: osdmap e68: 10 total, 10 up, 10 in 2024-03-07T07:37:31.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:30 smithi178 ceph-mon[36650]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-07T07:37:31.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:30 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:31.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:30 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:31.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:30 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:37:31.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:30 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:37:31.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:30 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:37:31.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:30 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:37:31.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:30 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:31.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:30 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:37:31.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:30 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:37:31.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:30 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:32.028 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:31 smithi177 ceph-mon[36578]: Detected new or changed devices on smithi160 2024-03-07T07:37:32.028 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:31 smithi177 ceph-mon[36578]: Adjusting osd_memory_target on smithi160 to 4272M 2024-03-07T07:37:32.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:31 smithi160 ceph-mon[31963]: Detected new or changed devices on smithi160 2024-03-07T07:37:32.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:31 smithi160 ceph-mon[31963]: Adjusting osd_memory_target on smithi160 to 4272M 2024-03-07T07:37:32.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:31 smithi178 ceph-mon[36650]: Detected new or changed devices on smithi160 2024-03-07T07:37:32.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:31 smithi178 ceph-mon[36650]: Adjusting osd_memory_target on smithi160 to 4272M 2024-03-07T07:37:32.355 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:32.352+0000 7f9dbbc35700 1 -- 172.21.15.177:0/3552964193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9db41012d0 msgr2=0x7f9db4101750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:37:32.355 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:32.352+0000 7f9dbbc35700 1 --2- 172.21.15.177:0/3552964193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9db41012d0 0x7f9db4101750 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9da4009990 tx=0x7f9da4009ca0 comp rx=0 tx=0).stop 2024-03-07T07:37:32.355 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:32.352+0000 7f9dbbc35700 1 -- 172.21.15.177:0/3552964193 shutdown_connections 2024-03-07T07:37:32.355 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:32.352+0000 7f9dbbc35700 1 --2- 172.21.15.177:0/3552964193 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9db4101c90 0x7f9db4109530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:37:32.355 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:32.352+0000 7f9dbbc35700 1 --2- 172.21.15.177:0/3552964193 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9db4102700 0x7f9db4102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:37:32.355 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:32.352+0000 7f9dbbc35700 1 --2- 172.21.15.177:0/3552964193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9db41012d0 0x7f9db4101750 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:37:32.355 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:32.352+0000 7f9dbbc35700 1 -- 172.21.15.177:0/3552964193 >> 172.21.15.177:0/3552964193 conn(0x7f9db40fbdc0 msgr2=0x7f9db40fe1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:37:32.355 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:32.352+0000 7f9dbbc35700 1 -- 172.21.15.177:0/3552964193 shutdown_connections 2024-03-07T07:37:32.355 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:32.352+0000 7f9dbbc35700 1 -- 172.21.15.177:0/3552964193 wait complete. 2024-03-07T07:37:32.355 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:32.353+0000 7f9dbbc35700 1 Processor -- start 2024-03-07T07:37:32.356 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:32.353+0000 7f9dbbc35700 1 -- start start 2024-03-07T07:37:32.356 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:32.353+0000 7f9dbbc35700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9db41012d0 0x7f9db419a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:37:32.356 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:32.353+0000 7f9dbbc35700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9db4101c90 0x7f9db419aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:37:32.356 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:32.353+0000 7f9dbbc35700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9db4102700 0x7f9db41a1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:37:32.356 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:32.353+0000 7f9dbbc35700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db410cc60 con 0x7f9db41012d0 2024-03-07T07:37:32.356 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:32.353+0000 7f9dbbc35700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db410cae0 con 0x7f9db4102700 2024-03-07T07:37:32.356 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:32.353+0000 7f9dbbc35700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db410cde0 con 0x7f9db4101c90 2024-03-07T07:37:32.356 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:32.353+0000 7f9dba1d2700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9db4102700 0x7f9db41a1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:37:32.356 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:32.353+0000 7f9dba1d2700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9db4102700 0x7f9db41a1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:57904/0 (socket says 172.21.15.177:57904) 2024-03-07T07:37:32.356 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:32.353+0000 7f9dba1d2700 1 -- 172.21.15.177:0/1143245687 learned_addr learned my addr 172.21.15.177:0/1143245687 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-03-07T07:37:32.357 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:32.353+0000 7f9db99d1700 1 --2- 172.21.15.177:0/1143245687 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9db41012d0 0x7f9db419a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:37:32.357 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:32.353+0000 7f9db91d0700 1 --2- 172.21.15.177:0/1143245687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9db4101c90 0x7f9db419aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:37:32.357 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:32.354+0000 7f9dba1d2700 1 -- 172.21.15.177:0/1143245687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9db4101c90 msgr2=0x7f9db419aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:37:32.357 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:32.354+0000 7f9dba1d2700 1 --2- 172.21.15.177:0/1143245687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9db4101c90 0x7f9db419aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:37:32.357 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:32.354+0000 7f9dba1d2700 1 -- 172.21.15.177:0/1143245687 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9db41012d0 msgr2=0x7f9db419a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:37:32.357 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:32.354+0000 7f9dba1d2700 1 --2- 172.21.15.177:0/1143245687 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9db41012d0 0x7f9db419a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:37:32.357 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:32.354+0000 7f9dba1d2700 1 -- 172.21.15.177:0/1143245687 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db41a2200 con 0x7f9db4102700 2024-03-07T07:37:32.357 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:32.354+0000 7f9db99d1700 1 --2- 172.21.15.177:0/1143245687 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9db41012d0 0x7f9db419a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:37:32.357 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:32.354+0000 7f9dba1d2700 1 --2- 172.21.15.177:0/1143245687 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9db4102700 0x7f9db41a1b60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9dac00ef30 tx=0x7f9dac00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:37:32.357 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:32.354+0000 7f9daaffd700 1 -- 172.21.15.177:0/1143245687 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dac0092e0 con 0x7f9db4102700 2024-03-07T07:37:32.357 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:32.354+0000 7f9dbbc35700 1 -- 172.21.15.177:0/1143245687 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9db41a2490 con 0x7f9db4102700 2024-03-07T07:37:32.357 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:32.354+0000 7f9daaffd700 1 -- 172.21.15.177:0/1143245687 <== mon.2 v2:172.21.15.177:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9dac004bb0 con 0x7f9db4102700 2024-03-07T07:37:32.357 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:32.354+0000 7f9daaffd700 1 -- 172.21.15.177:0/1143245687 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dac004280 con 0x7f9db4102700 2024-03-07T07:37:32.358 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:32.354+0000 7f9dbbc35700 1 -- 172.21.15.177:0/1143245687 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9db41a29d0 con 0x7f9db4102700 2024-03-07T07:37:32.359 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:32.355+0000 7f9dbbc35700 1 -- 172.21.15.177:0/1143245687 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9db4068e10 con 0x7f9db4102700 2024-03-07T07:37:32.359 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:32.356+0000 7f9daaffd700 1 -- 172.21.15.177:0/1143245687 <== mon.2 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f9dac01b030 con 0x7f9db4102700 2024-03-07T07:37:32.359 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:32.356+0000 7f9daaffd700 1 --2- 172.21.15.177:0/1143245687 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f9da00a1e10 0x7f9da00a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:37:32.359 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:32.357+0000 7f9daaffd700 1 -- 172.21.15.177:0/1143245687 <== mon.2 v2:172.21.15.177:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f9dac0cd9b0 con 0x7f9db4102700 2024-03-07T07:37:32.359 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:32.357+0000 7f9db99d1700 1 --2- 172.21.15.177:0/1143245687 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f9da00a1e10 0x7f9da00a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:37:32.359 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:32.357+0000 7f9db99d1700 1 --2- 172.21.15.177:0/1143245687 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f9da00a1e10 0x7f9da00a42d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9db0011490 tx=0x7f9db0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:37:32.360 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:32.360+0000 7f9daaffd700 1 -- 172.21.15.177:0/1143245687 <== mon.2 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9dac022020 con 0x7f9db4102700 2024-03-07T07:37:32.562 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:32.562+0000 7f9dbbc35700 1 -- 172.21.15.177:0/1143245687 --> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f9db40fdef0 con 0x7f9da00a1e10 2024-03-07T07:37:33.161 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:32 smithi177 ceph-mon[36578]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-07T07:37:33.161 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:32 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-07T07:37:33.161 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:32 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-07T07:37:33.161 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:32 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:37:33.177 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:32 smithi160 ceph-mon[31963]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-07T07:37:33.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:32 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-07T07:37:33.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:32 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-07T07:37:33.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:32 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:37:33.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:32 smithi178 ceph-mon[36650]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-07T07:37:33.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:32 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-07T07:37:33.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:32 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-07T07:37:33.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:32 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:37:34.141 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:33 smithi177 ceph-mon[36578]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:37:34.177 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:33 smithi160 ceph-mon[31963]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:37:34.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:33 smithi178 ceph-mon[36650]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:37:35.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:34 smithi160 ceph-mon[31963]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-07T07:37:35.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:34 smithi178 ceph-mon[36650]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-07T07:37:35.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:34 smithi177 ceph-mon[36578]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-07T07:37:36.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:35 smithi160 ceph-mon[31963]: from='client.? 172.21.15.177:0/663651149' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bdc9899d-d720-4429-ad53-4c388a4dbe75"}]: dispatch 2024-03-07T07:37:36.179 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:35 smithi160 ceph-mon[31963]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bdc9899d-d720-4429-ad53-4c388a4dbe75"}]: dispatch 2024-03-07T07:37:36.179 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:35 smithi160 ceph-mon[31963]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bdc9899d-d720-4429-ad53-4c388a4dbe75"}]': finished 2024-03-07T07:37:36.179 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:35 smithi160 ceph-mon[31963]: osdmap e69: 11 total, 10 up, 11 in 2024-03-07T07:37:36.179 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:35 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-07T07:37:36.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:35 smithi178 ceph-mon[36650]: from='client.? 172.21.15.177:0/663651149' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bdc9899d-d720-4429-ad53-4c388a4dbe75"}]: dispatch 2024-03-07T07:37:36.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:35 smithi178 ceph-mon[36650]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bdc9899d-d720-4429-ad53-4c388a4dbe75"}]: dispatch 2024-03-07T07:37:36.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:35 smithi178 ceph-mon[36650]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bdc9899d-d720-4429-ad53-4c388a4dbe75"}]': finished 2024-03-07T07:37:36.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:35 smithi178 ceph-mon[36650]: osdmap e69: 11 total, 10 up, 11 in 2024-03-07T07:37:36.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:35 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-07T07:37:36.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:35 smithi177 ceph-mon[36578]: from='client.? 172.21.15.177:0/663651149' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bdc9899d-d720-4429-ad53-4c388a4dbe75"}]: dispatch 2024-03-07T07:37:36.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:35 smithi177 ceph-mon[36578]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bdc9899d-d720-4429-ad53-4c388a4dbe75"}]: dispatch 2024-03-07T07:37:36.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:35 smithi177 ceph-mon[36578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bdc9899d-d720-4429-ad53-4c388a4dbe75"}]': finished 2024-03-07T07:37:36.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:35 smithi177 ceph-mon[36578]: osdmap e69: 11 total, 10 up, 11 in 2024-03-07T07:37:36.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:35 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-07T07:37:37.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:36 smithi160 ceph-mon[31963]: from='client.? 172.21.15.177:0/2823413355' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-07T07:37:37.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:36 smithi160 ceph-mon[31963]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-07T07:37:37.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:36 smithi178 ceph-mon[36650]: from='client.? 172.21.15.177:0/2823413355' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-07T07:37:37.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:36 smithi178 ceph-mon[36650]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-07T07:37:37.356 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:36 smithi177 ceph-mon[36578]: from='client.? 172.21.15.177:0/2823413355' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-07T07:37:37.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:36 smithi177 ceph-mon[36578]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-07T07:37:39.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:39 smithi178 ceph-mon[36650]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-07T07:37:39.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:39 smithi177 ceph-mon[36578]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-07T07:37:39.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:39 smithi160 ceph-mon[31963]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-07T07:37:41.561 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:41 smithi177 ceph-mon[36578]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-07T07:37:41.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:41 smithi160 ceph-mon[31963]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-07T07:37:41.808 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:41 smithi178 ceph-mon[36650]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-07T07:37:43.579 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:43 smithi177 ceph-mon[36578]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-07T07:37:43.579 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:43 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-07T07:37:43.579 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:43 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:37:43.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:43 smithi160 ceph-mon[31963]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-07T07:37:43.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:43 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-07T07:37:43.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:43 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:37:43.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:43 smithi178 ceph-mon[36650]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-07T07:37:43.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:43 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-07T07:37:43.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:43 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:37:44.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:44 smithi177 ceph-mon[36578]: Deploying daemon osd.10 on smithi177 2024-03-07T07:37:44.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:44 smithi160 ceph-mon[31963]: Deploying daemon osd.10 on smithi177 2024-03-07T07:37:44.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:44 smithi178 ceph-mon[36650]: Deploying daemon osd.10 on smithi177 2024-03-07T07:37:45.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:45 smithi177 ceph-mon[36578]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-07T07:37:45.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:45 smithi160 ceph-mon[31963]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-07T07:37:45.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:45 smithi178 ceph-mon[36650]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-07T07:37:47.526 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:47 smithi177 ceph-mon[36578]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-07T07:37:47.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:47 smithi160 ceph-mon[31963]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-07T07:37:47.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:47 smithi178 ceph-mon[36650]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-07T07:37:49.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:49 smithi177 ceph-mon[36578]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-07T07:37:49.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:49 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:49.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:49 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:49.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:49 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:37:49.608 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:49 smithi178 ceph-mon[36650]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-07T07:37:49.608 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:49 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:49.608 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:49 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:49.608 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:49 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:37:49.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:49 smithi160 ceph-mon[31963]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-07T07:37:49.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:49 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:49.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:49 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:49.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:49 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:37:51.441 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:51 smithi177 ceph-mon[36578]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-07T07:37:51.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:51 smithi160 ceph-mon[31963]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-07T07:37:51.775 INFO:teuthology.orchestra.run.smithi177.stdout:Created osd(s) 10 on host 'smithi177' 2024-03-07T07:37:51.775 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:51.774+0000 7f9daaffd700 1 -- 172.21.15.177:0/1143245687 <== mgr.14152 v2:172.21.15.160:6800/419630619 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f9db40fdef0 con 0x7f9da00a1e10 2024-03-07T07:37:51.777 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:51.776+0000 7f9dbbc35700 1 -- 172.21.15.177:0/1143245687 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f9da00a1e10 msgr2=0x7f9da00a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:37:51.777 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:51.776+0000 7f9dbbc35700 1 --2- 172.21.15.177:0/1143245687 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f9da00a1e10 0x7f9da00a42d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9db0011490 tx=0x7f9db0009260 comp rx=0 tx=0).stop 2024-03-07T07:37:51.777 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:51.776+0000 7f9dbbc35700 1 -- 172.21.15.177:0/1143245687 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9db4102700 msgr2=0x7f9db41a1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:37:51.777 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:51.776+0000 7f9dbbc35700 1 --2- 172.21.15.177:0/1143245687 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9db4102700 0x7f9db41a1b60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9dac00ef30 tx=0x7f9dac00c3a0 comp rx=0 tx=0).stop 2024-03-07T07:37:51.777 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:51.776+0000 7f9dbbc35700 1 -- 172.21.15.177:0/1143245687 shutdown_connections 2024-03-07T07:37:51.778 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:51.776+0000 7f9dbbc35700 1 --2- 172.21.15.177:0/1143245687 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9db4102700 0x7f9db41a1b60 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:37:51.778 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:51.776+0000 7f9dbbc35700 1 --2- 172.21.15.177:0/1143245687 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9db41012d0 0x7f9db419a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:37:51.778 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:51.776+0000 7f9dbbc35700 1 --2- 172.21.15.177:0/1143245687 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f9da00a1e10 0x7f9da00a42d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:37:51.778 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:51.776+0000 7f9dbbc35700 1 --2- 172.21.15.177:0/1143245687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9db4101c90 0x7f9db419aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:37:51.778 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:51.776+0000 7f9dbbc35700 1 -- 172.21.15.177:0/1143245687 >> 172.21.15.177:0/1143245687 conn(0x7f9db40fbdc0 msgr2=0x7f9db40fd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:37:51.778 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:51.777+0000 7f9dbbc35700 1 -- 172.21.15.177:0/1143245687 shutdown_connections 2024-03-07T07:37:51.778 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-07T07:37:51.777+0000 7f9dbbc35700 1 -- 172.21.15.177:0/1143245687 wait complete. 2024-03-07T07:37:51.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:51 smithi178 ceph-mon[36650]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-07T07:37:52.493 DEBUG:teuthology.orchestra.run.smithi177:osd.10> sudo journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@osd.10.service 2024-03-07T07:37:52.495 INFO:tasks.cephadm:Deploying osd.11 on smithi178 with /dev/vg_nvme/lv_1... 2024-03-07T07:37:52.496 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-07T07:37:52.737 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.c/config 2024-03-07T07:37:52.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:52 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:52.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:52 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:52.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:52 smithi178 ceph-mon[36650]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-07T07:37:52.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:52 smithi178 ceph-mon[36650]: from='osd.10 [v2:172.21.15.177:6824/2867473412,v1:172.21.15.177:6825/2867473412]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-07T07:37:52.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:52 smithi178 ceph-mon[36650]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-07T07:37:52.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:52 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:52.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:52 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:53.106 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:52 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:53.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:52 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:53.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:52 smithi177 ceph-mon[36578]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-07T07:37:53.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:52 smithi177 ceph-mon[36578]: from='osd.10 [v2:172.21.15.177:6824/2867473412,v1:172.21.15.177:6825/2867473412]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-07T07:37:53.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:52 smithi177 ceph-mon[36578]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-07T07:37:53.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:52 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:53.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:52 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:53.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:52 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:53.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:52 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:53.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:52 smithi160 ceph-mon[31963]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-07T07:37:53.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:52 smithi160 ceph-mon[31963]: from='osd.10 [v2:172.21.15.177:6824/2867473412,v1:172.21.15.177:6825/2867473412]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-07T07:37:53.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:52 smithi160 ceph-mon[31963]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-07T07:37:53.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:52 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:53.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:52 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:53.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:53 smithi177 ceph-mon[36578]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-07T07:37:53.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:53 smithi177 ceph-mon[36578]: osdmap e70: 11 total, 10 up, 11 in 2024-03-07T07:37:53.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:53 smithi177 ceph-mon[36578]: from='osd.10 [v2:172.21.15.177:6824/2867473412,v1:172.21.15.177:6825/2867473412]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-03-07T07:37:53.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:53 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-07T07:37:53.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:53 smithi177 ceph-mon[36578]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-03-07T07:37:53.858 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:37:53 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:37:53.772+0000 7f186001d700 -1 osd.10 0 waiting for initial osdmap 2024-03-07T07:37:53.858 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:37:53 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:37:53.783+0000 7f185be10700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-07T07:37:54.027 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:53 smithi178 ceph-mon[36650]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-07T07:37:54.027 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:53 smithi178 ceph-mon[36650]: osdmap e70: 11 total, 10 up, 11 in 2024-03-07T07:37:54.027 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:53 smithi178 ceph-mon[36650]: from='osd.10 [v2:172.21.15.177:6824/2867473412,v1:172.21.15.177:6825/2867473412]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-03-07T07:37:54.027 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:53 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-07T07:37:54.028 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:53 smithi178 ceph-mon[36650]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-03-07T07:37:54.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:53 smithi160 ceph-mon[31963]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-07T07:37:54.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:53 smithi160 ceph-mon[31963]: osdmap e70: 11 total, 10 up, 11 in 2024-03-07T07:37:54.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:53 smithi160 ceph-mon[31963]: from='osd.10 [v2:172.21.15.177:6824/2867473412,v1:172.21.15.177:6825/2867473412]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-03-07T07:37:54.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:53 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-07T07:37:54.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:53 smithi160 ceph-mon[31963]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-03-07T07:37:54.734 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-07T07:37:54.735 INFO:teuthology.orchestra.run.smithi178.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-07T07:37:54.735 INFO:teuthology.orchestra.run.smithi178.stderr: stderr: 10+0 records in 2024-03-07T07:37:54.735 INFO:teuthology.orchestra.run.smithi178.stderr:10+0 records out 2024-03-07T07:37:54.735 INFO:teuthology.orchestra.run.smithi178.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00839313 s, 1.2 GB/s 2024-03-07T07:37:54.735 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping successful for: 2024-03-07T07:37:55.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:54 smithi178 ceph-mon[36650]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2024-03-07T07:37:55.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:54 smithi178 ceph-mon[36650]: osdmap e71: 11 total, 10 up, 11 in 2024-03-07T07:37:55.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:54 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-07T07:37:55.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:54 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-07T07:37:55.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:54 smithi178 ceph-mon[36650]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-07T07:37:55.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:54 smithi178 ceph-mon[36650]: osd.10 [v2:172.21.15.177:6824/2867473412,v1:172.21.15.177:6825/2867473412] boot 2024-03-07T07:37:55.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:54 smithi178 ceph-mon[36650]: osdmap e72: 11 total, 11 up, 11 in 2024-03-07T07:37:55.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:54 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-07T07:37:55.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:54 smithi177 ceph-mon[36578]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2024-03-07T07:37:55.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:54 smithi177 ceph-mon[36578]: osdmap e71: 11 total, 10 up, 11 in 2024-03-07T07:37:55.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:54 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-07T07:37:55.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:54 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-07T07:37:55.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:54 smithi177 ceph-mon[36578]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-07T07:37:55.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:54 smithi177 ceph-mon[36578]: osd.10 [v2:172.21.15.177:6824/2867473412,v1:172.21.15.177:6825/2867473412] boot 2024-03-07T07:37:55.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:54 smithi177 ceph-mon[36578]: osdmap e72: 11 total, 11 up, 11 in 2024-03-07T07:37:55.108 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:54 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-07T07:37:55.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:54 smithi160 ceph-mon[31963]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2024-03-07T07:37:55.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:54 smithi160 ceph-mon[31963]: osdmap e71: 11 total, 10 up, 11 in 2024-03-07T07:37:55.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:54 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-07T07:37:55.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:54 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-07T07:37:55.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:54 smithi160 ceph-mon[31963]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-07T07:37:55.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:54 smithi160 ceph-mon[31963]: osd.10 [v2:172.21.15.177:6824/2867473412,v1:172.21.15.177:6825/2867473412] boot 2024-03-07T07:37:55.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:54 smithi160 ceph-mon[31963]: osdmap e72: 11 total, 11 up, 11 in 2024-03-07T07:37:55.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:54 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-07T07:37:55.473 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi178:vg_nvme/lv_1 2024-03-07T07:37:55.713 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.c/config 2024-03-07T07:37:56.048 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:55 smithi177 ceph-mon[36578]: purged_snaps scrub starts 2024-03-07T07:37:56.048 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:55 smithi177 ceph-mon[36578]: purged_snaps scrub ok 2024-03-07T07:37:56.048 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:55 smithi177 ceph-mon[36578]: osdmap e73: 11 total, 11 up, 11 in 2024-03-07T07:37:56.055 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:55 smithi178 ceph-mon[36650]: purged_snaps scrub starts 2024-03-07T07:37:56.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:55 smithi178 ceph-mon[36650]: purged_snaps scrub ok 2024-03-07T07:37:56.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:55 smithi178 ceph-mon[36650]: osdmap e73: 11 total, 11 up, 11 in 2024-03-07T07:37:56.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:55 smithi160 ceph-mon[31963]: purged_snaps scrub starts 2024-03-07T07:37:56.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:55 smithi160 ceph-mon[31963]: purged_snaps scrub ok 2024-03-07T07:37:56.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:55 smithi160 ceph-mon[31963]: osdmap e73: 11 total, 11 up, 11 in 2024-03-07T07:37:57.215 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:57 smithi178 ceph-mon[36650]: Detected new or changed devices on smithi177 2024-03-07T07:37:57.215 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:57 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:57.215 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:57 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:57.215 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:57 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:37:57.215 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:57 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:37:57.215 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:57 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:37:57.215 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:57 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:37:57.215 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:57 smithi178 ceph-mon[36650]: Adjusting osd_memory_target on smithi177 to 4270M 2024-03-07T07:37:57.215 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:57 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:57.215 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:57 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:37:57.216 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:57 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:37:57.216 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:57 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:57.216 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:57 smithi178 ceph-mon[36650]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-03-07T07:37:57.216 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:57 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:37:57.216 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:57 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:37:57.216 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:57 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:37:57.216 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:57 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:57.216 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:57 smithi178 ceph-mon[36650]: osdmap e74: 11 total, 11 up, 11 in 2024-03-07T07:37:57.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:57 smithi177 ceph-mon[36578]: Detected new or changed devices on smithi177 2024-03-07T07:37:57.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:57 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:57.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:57 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:57.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:57 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:37:57.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:57 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:37:57.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:57 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:37:57.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:57 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:37:57.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:57 smithi177 ceph-mon[36578]: Adjusting osd_memory_target on smithi177 to 4270M 2024-03-07T07:37:57.358 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:57 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:57.358 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:57 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:37:57.358 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:57 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:37:57.358 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:57 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:57.358 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:57 smithi177 ceph-mon[36578]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-03-07T07:37:57.358 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:57 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:37:57.358 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:57 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:37:57.358 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:57 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:37:57.358 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:57 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:57.358 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:57 smithi177 ceph-mon[36578]: osdmap e74: 11 total, 11 up, 11 in 2024-03-07T07:37:57.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:57.424+0000 7f17ccd43700 1 -- 172.21.15.178:0/4182199517 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17c8102280 msgr2=0x7f17c81026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:37:57.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:57.424+0000 7f17ccd43700 1 --2- 172.21.15.178:0/4182199517 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17c8102280 0x7f17c81026e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f17bc009990 tx=0x7f17bc009ca0 comp rx=0 tx=0).stop 2024-03-07T07:37:57.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:57.424+0000 7f17ccd43700 1 -- 172.21.15.178:0/4182199517 shutdown_connections 2024-03-07T07:37:57.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:57.424+0000 7f17ccd43700 1 --2- 172.21.15.178:0/4182199517 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17c8102c20 0x7f17c81094d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:37:57.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:57.424+0000 7f17ccd43700 1 --2- 172.21.15.178:0/4182199517 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f17c8101080 0x7f17c81014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:37:57.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:57.424+0000 7f17ccd43700 1 --2- 172.21.15.178:0/4182199517 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17c8102280 0x7f17c81026e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:37:57.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:57.424+0000 7f17ccd43700 1 -- 172.21.15.178:0/4182199517 >> 172.21.15.178:0/4182199517 conn(0x7f17c80fc640 msgr2=0x7f17c80fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:37:57.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:57.425+0000 7f17ccd43700 1 -- 172.21.15.178:0/4182199517 shutdown_connections 2024-03-07T07:37:57.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:57.425+0000 7f17ccd43700 1 -- 172.21.15.178:0/4182199517 wait complete. 2024-03-07T07:37:57.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:57.425+0000 7f17ccd43700 1 Processor -- start 2024-03-07T07:37:57.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:57.425+0000 7f17ccd43700 1 -- start start 2024-03-07T07:37:57.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:57.426+0000 7f17ccd43700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17c8101080 0x7f17c8197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:37:57.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:57.426+0000 7f17ccd43700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f17c8102280 0x7f17c8197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:37:57.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:57.426+0000 7f17ccd43700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17c8102c20 0x7f17c819e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:37:57.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:57.426+0000 7f17ccd43700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c810cd60 con 0x7f17c8102280 2024-03-07T07:37:57.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:57.426+0000 7f17ccd43700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c810cbe0 con 0x7f17c8101080 2024-03-07T07:37:57.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:57.426+0000 7f17ccd43700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c810cee0 con 0x7f17c8102c20 2024-03-07T07:37:57.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:57.426+0000 7f17c6d9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17c8102c20 0x7f17c819e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:37:57.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:57.426+0000 7f17c6d9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17c8102c20 0x7f17c819e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:47084/0 (socket says 172.21.15.178:47084) 2024-03-07T07:37:57.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:57.426+0000 7f17c6d9d700 1 -- 172.21.15.178:0/2193645493 learned_addr learned my addr 172.21.15.178:0/2193645493 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-03-07T07:37:57.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:57.426+0000 7f17c659c700 1 --2- 172.21.15.178:0/2193645493 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17c8101080 0x7f17c8197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:37:57.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:57.426+0000 7f17c5d9b700 1 --2- 172.21.15.178:0/2193645493 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f17c8102280 0x7f17c8197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:37:57.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:57.426+0000 7f17c6d9d700 1 -- 172.21.15.178:0/2193645493 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17c8101080 msgr2=0x7f17c8197400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:37:57.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:57.426+0000 7f17c6d9d700 1 --2- 172.21.15.178:0/2193645493 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17c8101080 0x7f17c8197400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:37:57.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:57.426+0000 7f17c6d9d700 1 -- 172.21.15.178:0/2193645493 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f17c8102280 msgr2=0x7f17c8197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:37:57.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:57.426+0000 7f17c6d9d700 1 --2- 172.21.15.178:0/2193645493 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f17c8102280 0x7f17c8197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:37:57.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:57.426+0000 7f17c6d9d700 1 -- 172.21.15.178:0/2193645493 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c819f060 con 0x7f17c8102c20 2024-03-07T07:37:57.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:57.426+0000 7f17c6d9d700 1 --2- 172.21.15.178:0/2193645493 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17c8102c20 0x7f17c819e9c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f17c000b9a0 tx=0x7f17c000bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:37:57.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:57.426+0000 7f17b77fe700 1 -- 172.21.15.178:0/2193645493 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c0019070 con 0x7f17c8102c20 2024-03-07T07:37:57.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:57.426+0000 7f17b77fe700 1 -- 172.21.15.178:0/2193645493 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f17c0014440 con 0x7f17c8102c20 2024-03-07T07:37:57.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:57.426+0000 7f17ccd43700 1 -- 172.21.15.178:0/2193645493 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17c819f350 con 0x7f17c8102c20 2024-03-07T07:37:57.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:57.426+0000 7f17b77fe700 1 -- 172.21.15.178:0/2193645493 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c00075d0 con 0x7f17c8102c20 2024-03-07T07:37:57.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:57.426+0000 7f17ccd43700 1 -- 172.21.15.178:0/2193645493 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c819f8c0 con 0x7f17c8102c20 2024-03-07T07:37:57.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:57.427+0000 7f17ccd43700 1 -- 172.21.15.178:0/2193645493 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17c8068e10 con 0x7f17c8102c20 2024-03-07T07:37:57.430 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:57 smithi160 ceph-mon[31963]: Detected new or changed devices on smithi177 2024-03-07T07:37:57.430 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:57 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:57.430 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:57 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:57.430 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:57 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:37:57.430 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:57 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:37:57.431 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:57 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:37:57.431 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:57 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:37:57.431 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:57 smithi160 ceph-mon[31963]: Adjusting osd_memory_target on smithi177 to 4270M 2024-03-07T07:37:57.431 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:57 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:57.431 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:57 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:37:57.431 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:57 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:37:57.431 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:57 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:57.431 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:57 smithi160 ceph-mon[31963]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-03-07T07:37:57.431 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:57 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:37:57.431 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:57 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:37:57.431 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:57 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:37:57.431 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:57 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:37:57.431 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:57 smithi160 ceph-mon[31963]: osdmap e74: 11 total, 11 up, 11 in 2024-03-07T07:37:57.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:57.428+0000 7f17b77fe700 1 -- 172.21.15.178:0/2193645493 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f17c001c070 con 0x7f17c8102c20 2024-03-07T07:37:57.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:57.429+0000 7f17b77fe700 1 --2- 172.21.15.178:0/2193645493 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f17b00a1e10 0x7f17b00a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:37:57.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:57.429+0000 7f17b77fe700 1 -- 172.21.15.178:0/2193645493 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6723+0+0 (secure 0 0 0) 0x7f17c00ccbe0 con 0x7f17c8102c20 2024-03-07T07:37:57.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:57.429+0000 7f17c659c700 1 --2- 172.21.15.178:0/2193645493 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f17b00a1e10 0x7f17b00a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:37:57.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:57.429+0000 7f17c659c700 1 --2- 172.21.15.178:0/2193645493 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f17b00a1e10 0x7f17b00a42d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f17b8006fd0 tx=0x7f17b8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:37:57.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:57.433+0000 7f17b77fe700 1 -- 172.21.15.178:0/2193645493 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f17c009a6d0 con 0x7f17c8102c20 2024-03-07T07:37:57.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:37:57.638+0000 7f17ccd43700 1 -- 172.21.15.178:0/2193645493 --> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f17c8062590 con 0x7f17b00a1e10 2024-03-07T07:37:58.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:58 smithi178 ceph-mon[36650]: osdmap e75: 11 total, 11 up, 11 in 2024-03-07T07:37:58.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:58 smithi178 ceph-mon[36650]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:37:58.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:58 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-07T07:37:58.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:58 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-07T07:37:58.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:58 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:37:58.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:58 smithi178 ceph-mon[36650]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-03-07T07:37:58.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:58 smithi177 ceph-mon[36578]: osdmap e75: 11 total, 11 up, 11 in 2024-03-07T07:37:58.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:58 smithi177 ceph-mon[36578]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:37:58.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:58 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-07T07:37:58.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:58 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-07T07:37:58.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:58 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:37:58.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:58 smithi177 ceph-mon[36578]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-03-07T07:37:58.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:58 smithi160 ceph-mon[31963]: osdmap e75: 11 total, 11 up, 11 in 2024-03-07T07:37:58.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:58 smithi160 ceph-mon[31963]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:37:58.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:58 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-07T07:37:58.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:58 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-07T07:37:58.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:58 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:37:58.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:58 smithi160 ceph-mon[31963]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-03-07T07:37:59.785 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:37:59 smithi178 ceph-mon[36650]: osdmap e76: 11 total, 11 up, 11 in 2024-03-07T07:37:59.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:37:59 smithi177 ceph-mon[36578]: osdmap e76: 11 total, 11 up, 11 in 2024-03-07T07:37:59.927 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:37:59 smithi160 ceph-mon[31963]: osdmap e76: 11 total, 11 up, 11 in 2024-03-07T07:38:00.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:00 smithi177 ceph-mon[36578]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-03-07T07:38:00.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:00 smithi177 ceph-mon[36578]: from='client.? 172.21.15.178:0/1805329529' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc9adb54-ef49-47c6-90e3-fc03bc0f5cda"}]: dispatch 2024-03-07T07:38:00.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:00 smithi177 ceph-mon[36578]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc9adb54-ef49-47c6-90e3-fc03bc0f5cda"}]: dispatch 2024-03-07T07:38:00.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:00 smithi177 ceph-mon[36578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cc9adb54-ef49-47c6-90e3-fc03bc0f5cda"}]': finished 2024-03-07T07:38:00.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:00 smithi177 ceph-mon[36578]: osdmap e77: 12 total, 11 up, 12 in 2024-03-07T07:38:00.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:00 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-07T07:38:00.857 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:00 smithi178 ceph-mon[36650]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-03-07T07:38:00.858 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:00 smithi178 ceph-mon[36650]: from='client.? 172.21.15.178:0/1805329529' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc9adb54-ef49-47c6-90e3-fc03bc0f5cda"}]: dispatch 2024-03-07T07:38:00.858 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:00 smithi178 ceph-mon[36650]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc9adb54-ef49-47c6-90e3-fc03bc0f5cda"}]: dispatch 2024-03-07T07:38:00.858 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:00 smithi178 ceph-mon[36650]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cc9adb54-ef49-47c6-90e3-fc03bc0f5cda"}]': finished 2024-03-07T07:38:00.858 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:00 smithi178 ceph-mon[36650]: osdmap e77: 12 total, 11 up, 12 in 2024-03-07T07:38:00.858 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:00 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-07T07:38:00.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:00 smithi160 ceph-mon[31963]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-03-07T07:38:00.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:00 smithi160 ceph-mon[31963]: from='client.? 172.21.15.178:0/1805329529' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc9adb54-ef49-47c6-90e3-fc03bc0f5cda"}]: dispatch 2024-03-07T07:38:00.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:00 smithi160 ceph-mon[31963]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc9adb54-ef49-47c6-90e3-fc03bc0f5cda"}]: dispatch 2024-03-07T07:38:00.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:00 smithi160 ceph-mon[31963]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cc9adb54-ef49-47c6-90e3-fc03bc0f5cda"}]': finished 2024-03-07T07:38:00.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:00 smithi160 ceph-mon[31963]: osdmap e77: 12 total, 11 up, 12 in 2024-03-07T07:38:00.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:00 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-07T07:38:01.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:01 smithi177 ceph-mon[36578]: from='client.? 172.21.15.178:0/3001975655' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-07T07:38:01.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:01 smithi160 ceph-mon[31963]: from='client.? 172.21.15.178:0/3001975655' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-07T07:38:02.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:01 smithi178 ceph-mon[36650]: from='client.? 172.21.15.178:0/3001975655' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-07T07:38:02.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:02 smithi177 ceph-mon[36578]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-03-07T07:38:02.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:02 smithi160 ceph-mon[31963]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-03-07T07:38:03.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:02 smithi178 ceph-mon[36650]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-03-07T07:38:05.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:05 smithi177 ceph-mon[36578]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-03-07T07:38:05.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:05 smithi178 ceph-mon[36650]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-03-07T07:38:05.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:05 smithi160 ceph-mon[31963]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-03-07T07:38:07.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:07 smithi178 ceph-mon[36650]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-07T07:38:07.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:07 smithi177 ceph-mon[36578]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-07T07:38:07.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:07 smithi160 ceph-mon[31963]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-07T07:38:08.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:08 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-07T07:38:08.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:08 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:38:08.621 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:08 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-07T07:38:08.621 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:08 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:38:08.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:08 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-07T07:38:08.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:08 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:38:09.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:09 smithi177 ceph-mon[36578]: Deploying daemon osd.11 on smithi178 2024-03-07T07:38:09.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:09 smithi177 ceph-mon[36578]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-03-07T07:38:09.651 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:09 smithi178 ceph-mon[36650]: Deploying daemon osd.11 on smithi178 2024-03-07T07:38:09.651 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:09 smithi178 ceph-mon[36650]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-03-07T07:38:09.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:09 smithi160 ceph-mon[31963]: Deploying daemon osd.11 on smithi178 2024-03-07T07:38:09.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:09 smithi160 ceph-mon[31963]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-03-07T07:38:11.591 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:11 smithi178 ceph-mon[36650]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-07T07:38:11.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:11 smithi177 ceph-mon[36578]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-07T07:38:11.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:11 smithi160 ceph-mon[31963]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-07T07:38:13.599 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:13 smithi178 ceph-mon[36650]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-07T07:38:13.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:13 smithi177 ceph-mon[36578]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-07T07:38:13.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:13 smithi160 ceph-mon[31963]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-07T07:38:14.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:14 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:14.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:14 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:14.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:14 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:38:14.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:14 smithi160 ceph-mon[31963]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-07T07:38:14.999 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:14 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:14.999 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:14 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:14.999 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:14 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:38:14.999 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:14 smithi178 ceph-mon[36650]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-07T07:38:15.106 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:14 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:15.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:14 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:15.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:14 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:38:15.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:14 smithi177 ceph-mon[36578]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-07T07:38:17.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:17.204+0000 7f17b77fe700 1 -- 172.21.15.178:0/2193645493 <== mgr.14152 v2:172.21.15.160:6800/419630619 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f17c8062590 con 0x7f17b00a1e10 2024-03-07T07:38:17.205 INFO:teuthology.orchestra.run.smithi178.stdout:Created osd(s) 11 on host 'smithi178' 2024-03-07T07:38:17.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:17.207+0000 7f17ccd43700 1 -- 172.21.15.178:0/2193645493 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f17b00a1e10 msgr2=0x7f17b00a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:17.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:17.207+0000 7f17ccd43700 1 --2- 172.21.15.178:0/2193645493 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f17b00a1e10 0x7f17b00a42d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f17b8006fd0 tx=0x7f17b8008040 comp rx=0 tx=0).stop 2024-03-07T07:38:17.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:17.207+0000 7f17ccd43700 1 -- 172.21.15.178:0/2193645493 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17c8102c20 msgr2=0x7f17c819e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:17.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:17.207+0000 7f17ccd43700 1 --2- 172.21.15.178:0/2193645493 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17c8102c20 0x7f17c819e9c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f17c000b9a0 tx=0x7f17c000bd60 comp rx=0 tx=0).stop 2024-03-07T07:38:17.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:17.207+0000 7f17ccd43700 1 -- 172.21.15.178:0/2193645493 shutdown_connections 2024-03-07T07:38:17.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:17.207+0000 7f17ccd43700 1 --2- 172.21.15.178:0/2193645493 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f17b00a1e10 0x7f17b00a42d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:17.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:17.207+0000 7f17ccd43700 1 --2- 172.21.15.178:0/2193645493 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17c8102c20 0x7f17c819e9c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:17.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:17.207+0000 7f17ccd43700 1 --2- 172.21.15.178:0/2193645493 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17c8101080 0x7f17c8197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:17.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:17.207+0000 7f17ccd43700 1 --2- 172.21.15.178:0/2193645493 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f17c8102280 0x7f17c8197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:17.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:17.207+0000 7f17ccd43700 1 -- 172.21.15.178:0/2193645493 >> 172.21.15.178:0/2193645493 conn(0x7f17c80fc640 msgr2=0x7f17c8105650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:38:17.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:17.207+0000 7f17ccd43700 1 -- 172.21.15.178:0/2193645493 shutdown_connections 2024-03-07T07:38:17.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:17.208+0000 7f17ccd43700 1 -- 172.21.15.178:0/2193645493 wait complete. 2024-03-07T07:38:17.512 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:17 smithi178 ceph-mon[36650]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-07T07:38:17.512 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:17 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:17.512 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:17 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:17.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:17 smithi177 ceph-mon[36578]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-07T07:38:17.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:17 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:17.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:17 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:17.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:17 smithi160 ceph-mon[31963]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-07T07:38:17.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:17 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:17.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:17 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:18.011 DEBUG:teuthology.orchestra.run.smithi178:osd.11> sudo journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@osd.11.service 2024-03-07T07:38:18.013 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-03-07T07:38:18.014 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph osd stat -f json 2024-03-07T07:38:18.272 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:38:18.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:18 smithi177 ceph-mon[36578]: from='osd.11 [v2:172.21.15.178:6824/4068724731,v1:172.21.15.178:6825/4068724731]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-07T07:38:18.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:18 smithi177 ceph-mon[36578]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-07T07:38:18.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:18 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:18.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:18 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:18.620 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:18 smithi160 ceph-mon[31963]: from='osd.11 [v2:172.21.15.178:6824/4068724731,v1:172.21.15.178:6825/4068724731]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-07T07:38:18.620 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:18 smithi160 ceph-mon[31963]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-07T07:38:18.620 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:18 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:18.620 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:18 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:18.754 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:18 smithi178 ceph-mon[36650]: from='osd.11 [v2:172.21.15.178:6824/4068724731,v1:172.21.15.178:6825/4068724731]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-07T07:38:18.755 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:18 smithi178 ceph-mon[36650]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-07T07:38:18.755 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:18 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:18.755 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:18 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:19.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:19 smithi177 ceph-mon[36578]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-07T07:38:19.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:19 smithi177 ceph-mon[36578]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-07T07:38:19.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:19 smithi177 ceph-mon[36578]: osdmap e78: 12 total, 11 up, 12 in 2024-03-07T07:38:19.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:19 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-07T07:38:19.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:19 smithi177 ceph-mon[36578]: from='osd.11 [v2:172.21.15.178:6824/4068724731,v1:172.21.15.178:6825/4068724731]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-03-07T07:38:19.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:19 smithi177 ceph-mon[36578]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-03-07T07:38:19.666 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:19 smithi160 ceph-mon[31963]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-07T07:38:19.666 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:19 smithi160 ceph-mon[31963]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-07T07:38:19.666 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:19 smithi160 ceph-mon[31963]: osdmap e78: 12 total, 11 up, 12 in 2024-03-07T07:38:19.667 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:19 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-07T07:38:19.667 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:19 smithi160 ceph-mon[31963]: from='osd.11 [v2:172.21.15.178:6824/4068724731,v1:172.21.15.178:6825/4068724731]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-03-07T07:38:19.667 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:19 smithi160 ceph-mon[31963]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-03-07T07:38:19.774 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:19 smithi178 ceph-mon[36650]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-07T07:38:19.774 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:19 smithi178 ceph-mon[36650]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-07T07:38:19.774 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:19 smithi178 ceph-mon[36650]: osdmap e78: 12 total, 11 up, 12 in 2024-03-07T07:38:19.774 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:19 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-07T07:38:19.775 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:19 smithi178 ceph-mon[36650]: from='osd.11 [v2:172.21.15.178:6824/4068724731,v1:172.21.15.178:6825/4068724731]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-03-07T07:38:19.775 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:19 smithi178 ceph-mon[36650]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-03-07T07:38:20.056 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:38:19 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:38:19.773+0000 7f2e56346700 -1 osd.11 0 waiting for initial osdmap 2024-03-07T07:38:20.056 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:38:19 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:38:19.785+0000 7f2e4e131700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-07T07:38:20.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.072+0000 7f02ac4e5700 1 -- 172.21.15.160:0/605418584 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02a4103d10 msgr2=0x7f02a410a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:20.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.072+0000 7f02ac4e5700 1 --2- 172.21.15.160:0/605418584 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02a4103d10 0x7f02a410a5c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f029c00b0d0 tx=0x7f029c00b3e0 comp rx=0 tx=0).stop 2024-03-07T07:38:20.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.072+0000 7f02ac4e5700 1 -- 172.21.15.160:0/605418584 shutdown_connections 2024-03-07T07:38:20.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.072+0000 7f02ac4e5700 1 --2- 172.21.15.160:0/605418584 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02a4103d10 0x7f02a410a5c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:20.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.072+0000 7f02ac4e5700 1 --2- 172.21.15.160:0/605418584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02a4102170 0x7f02a4102590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:20.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.072+0000 7f02ac4e5700 1 --2- 172.21.15.160:0/605418584 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f02a4103370 0x7f02a41037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:20.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.072+0000 7f02ac4e5700 1 -- 172.21.15.160:0/605418584 >> 172.21.15.160:0/605418584 conn(0x7f02a40fd730 msgr2=0x7f02a40ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:38:20.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.072+0000 7f02ac4e5700 1 -- 172.21.15.160:0/605418584 shutdown_connections 2024-03-07T07:38:20.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.072+0000 7f02ac4e5700 1 -- 172.21.15.160:0/605418584 wait complete. 2024-03-07T07:38:20.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.073+0000 7f02ac4e5700 1 Processor -- start 2024-03-07T07:38:20.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.073+0000 7f02ac4e5700 1 -- start start 2024-03-07T07:38:20.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.073+0000 7f02ac4e5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02a4102170 0x7f02a4075ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:20.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.073+0000 7f02ac4e5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02a4103370 0x7f02a4074540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:20.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.073+0000 7f02ac4e5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f02a4103d10 0x7f02a4074a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:20.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.073+0000 7f02ac4e5700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a410dd40 con 0x7f02a4102170 2024-03-07T07:38:20.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.073+0000 7f02ac4e5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a410dbc0 con 0x7f02a4103d10 2024-03-07T07:38:20.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.073+0000 7f02ac4e5700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a410dec0 con 0x7f02a4103370 2024-03-07T07:38:20.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.073+0000 7f02aa281700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02a4102170 0x7f02a4075ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:20.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.073+0000 7f02aa281700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02a4102170 0x7f02a4075ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:36956/0 (socket says 172.21.15.160:36956) 2024-03-07T07:38:20.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.073+0000 7f02aa281700 1 -- 172.21.15.160:0/2012070925 learned_addr learned my addr 172.21.15.160:0/2012070925 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:38:20.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.073+0000 7f02a9a80700 1 --2- 172.21.15.160:0/2012070925 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02a4103370 0x7f02a4074540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:20.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.073+0000 7f02aa281700 1 -- 172.21.15.160:0/2012070925 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02a4103370 msgr2=0x7f02a4074540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:20.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.073+0000 7f02aa281700 1 --2- 172.21.15.160:0/2012070925 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02a4103370 0x7f02a4074540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:20.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.073+0000 7f02aa281700 1 -- 172.21.15.160:0/2012070925 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f02a4103d10 msgr2=0x7f02a4074a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:20.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.073+0000 7f02aaa82700 1 --2- 172.21.15.160:0/2012070925 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f02a4103d10 0x7f02a4074a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:20.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.073+0000 7f02aa281700 1 --2- 172.21.15.160:0/2012070925 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f02a4103d10 0x7f02a4074a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:20.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.073+0000 7f02aa281700 1 -- 172.21.15.160:0/2012070925 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a4075330 con 0x7f02a4102170 2024-03-07T07:38:20.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.073+0000 7f02aa281700 1 --2- 172.21.15.160:0/2012070925 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02a4102170 0x7f02a4075ef0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f02a000d980 tx=0x7f02a000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:38:20.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.074+0000 7f029b7fe700 1 -- 172.21.15.160:0/2012070925 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a0016070 con 0x7f02a4102170 2024-03-07T07:38:20.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.074+0000 7f029b7fe700 1 -- 172.21.15.160:0/2012070925 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f02a000bb60 con 0x7f02a4102170 2024-03-07T07:38:20.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.074+0000 7f02ac4e5700 1 -- 172.21.15.160:0/2012070925 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02a41a2490 con 0x7f02a4102170 2024-03-07T07:38:20.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.074+0000 7f029b7fe700 1 -- 172.21.15.160:0/2012070925 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a000f3c0 con 0x7f02a4102170 2024-03-07T07:38:20.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.074+0000 7f02ac4e5700 1 -- 172.21.15.160:0/2012070925 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02a41a29d0 con 0x7f02a4102170 2024-03-07T07:38:20.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.074+0000 7f02ac4e5700 1 -- 172.21.15.160:0/2012070925 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02a4068e10 con 0x7f02a4102170 2024-03-07T07:38:20.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.076+0000 7f029b7fe700 1 -- 172.21.15.160:0/2012070925 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f02a0019070 con 0x7f02a4102170 2024-03-07T07:38:20.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.076+0000 7f029b7fe700 1 --2- 172.21.15.160:0/2012070925 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f02900a1e90 0x7f02900a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:20.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.077+0000 7f02a9a80700 1 --2- 172.21.15.160:0/2012070925 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f02900a1e90 0x7f02900a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:20.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.077+0000 7f029b7fe700 1 -- 172.21.15.160:0/2012070925 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f02a00cdcb0 con 0x7f02a4102170 2024-03-07T07:38:20.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.077+0000 7f02a9a80700 1 --2- 172.21.15.160:0/2012070925 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f02900a1e90 0x7f02900a4350 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0294005950 tx=0x7f02940058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:38:20.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.080+0000 7f029b7fe700 1 -- 172.21.15.160:0/2012070925 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f02a0020020 con 0x7f02a4102170 2024-03-07T07:38:20.275 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.274+0000 7f02ac4e5700 1 -- 172.21.15.160:0/2012070925 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f02a4062d60 con 0x7f02a4102170 2024-03-07T07:38:20.275 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.274+0000 7f029b7fe700 1 -- 172.21.15.160:0/2012070925 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v79) v1 ==== 74+0+133 (secure 0 0 0) 0x7f02a009b6a0 con 0x7f02a4102170 2024-03-07T07:38:20.275 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:38:20.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.277+0000 7f02ac4e5700 1 -- 172.21.15.160:0/2012070925 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f02900a1e90 msgr2=0x7f02900a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:20.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.277+0000 7f02ac4e5700 1 --2- 172.21.15.160:0/2012070925 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f02900a1e90 0x7f02900a4350 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0294005950 tx=0x7f02940058e0 comp rx=0 tx=0).stop 2024-03-07T07:38:20.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.277+0000 7f02ac4e5700 1 -- 172.21.15.160:0/2012070925 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02a4102170 msgr2=0x7f02a4075ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:20.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.277+0000 7f02ac4e5700 1 --2- 172.21.15.160:0/2012070925 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02a4102170 0x7f02a4075ef0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f02a000d980 tx=0x7f02a000dd40 comp rx=0 tx=0).stop 2024-03-07T07:38:20.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.277+0000 7f02ac4e5700 1 -- 172.21.15.160:0/2012070925 shutdown_connections 2024-03-07T07:38:20.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.277+0000 7f02ac4e5700 1 --2- 172.21.15.160:0/2012070925 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f02a4103d10 0x7f02a4074a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:20.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.277+0000 7f02ac4e5700 1 --2- 172.21.15.160:0/2012070925 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02a4102170 0x7f02a4075ef0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:20.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.277+0000 7f02ac4e5700 1 --2- 172.21.15.160:0/2012070925 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f02900a1e90 0x7f02900a4350 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:20.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.277+0000 7f02ac4e5700 1 --2- 172.21.15.160:0/2012070925 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02a4103370 0x7f02a4074540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:20.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.277+0000 7f02ac4e5700 1 -- 172.21.15.160:0/2012070925 >> 172.21.15.160:0/2012070925 conn(0x7f02a40fd730 msgr2=0x7f02a4106610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:38:20.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.277+0000 7f02ac4e5700 1 -- 172.21.15.160:0/2012070925 shutdown_connections 2024-03-07T07:38:20.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:20.277+0000 7f02ac4e5700 1 -- 172.21.15.160:0/2012070925 wait complete. 2024-03-07T07:38:20.566 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:20 smithi160 ceph-mon[31963]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-03-07T07:38:20.566 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:20 smithi160 ceph-mon[31963]: osdmap e79: 12 total, 11 up, 12 in 2024-03-07T07:38:20.566 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:20 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-07T07:38:20.566 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:20 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-07T07:38:20.566 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:20 smithi160 ceph-mon[31963]: from='osd.11 ' entity='osd.11' 2024-03-07T07:38:20.567 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:20 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/2012070925' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-07T07:38:20.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:20 smithi177 ceph-mon[36578]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-03-07T07:38:20.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:20 smithi177 ceph-mon[36578]: osdmap e79: 12 total, 11 up, 12 in 2024-03-07T07:38:20.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:20 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-07T07:38:20.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:20 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-07T07:38:20.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:20 smithi177 ceph-mon[36578]: from='osd.11 ' entity='osd.11' 2024-03-07T07:38:20.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:20 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/2012070925' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-07T07:38:20.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:20 smithi178 ceph-mon[36650]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-03-07T07:38:20.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:20 smithi178 ceph-mon[36650]: osdmap e79: 12 total, 11 up, 12 in 2024-03-07T07:38:20.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:20 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-07T07:38:20.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:20 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-07T07:38:20.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:20 smithi178 ceph-mon[36650]: from='osd.11 ' entity='osd.11' 2024-03-07T07:38:20.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:20 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/2012070925' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-07T07:38:21.012 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":79,"num_osds":12,"num_up_osds":11,"osd_up_since":1709797074,"num_in_osds":12,"osd_in_since":1709797080,"num_remapped_pgs":0} 2024-03-07T07:38:21.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:21 smithi178 ceph-mon[36650]: purged_snaps scrub starts 2024-03-07T07:38:21.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:21 smithi178 ceph-mon[36650]: purged_snaps scrub ok 2024-03-07T07:38:21.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:21 smithi178 ceph-mon[36650]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-07T07:38:21.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:21 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-07T07:38:21.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:21 smithi178 ceph-mon[36650]: osd.11 [v2:172.21.15.178:6824/4068724731,v1:172.21.15.178:6825/4068724731] boot 2024-03-07T07:38:21.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:21 smithi178 ceph-mon[36650]: osdmap e80: 12 total, 12 up, 12 in 2024-03-07T07:38:21.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:21 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-07T07:38:21.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:21 smithi177 ceph-mon[36578]: purged_snaps scrub starts 2024-03-07T07:38:21.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:21 smithi177 ceph-mon[36578]: purged_snaps scrub ok 2024-03-07T07:38:21.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:21 smithi177 ceph-mon[36578]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-07T07:38:21.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:21 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-07T07:38:21.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:21 smithi177 ceph-mon[36578]: osd.11 [v2:172.21.15.178:6824/4068724731,v1:172.21.15.178:6825/4068724731] boot 2024-03-07T07:38:21.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:21 smithi177 ceph-mon[36578]: osdmap e80: 12 total, 12 up, 12 in 2024-03-07T07:38:21.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:21 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-07T07:38:21.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:21 smithi160 ceph-mon[31963]: purged_snaps scrub starts 2024-03-07T07:38:21.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:21 smithi160 ceph-mon[31963]: purged_snaps scrub ok 2024-03-07T07:38:21.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:21 smithi160 ceph-mon[31963]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-07T07:38:21.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:21 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-07T07:38:21.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:21 smithi160 ceph-mon[31963]: osd.11 [v2:172.21.15.178:6824/4068724731,v1:172.21.15.178:6825/4068724731] boot 2024-03-07T07:38:21.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:21 smithi160 ceph-mon[31963]: osdmap e80: 12 total, 12 up, 12 in 2024-03-07T07:38:21.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:21 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-07T07:38:22.014 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph osd stat -f json 2024-03-07T07:38:22.254 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:38:22.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:22 smithi160 ceph-mon[31963]: Detected new or changed devices on smithi178 2024-03-07T07:38:22.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:22 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:22.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:22 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:22.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:22 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:38:22.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:22 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:38:22.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:22 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:38:22.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:22 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:38:22.929 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:22 smithi160 ceph-mon[31963]: Adjusting osd_memory_target on smithi178 to 4270M 2024-03-07T07:38:22.929 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:22 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:22.929 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:22 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:38:22.929 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:22 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:38:22.929 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:22 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:22.929 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:22 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:38:22.929 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:22 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:38:22.929 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:22 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:38:22.929 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:22 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:22.929 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:22 smithi160 ceph-mon[31963]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:22.929 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:22 smithi160 ceph-mon[31963]: osdmap e81: 12 total, 12 up, 12 in 2024-03-07T07:38:23.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:22 smithi178 ceph-mon[36650]: Detected new or changed devices on smithi178 2024-03-07T07:38:23.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:22 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:23.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:22 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:23.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:22 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:38:23.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:22 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:38:23.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:22 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:38:23.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:22 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:38:23.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:22 smithi178 ceph-mon[36650]: Adjusting osd_memory_target on smithi178 to 4270M 2024-03-07T07:38:23.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:22 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:23.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:22 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:38:23.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:22 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:38:23.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:22 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:23.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:22 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:38:23.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:22 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:38:23.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:22 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:38:23.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:22 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:23.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:22 smithi178 ceph-mon[36650]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:23.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:22 smithi178 ceph-mon[36650]: osdmap e81: 12 total, 12 up, 12 in 2024-03-07T07:38:23.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:22 smithi177 ceph-mon[36578]: Detected new or changed devices on smithi178 2024-03-07T07:38:23.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:22 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:23.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:22 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:23.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:22 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:38:23.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:22 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:38:23.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:22 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:38:23.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:22 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-07T07:38:23.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:22 smithi177 ceph-mon[36578]: Adjusting osd_memory_target on smithi178 to 4270M 2024-03-07T07:38:23.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:22 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:23.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:22 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:38:23.108 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:22 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:38:23.108 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:22 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:23.108 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:22 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:38:23.108 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:22 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:38:23.108 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:22 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:38:23.108 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:22 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:23.108 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:22 smithi177 ceph-mon[36578]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:23.108 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:22 smithi177 ceph-mon[36578]: osdmap e81: 12 total, 12 up, 12 in 2024-03-07T07:38:24.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.008+0000 7f9c2f31a700 1 -- 172.21.15.160:0/177082979 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c28102cf0 msgr2=0x7f9c281050e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:24.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.008+0000 7f9c2f31a700 1 --2- 172.21.15.160:0/177082979 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c28102cf0 0x7f9c281050e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9c24009990 tx=0x7f9c24009ca0 comp rx=0 tx=0).stop 2024-03-07T07:38:24.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.008+0000 7f9c2f31a700 1 -- 172.21.15.160:0/177082979 shutdown_connections 2024-03-07T07:38:24.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.008+0000 7f9c2f31a700 1 --2- 172.21.15.160:0/177082979 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c28107f50 0x7f9c2810a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:24.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.008+0000 7f9c2f31a700 1 --2- 172.21.15.160:0/177082979 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c28102cf0 0x7f9c281050e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:24.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.008+0000 7f9c2f31a700 1 --2- 172.21.15.160:0/177082979 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c28105620 0x7f9c28107a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:24.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.008+0000 7f9c2f31a700 1 -- 172.21.15.160:0/177082979 >> 172.21.15.160:0/177082979 conn(0x7f9c280fc640 msgr2=0x7f9c280feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:38:24.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.008+0000 7f9c2f31a700 1 -- 172.21.15.160:0/177082979 shutdown_connections 2024-03-07T07:38:24.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.008+0000 7f9c2f31a700 1 -- 172.21.15.160:0/177082979 wait complete. 2024-03-07T07:38:24.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.008+0000 7f9c2f31a700 1 Processor -- start 2024-03-07T07:38:24.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.008+0000 7f9c2f31a700 1 -- start start 2024-03-07T07:38:24.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.009+0000 7f9c2f31a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c28102cf0 0x7f9c28198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:24.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.009+0000 7f9c2f31a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c28105620 0x7f9c281988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:24.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.009+0000 7f9c2f31a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c28107f50 0x7f9c2819f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:24.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.009+0000 7f9c2f31a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2810dd70 con 0x7f9c28102cf0 2024-03-07T07:38:24.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.009+0000 7f9c2f31a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2810dbf0 con 0x7f9c28105620 2024-03-07T07:38:24.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.009+0000 7f9c2f31a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2810def0 con 0x7f9c28107f50 2024-03-07T07:38:24.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.009+0000 7f9c2d0b6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c28102cf0 0x7f9c28198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:24.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.009+0000 7f9c2d0b6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c28102cf0 0x7f9c28198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:36974/0 (socket says 172.21.15.160:36974) 2024-03-07T07:38:24.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.009+0000 7f9c2d0b6700 1 -- 172.21.15.160:0/1181008259 learned_addr learned my addr 172.21.15.160:0/1181008259 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:38:24.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.009+0000 7f9c2c8b5700 1 --2- 172.21.15.160:0/1181008259 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c28105620 0x7f9c281988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:24.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.009+0000 7f9c2d8b7700 1 --2- 172.21.15.160:0/1181008259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c28107f50 0x7f9c2819f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:24.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.009+0000 7f9c2d0b6700 1 -- 172.21.15.160:0/1181008259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c28107f50 msgr2=0x7f9c2819f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:24.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.009+0000 7f9c2d0b6700 1 --2- 172.21.15.160:0/1181008259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c28107f50 0x7f9c2819f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:24.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.009+0000 7f9c2d0b6700 1 -- 172.21.15.160:0/1181008259 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c28105620 msgr2=0x7f9c281988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:24.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.009+0000 7f9c2d0b6700 1 --2- 172.21.15.160:0/1181008259 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c28105620 0x7f9c281988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:24.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.009+0000 7f9c2d0b6700 1 -- 172.21.15.160:0/1181008259 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c2819ffe0 con 0x7f9c28102cf0 2024-03-07T07:38:24.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.009+0000 7f9c2d8b7700 1 --2- 172.21.15.160:0/1181008259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c28107f50 0x7f9c2819f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T07:38:24.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.009+0000 7f9c2d0b6700 1 --2- 172.21.15.160:0/1181008259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c28102cf0 0x7f9c28198380 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9c24004790 tx=0x7f9c24009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:38:24.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.009+0000 7f9c1e7fc700 1 -- 172.21.15.160:0/1181008259 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c24027070 con 0x7f9c28102cf0 2024-03-07T07:38:24.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.009+0000 7f9c1e7fc700 1 -- 172.21.15.160:0/1181008259 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c24004210 con 0x7f9c28102cf0 2024-03-07T07:38:24.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.009+0000 7f9c2f31a700 1 -- 172.21.15.160:0/1181008259 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c281a0270 con 0x7f9c28102cf0 2024-03-07T07:38:24.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.009+0000 7f9c1e7fc700 1 -- 172.21.15.160:0/1181008259 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2400f460 con 0x7f9c28102cf0 2024-03-07T07:38:24.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.009+0000 7f9c2f31a700 1 -- 172.21.15.160:0/1181008259 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c281a06c0 con 0x7f9c28102cf0 2024-03-07T07:38:24.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.010+0000 7f9c2f31a700 1 -- 172.21.15.160:0/1181008259 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c28068e10 con 0x7f9c28102cf0 2024-03-07T07:38:24.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.011+0000 7f9c1e7fc700 1 -- 172.21.15.160:0/1181008259 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f9c2402a070 con 0x7f9c28102cf0 2024-03-07T07:38:24.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.012+0000 7f9c1e7fc700 1 --2- 172.21.15.160:0/1181008259 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f9c140a1e10 0x7f9c140a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:24.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.012+0000 7f9c2c8b5700 1 --2- 172.21.15.160:0/1181008259 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f9c140a1e10 0x7f9c140a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:24.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.012+0000 7f9c1e7fc700 1 -- 172.21.15.160:0/1181008259 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9c240cec00 con 0x7f9c28102cf0 2024-03-07T07:38:24.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.012+0000 7f9c2c8b5700 1 --2- 172.21.15.160:0/1181008259 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f9c140a1e10 0x7f9c140a42d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9c18005950 tx=0x7f9c180058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:38:24.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.015+0000 7f9c1e7fc700 1 -- 172.21.15.160:0/1181008259 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9c24013070 con 0x7f9c28102cf0 2024-03-07T07:38:24.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.212+0000 7f9c2f31a700 1 -- 172.21.15.160:0/1181008259 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f9c28062d60 con 0x7f9c28102cf0 2024-03-07T07:38:24.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.213+0000 7f9c1e7fc700 1 -- 172.21.15.160:0/1181008259 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) v1 ==== 74+0+133 (secure 0 0 0) 0x7f9c2409c560 con 0x7f9c28102cf0 2024-03-07T07:38:24.213 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:38:24.215 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.215+0000 7f9c2f31a700 1 -- 172.21.15.160:0/1181008259 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f9c140a1e10 msgr2=0x7f9c140a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:24.216 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.215+0000 7f9c2f31a700 1 --2- 172.21.15.160:0/1181008259 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f9c140a1e10 0x7f9c140a42d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9c18005950 tx=0x7f9c180058e0 comp rx=0 tx=0).stop 2024-03-07T07:38:24.216 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.215+0000 7f9c2f31a700 1 -- 172.21.15.160:0/1181008259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c28102cf0 msgr2=0x7f9c28198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:24.216 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.215+0000 7f9c2f31a700 1 --2- 172.21.15.160:0/1181008259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c28102cf0 0x7f9c28198380 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9c24004790 tx=0x7f9c24009580 comp rx=0 tx=0).stop 2024-03-07T07:38:24.216 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.215+0000 7f9c2f31a700 1 -- 172.21.15.160:0/1181008259 shutdown_connections 2024-03-07T07:38:24.216 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.215+0000 7f9c2f31a700 1 --2- 172.21.15.160:0/1181008259 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f9c140a1e10 0x7f9c140a42d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:24.217 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.215+0000 7f9c2f31a700 1 --2- 172.21.15.160:0/1181008259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c28107f50 0x7f9c2819f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:24.217 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.215+0000 7f9c2f31a700 1 --2- 172.21.15.160:0/1181008259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c28102cf0 0x7f9c28198380 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:24.217 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.215+0000 7f9c2f31a700 1 --2- 172.21.15.160:0/1181008259 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c28105620 0x7f9c281988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:24.217 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.215+0000 7f9c2f31a700 1 -- 172.21.15.160:0/1181008259 >> 172.21.15.160:0/1181008259 conn(0x7f9c280fc640 msgr2=0x7f9c28103630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:38:24.217 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.215+0000 7f9c2f31a700 1 -- 172.21.15.160:0/1181008259 shutdown_connections 2024-03-07T07:38:24.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:24.215+0000 7f9c2f31a700 1 -- 172.21.15.160:0/1181008259 wait complete. 2024-03-07T07:38:24.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:24 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/1181008259' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-07T07:38:24.593 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:24 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/1181008259' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-07T07:38:24.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:24 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/1181008259' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-07T07:38:25.030 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1709797100,"num_in_osds":12,"osd_in_since":1709797080,"num_remapped_pgs":0} 2024-03-07T07:38:25.031 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph osd dump --format=json 2024-03-07T07:38:25.269 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:38:25.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:25 smithi177 ceph-mon[36578]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:25.607 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:25 smithi160 ceph-mon[31963]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:25.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:25 smithi178 ceph-mon[36650]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:27.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:26.998+0000 7f6ff6096700 1 -- 172.21.15.160:0/1723563516 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ff0101cf0 msgr2=0x7f6ff010df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:27.001 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:26.998+0000 7f6ff6096700 1 --2- 172.21.15.160:0/1723563516 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ff0101cf0 0x7f6ff010df20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6fe800b0a0 tx=0x7f6fe800b3b0 comp rx=0 tx=0).stop 2024-03-07T07:38:27.001 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:26.998+0000 7f6ff6096700 1 -- 172.21.15.160:0/1723563516 shutdown_connections 2024-03-07T07:38:27.001 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:26.998+0000 7f6ff6096700 1 --2- 172.21.15.160:0/1723563516 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ff0101cf0 0x7f6ff010df20 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:27.001 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:26.998+0000 7f6ff6096700 1 --2- 172.21.15.160:0/1723563516 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ff0107240 0x7f6ff0107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:27.001 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:26.998+0000 7f6ff6096700 1 --2- 172.21.15.160:0/1723563516 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ff0101330 0x7f6ff01017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:27.001 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:26.998+0000 7f6ff6096700 1 -- 172.21.15.160:0/1723563516 >> 172.21.15.160:0/1723563516 conn(0x7f6ff0077eb0 msgr2=0x7f6ff00782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:38:27.001 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:26.998+0000 7f6ff6096700 1 -- 172.21.15.160:0/1723563516 shutdown_connections 2024-03-07T07:38:27.001 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:26.998+0000 7f6ff6096700 1 -- 172.21.15.160:0/1723563516 wait complete. 2024-03-07T07:38:27.001 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:26.999+0000 7f6ff6096700 1 Processor -- start 2024-03-07T07:38:27.001 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:26.999+0000 7f6ff6096700 1 -- start start 2024-03-07T07:38:27.001 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:26.999+0000 7f6ff6096700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ff0101330 0x7f6ff019e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:27.002 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:26.999+0000 7f6ff6096700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ff0101cf0 0x7f6ff019eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:27.002 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:26.999+0000 7f6ff6096700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ff0107240 0x7f6ff01986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:27.002 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:26.999+0000 7f6ff6096700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff01113a0 con 0x7f6ff0101330 2024-03-07T07:38:27.002 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:26.999+0000 7f6ff6096700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff0111220 con 0x7f6ff0107240 2024-03-07T07:38:27.002 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:26.999+0000 7f6ff6096700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff0111520 con 0x7f6ff0101cf0 2024-03-07T07:38:27.002 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:26.999+0000 7f6fef7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ff0101330 0x7f6ff019e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:27.002 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:26.999+0000 7f6fef7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ff0101330 0x7f6ff019e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:36984/0 (socket says 172.21.15.160:36984) 2024-03-07T07:38:27.002 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:26.999+0000 7f6fef7fe700 1 -- 172.21.15.160:0/3262297748 learned_addr learned my addr 172.21.15.160:0/3262297748 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:38:27.002 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:26.999+0000 7f6fe6ffd700 1 --2- 172.21.15.160:0/3262297748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ff0101cf0 0x7f6ff019eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:27.002 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:26.999+0000 7f6feffff700 1 --2- 172.21.15.160:0/3262297748 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ff0107240 0x7f6ff01986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:27.002 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:26.999+0000 7f6fef7fe700 1 -- 172.21.15.160:0/3262297748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ff0101cf0 msgr2=0x7f6ff019eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:27.002 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:26.999+0000 7f6fef7fe700 1 --2- 172.21.15.160:0/3262297748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ff0101cf0 0x7f6ff019eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:27.002 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:26.999+0000 7f6fef7fe700 1 -- 172.21.15.160:0/3262297748 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ff0107240 msgr2=0x7f6ff01986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:27.002 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:26.999+0000 7f6fef7fe700 1 --2- 172.21.15.160:0/3262297748 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ff0107240 0x7f6ff01986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:27.003 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:26.999+0000 7f6fef7fe700 1 -- 172.21.15.160:0/3262297748 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff0198fb0 con 0x7f6ff0101330 2024-03-07T07:38:27.003 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:26.999+0000 7f6feffff700 1 --2- 172.21.15.160:0/3262297748 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ff0107240 0x7f6ff01986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T07:38:27.003 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:27.000+0000 7f6fef7fe700 1 --2- 172.21.15.160:0/3262297748 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ff0101330 0x7f6ff019e600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6fe000d980 tx=0x7f6fe000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:38:27.003 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:27.000+0000 7f6fed7fa700 1 -- 172.21.15.160:0/3262297748 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe0016070 con 0x7f6ff0101330 2024-03-07T07:38:27.003 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:27.000+0000 7f6ff6096700 1 -- 172.21.15.160:0/3262297748 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ff01992a0 con 0x7f6ff0101330 2024-03-07T07:38:27.003 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:27.000+0000 7f6fed7fa700 1 -- 172.21.15.160:0/3262297748 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6fe0004d10 con 0x7f6ff0101330 2024-03-07T07:38:27.003 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:27.000+0000 7f6fed7fa700 1 -- 172.21.15.160:0/3262297748 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe00055f0 con 0x7f6ff0101330 2024-03-07T07:38:27.003 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:27.000+0000 7f6ff6096700 1 -- 172.21.15.160:0/3262297748 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ff01a5580 con 0x7f6ff0101330 2024-03-07T07:38:27.004 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:27.001+0000 7f6ff6096700 1 -- 172.21.15.160:0/3262297748 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ff01027d0 con 0x7f6ff0101330 2024-03-07T07:38:27.004 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:27.002+0000 7f6fed7fa700 1 -- 172.21.15.160:0/3262297748 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f6fe0004750 con 0x7f6ff0101330 2024-03-07T07:38:27.004 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:27.003+0000 7f6fed7fa700 1 --2- 172.21.15.160:0/3262297748 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f6fdc0a1e10 0x7f6fdc0a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:27.004 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:27.003+0000 7f6fe6ffd700 1 --2- 172.21.15.160:0/3262297748 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f6fdc0a1e10 0x7f6fdc0a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:27.004 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:27.003+0000 7f6fed7fa700 1 -- 172.21.15.160:0/3262297748 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6fe00cd710 con 0x7f6ff0101330 2024-03-07T07:38:27.004 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:27.003+0000 7f6fe6ffd700 1 --2- 172.21.15.160:0/3262297748 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f6fdc0a1e10 0x7f6fdc0a42d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0199e90 tx=0x7f6fd800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:38:27.007 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:27.007+0000 7f6fed7fa700 1 -- 172.21.15.160:0/3262297748 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6fe001f020 con 0x7f6ff0101330 2024-03-07T07:38:27.200 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:27.199+0000 7f6ff6096700 1 -- 172.21.15.160:0/3262297748 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6ff01a5750 con 0x7f6ff0101330 2024-03-07T07:38:27.201 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:27.201+0000 7f6fed7fa700 1 -- 172.21.15.160:0/3262297748 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18479 (secure 0 0 0) 0x7f6fe009afe0 con 0x7f6ff0101330 2024-03-07T07:38:27.201 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:38:27.201 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":81,"fsid":"826f6582-dc54-11ee-95c6-87774f69a715","created":"2024-03-07T07:31:14.229288+0000","modified":"2024-03-07T07:38:22.331696+0000","last_up_change":"2024-03-07T07:38:20.776225+0000","last_in_change":"2024-03-07T07:38:00.508917+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T07:34:34.529424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bee66c67-fb6e-4717-9685-ca62587d5f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6803","nonce":1368482926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6805","nonce":1368482926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6809","nonce":1368482926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6807","nonce":1368482926}]},"public_addr":"172.21.15.160:6803/1368482926","cluster_addr":"172.21.15.160:6805/1368482926","heartbeat_back_addr":"172.21.15.160:6809/1368482926","heartbeat_front_addr":"172.21.15.160:6807/1368482926","state":["exists","up"]},{"osd":1,"uuid":"49236251-f750-424e-992b-bddf55762556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6801","nonce":2922391884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6803","nonce":2922391884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6807","nonce":2922391884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6805","nonce":2922391884}]},"public_addr":"172.21.15.177:6801/2922391884","cluster_addr":"172.21.15.177:6803/2922391884","heartbeat_back_addr":"172.21.15.177:6807/2922391884","heartbeat_front_addr":"172.21.15.177:6805/2922391884","state":["exists","up"]},{"osd":2,"uuid":"7a432b28-f7ee-43fe-9196-7aaeccfecf77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6801","nonce":1217901551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6803","nonce":1217901551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6807","nonce":1217901551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6805","nonce":1217901551}]},"public_addr":"172.21.15.178:6801/1217901551","cluster_addr":"172.21.15.178:6803/1217901551","heartbeat_back_addr":"172.21.15.178:6807/1217901551","heartbeat_front_addr":"172.21.15.178:6805/1217901551","state":["exists","up"]},{"osd":3,"uuid":"4ad22a20-f1be-4002-a48d-ecd3953e63f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6811","nonce":42543989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6813","nonce":42543989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6817","nonce":42543989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6815","nonce":42543989}]},"public_addr":"172.21.15.160:6811/42543989","cluster_addr":"172.21.15.160:6813/42543989","heartbeat_back_addr":"172.21.15.160:6817/42543989","heartbeat_front_addr":"172.21.15.160:6815/42543989","state":["exists","up"]},{"osd":4,"uuid":"2b65b104-ac82-4ce3-b6c5-11a399103045","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6809","nonce":88178553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6811","nonce":88178553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6815","nonce":88178553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6813","nonce":88178553}]},"public_addr":"172.21.15.177:6809/88178553","cluster_addr":"172.21.15.177:6811/88178553","heartbeat_back_addr":"172.21.15.177:6815/88178553","heartbeat_front_addr":"172.21.15.177:6813/88178553","state":["exists","up"]},{"osd":5,"uuid":"30a1e714-08ff-44e8-9264-9ef164604b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6809","nonce":964732891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6811","nonce":964732891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6815","nonce":964732891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6813","nonce":964732891}]},"public_addr":"172.21.15.178:6809/964732891","cluster_addr":"172.21.15.178:6811/964732891","heartbeat_back_addr":"172.21.15.178:6815/964732891","heartbeat_front_addr":"172.21.15.178:6813/964732891","state":["exists","up"]},{"osd":6,"uuid":"b1e1646e-f8a9-42b8-b62d-9d7b5d6a1ff4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6819","nonce":3251327001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6821","nonce":3251327001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6825","nonce":3251327001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6823","nonce":3251327001}]},"public_addr":"172.21.15.160:6819/3251327001","cluster_addr":"172.21.15.160:6821/3251327001","heartbeat_back_addr":"172.21.15.160:6825/3251327001","heartbeat_front_addr":"172.21.15.160:6823/3251327001","state":["exists","up"]},{"osd":7,"uuid":"e66b400b-9747-4f6d-b05b-db22468e2750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6817","nonce":1175500850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6819","nonce":1175500850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6823","nonce":1175500850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6821","nonce":1175500850}]},"public_addr":"172.21.15.177:6817/1175500850","cluster_addr":"172.21.15.177:6819/1175500850","heartbeat_back_addr":"172.21.15.177:6823/1175500850","heartbeat_front_addr":"172.21.15.177:6821/1175500850","state":["exists","up"]},{"osd":8,"uuid":"3ee0f2c4-fca4-49fc-a98b-07aa0fcba0e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6817","nonce":688015531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6819","nonce":688015531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6823","nonce":688015531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6821","nonce":688015531}]},"public_addr":"172.21.15.178:6817/688015531","cluster_addr":"172.21.15.178:6819/688015531","heartbeat_back_addr":"172.21.15.178:6823/688015531","heartbeat_front_addr":"172.21.15.178:6821/688015531","state":["exists","up"]},{"osd":9,"uuid":"856039d7-f7ee-4dbd-b019-0073e705ce0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6827","nonce":3265797851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6829","nonce":3265797851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6833","nonce":3265797851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6831","nonce":3265797851}]},"public_addr":"172.21.15.160:6827/3265797851","cluster_addr":"172.21.15.160:6829/3265797851","heartbeat_back_addr":"172.21.15.160:6833/3265797851","heartbeat_front_addr":"172.21.15.160:6831/3265797851","state":["exists","up"]},{"osd":10,"uuid":"bdc9899d-d720-4429-ad53-4c388a4dbe75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6825","nonce":2867473412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6827","nonce":2867473412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6831","nonce":2867473412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6829","nonce":2867473412}]},"public_addr":"172.21.15.177:6825/2867473412","cluster_addr":"172.21.15.177:6827/2867473412","heartbeat_back_addr":"172.21.15.177:6831/2867473412","heartbeat_front_addr":"172.21.15.177:6829/2867473412","state":["exists","up"]},{"osd":11,"uuid":"cc9adb54-ef49-47c6-90e3-fc03bc0f5cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6825","nonce":4068724731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6827","nonce":4068724731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6831","nonce":4068724731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6829","nonce":4068724731}]},"public_addr":"172.21.15.178:6825/4068724731","cluster_addr":"172.21.15.178:6827/4068724731","heartbeat_back_addr":"172.21.15.178:6831/4068724731","heartbeat_front_addr":"172.21.15.178:6829/4068724731","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:33:44.141585+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:07.685585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:31.044386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:56.308925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:35:21.582097+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:35:45.983540+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:36:11.530242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:36:36.844681+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:02.303437+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:27.629867+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:53.289495+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:38:18.536169+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/3027450612":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/874709722":"2024-03-08T07:32:08.190837+0000","172.21.15.160:6800/1130675225":"2024-03-08T07:31:41.166858+0000","172.21.15.160:0/3857193980":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/3861968297":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6800/3859174315":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/3013534743":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6801/3859174315":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/1310937667":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6801/1130675225":"2024-03-08T07:31:41.166858+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T07:38:27.203 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:27.203+0000 7f6ff6096700 1 -- 172.21.15.160:0/3262297748 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f6fdc0a1e10 msgr2=0x7f6fdc0a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:27.203 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:27.203+0000 7f6ff6096700 1 --2- 172.21.15.160:0/3262297748 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f6fdc0a1e10 0x7f6fdc0a42d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0199e90 tx=0x7f6fd800a300 comp rx=0 tx=0).stop 2024-03-07T07:38:27.203 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:27.203+0000 7f6ff6096700 1 -- 172.21.15.160:0/3262297748 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ff0101330 msgr2=0x7f6ff019e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:27.203 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:27.203+0000 7f6ff6096700 1 --2- 172.21.15.160:0/3262297748 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ff0101330 0x7f6ff019e600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6fe000d980 tx=0x7f6fe000dc90 comp rx=0 tx=0).stop 2024-03-07T07:38:27.203 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:27.203+0000 7f6ff6096700 1 -- 172.21.15.160:0/3262297748 shutdown_connections 2024-03-07T07:38:27.204 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:27.203+0000 7f6ff6096700 1 --2- 172.21.15.160:0/3262297748 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ff0107240 0x7f6ff01986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:27.204 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:27.203+0000 7f6ff6096700 1 --2- 172.21.15.160:0/3262297748 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ff0101330 0x7f6ff019e600 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:27.204 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:27.203+0000 7f6ff6096700 1 --2- 172.21.15.160:0/3262297748 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f6fdc0a1e10 0x7f6fdc0a42d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:27.204 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:27.203+0000 7f6ff6096700 1 --2- 172.21.15.160:0/3262297748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ff0101cf0 0x7f6ff019eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:27.204 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:27.203+0000 7f6ff6096700 1 -- 172.21.15.160:0/3262297748 >> 172.21.15.160:0/3262297748 conn(0x7f6ff0077eb0 msgr2=0x7f6ff0100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:38:27.204 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:27.203+0000 7f6ff6096700 1 -- 172.21.15.160:0/3262297748 shutdown_connections 2024-03-07T07:38:27.204 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:27.203+0000 7f6ff6096700 1 -- 172.21.15.160:0/3262297748 wait complete. 2024-03-07T07:38:27.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:27 smithi177 ceph-mon[36578]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:27.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:27 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/3262297748' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:38:27.679 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:27 smithi160 ceph-mon[31963]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:27.679 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:27 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/3262297748' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:38:27.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:27 smithi178 ceph-mon[36650]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:27.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:27 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/3262297748' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:38:27.885 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-07T07:34:34.529424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-07T07:38:27.885 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-07T07:38:28.126 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:38:29.597 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:29 smithi160 ceph-mon[31963]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:29.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:29 smithi177 ceph-mon[36578]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:29.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:29 smithi178 ceph-mon[36650]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:29.818 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:29.815+0000 7f47e5f6b700 1 -- 172.21.15.160:0/2040844304 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47e0102db0 msgr2=0x7f47e0106df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:29.818 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:29.815+0000 7f47e5f6b700 1 --2- 172.21.15.160:0/2040844304 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47e0102db0 0x7f47e0106df0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f47d40099c0 tx=0x7f47d4009cd0 comp rx=0 tx=0).stop 2024-03-07T07:38:29.818 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:29.815+0000 7f47e5f6b700 1 -- 172.21.15.160:0/2040844304 shutdown_connections 2024-03-07T07:38:29.818 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:29.815+0000 7f47e5f6b700 1 --2- 172.21.15.160:0/2040844304 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47e01074d0 0x7f47e010f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:29.819 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:29.815+0000 7f47e5f6b700 1 --2- 172.21.15.160:0/2040844304 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47e0102400 0x7f47e01027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:29.819 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:29.815+0000 7f47e5f6b700 1 --2- 172.21.15.160:0/2040844304 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47e0102db0 0x7f47e0106df0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:29.819 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:29.815+0000 7f47e5f6b700 1 -- 172.21.15.160:0/2040844304 >> 172.21.15.160:0/2040844304 conn(0x7f47e00792b0 msgr2=0x7f47e00796c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:38:29.819 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:29.815+0000 7f47e5f6b700 1 -- 172.21.15.160:0/2040844304 shutdown_connections 2024-03-07T07:38:29.819 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:29.815+0000 7f47e5f6b700 1 -- 172.21.15.160:0/2040844304 wait complete. 2024-03-07T07:38:29.819 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:29.816+0000 7f47e5f6b700 1 Processor -- start 2024-03-07T07:38:29.819 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:29.816+0000 7f47e5f6b700 1 -- start start 2024-03-07T07:38:29.819 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:29.816+0000 7f47e5f6b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47e0102400 0x7f47e019e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:29.819 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:29.816+0000 7f47e5f6b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47e0102db0 0x7f47e019ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:29.819 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:29.816+0000 7f47e5f6b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47e01074d0 0x7f47e0198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:29.819 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:29.816+0000 7f47e5f6b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e01124c0 con 0x7f47e0102db0 2024-03-07T07:38:29.819 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:29.816+0000 7f47e5f6b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e0112340 con 0x7f47e0102400 2024-03-07T07:38:29.819 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:29.816+0000 7f47e5f6b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e0112640 con 0x7f47e01074d0 2024-03-07T07:38:29.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:29.816+0000 7f47deffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47e0102db0 0x7f47e019ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:29.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:29.816+0000 7f47deffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47e0102db0 0x7f47e019ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44656/0 (socket says 172.21.15.160:44656) 2024-03-07T07:38:29.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:29.816+0000 7f47deffd700 1 -- 172.21.15.160:0/730544757 learned_addr learned my addr 172.21.15.160:0/730544757 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:38:29.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:29.816+0000 7f47df7fe700 1 --2- 172.21.15.160:0/730544757 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47e0102400 0x7f47e019e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:29.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:29.816+0000 7f47dffff700 1 --2- 172.21.15.160:0/730544757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47e01074d0 0x7f47e0198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:29.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:29.816+0000 7f47deffd700 1 -- 172.21.15.160:0/730544757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47e01074d0 msgr2=0x7f47e0198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:29.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:29.816+0000 7f47deffd700 1 --2- 172.21.15.160:0/730544757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47e01074d0 0x7f47e0198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:29.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:29.816+0000 7f47deffd700 1 -- 172.21.15.160:0/730544757 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47e0102400 msgr2=0x7f47e019e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:29.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:29.816+0000 7f47deffd700 1 --2- 172.21.15.160:0/730544757 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47e0102400 0x7f47e019e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:29.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:29.816+0000 7f47deffd700 1 -- 172.21.15.160:0/730544757 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e0199000 con 0x7f47e0102db0 2024-03-07T07:38:29.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:29.817+0000 7f47deffd700 1 --2- 172.21.15.160:0/730544757 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47e0102db0 0x7f47e019ebe0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f47d400fa00 tx=0x7f47d400fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:38:29.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:29.817+0000 7f47dcff9700 1 -- 172.21.15.160:0/730544757 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d4027070 con 0x7f47e0102db0 2024-03-07T07:38:29.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:29.817+0000 7f47e5f6b700 1 -- 172.21.15.160:0/730544757 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47e0199290 con 0x7f47e0102db0 2024-03-07T07:38:29.821 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:29.817+0000 7f47dcff9700 1 -- 172.21.15.160:0/730544757 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f47d4003680 con 0x7f47e0102db0 2024-03-07T07:38:29.821 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:29.817+0000 7f47dcff9700 1 -- 172.21.15.160:0/730544757 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d4017650 con 0x7f47e0102db0 2024-03-07T07:38:29.821 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:29.817+0000 7f47e5f6b700 1 -- 172.21.15.160:0/730544757 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47e01a5500 con 0x7f47e0102db0 2024-03-07T07:38:29.821 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:29.818+0000 7f47e5f6b700 1 -- 172.21.15.160:0/730544757 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47e01038d0 con 0x7f47e0102db0 2024-03-07T07:38:29.821 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:29.819+0000 7f47dcff9700 1 -- 172.21.15.160:0/730544757 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f47d402a070 con 0x7f47e0102db0 2024-03-07T07:38:29.822 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:29.820+0000 7f47dcff9700 1 --2- 172.21.15.160:0/730544757 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f47c80a1e10 0x7f47c80a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:29.822 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:29.820+0000 7f47df7fe700 1 --2- 172.21.15.160:0/730544757 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f47c80a1e10 0x7f47c80a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:29.822 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:29.820+0000 7f47dcff9700 1 -- 172.21.15.160:0/730544757 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f47d40cee40 con 0x7f47e0102db0 2024-03-07T07:38:29.822 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:29.820+0000 7f47df7fe700 1 --2- 172.21.15.160:0/730544757 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f47c80a1e10 0x7f47c80a42d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f47d0005fd0 tx=0x7f47d0005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:38:29.824 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:29.824+0000 7f47dcff9700 1 -- 172.21.15.160:0/730544757 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f47d4031070 con 0x7f47e0102db0 2024-03-07T07:38:30.017 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:30.017+0000 7f47e5f6b700 1 -- 172.21.15.160:0/730544757 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f47e0068e40 con 0x7f47e0102db0 2024-03-07T07:38:30.018 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:30.017+0000 7f47dcff9700 1 -- 172.21.15.160:0/730544757 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f47e0068e40 con 0x7f47e0102db0 2024-03-07T07:38:30.018 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 1 2024-03-07T07:38:30.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:30.019+0000 7f47e5f6b700 1 -- 172.21.15.160:0/730544757 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f47c80a1e10 msgr2=0x7f47c80a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:30.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:30.019+0000 7f47e5f6b700 1 --2- 172.21.15.160:0/730544757 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f47c80a1e10 0x7f47c80a42d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f47d0005fd0 tx=0x7f47d0005f60 comp rx=0 tx=0).stop 2024-03-07T07:38:30.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:30.019+0000 7f47e5f6b700 1 -- 172.21.15.160:0/730544757 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47e0102db0 msgr2=0x7f47e019ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:30.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:30.019+0000 7f47e5f6b700 1 --2- 172.21.15.160:0/730544757 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47e0102db0 0x7f47e019ebe0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f47d400fa00 tx=0x7f47d400fae0 comp rx=0 tx=0).stop 2024-03-07T07:38:30.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:30.020+0000 7f47e5f6b700 1 -- 172.21.15.160:0/730544757 shutdown_connections 2024-03-07T07:38:30.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:30.020+0000 7f47e5f6b700 1 --2- 172.21.15.160:0/730544757 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f47c80a1e10 0x7f47c80a42d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:30.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:30.020+0000 7f47e5f6b700 1 --2- 172.21.15.160:0/730544757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47e01074d0 0x7f47e0198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:30.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:30.020+0000 7f47e5f6b700 1 --2- 172.21.15.160:0/730544757 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47e0102400 0x7f47e019e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:30.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:30.020+0000 7f47e5f6b700 1 --2- 172.21.15.160:0/730544757 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47e0102db0 0x7f47e019ebe0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:30.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:30.020+0000 7f47e5f6b700 1 -- 172.21.15.160:0/730544757 >> 172.21.15.160:0/730544757 conn(0x7f47e00792b0 msgr2=0x7f47e01001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:38:30.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:30.020+0000 7f47e5f6b700 1 -- 172.21.15.160:0/730544757 shutdown_connections 2024-03-07T07:38:30.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:30.020+0000 7f47e5f6b700 1 -- 172.21.15.160:0/730544757 wait complete. 2024-03-07T07:38:30.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:30 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/730544757' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-07T07:38:30.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:30 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/730544757' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-07T07:38:30.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:30 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/730544757' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-07T07:38:30.851 INFO:tasks.cephadm:Adding mds.a on smithi160 2024-03-07T07:38:30.851 INFO:tasks.cephadm:Adding mds.d on smithi160 2024-03-07T07:38:30.851 INFO:tasks.cephadm:Adding mds.g on smithi160 2024-03-07T07:38:30.851 INFO:tasks.cephadm:Adding mds.j on smithi160 2024-03-07T07:38:30.851 INFO:tasks.cephadm:Adding mds.b on smithi177 2024-03-07T07:38:30.851 INFO:tasks.cephadm:Adding mds.e on smithi177 2024-03-07T07:38:30.851 INFO:tasks.cephadm:Adding mds.h on smithi177 2024-03-07T07:38:30.851 INFO:tasks.cephadm:Adding mds.k on smithi177 2024-03-07T07:38:30.852 INFO:tasks.cephadm:Adding mds.c on smithi178 2024-03-07T07:38:30.852 INFO:tasks.cephadm:Adding mds.f on smithi178 2024-03-07T07:38:30.852 INFO:tasks.cephadm:Adding mds.i on smithi178 2024-03-07T07:38:30.852 INFO:tasks.cephadm:Adding mds.l on smithi178 2024-03-07T07:38:30.852 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph orch apply mds all '12;smithi160=a;smithi160=d;smithi160=g;smithi160=j;smithi177=b;smithi177=e;smithi177=h;smithi177=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l' 2024-03-07T07:38:31.105 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.c/config 2024-03-07T07:38:31.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:31 smithi178 ceph-mon[36650]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:31.590 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:31 smithi160 ceph-mon[31963]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:31.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:31 smithi177 ceph-mon[36578]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:32.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:32.857+0000 7f14def44700 1 -- 172.21.15.178:0/1613055198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14d8101070 msgr2=0x7f14d8101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:32.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:32.857+0000 7f14def44700 1 --2- 172.21.15.178:0/1613055198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14d8101070 0x7f14d8101490 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f14d000b0a0 tx=0x7f14d000b3b0 comp rx=0 tx=0).stop 2024-03-07T07:38:32.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:32.857+0000 7f14def44700 1 -- 172.21.15.178:0/1613055198 shutdown_connections 2024-03-07T07:38:32.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:32.857+0000 7f14def44700 1 --2- 172.21.15.178:0/1613055198 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f14d8102c10 0x7f14d81094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:32.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:32.857+0000 7f14def44700 1 --2- 172.21.15.178:0/1613055198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14d8101070 0x7f14d8101490 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:32.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:32.857+0000 7f14def44700 1 --2- 172.21.15.178:0/1613055198 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f14d8102270 0x7f14d81026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:32.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:32.857+0000 7f14def44700 1 -- 172.21.15.178:0/1613055198 >> 172.21.15.178:0/1613055198 conn(0x7f14d80fc610 msgr2=0x7f14d80fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:38:32.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:32.857+0000 7f14def44700 1 -- 172.21.15.178:0/1613055198 shutdown_connections 2024-03-07T07:38:32.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:32.857+0000 7f14def44700 1 -- 172.21.15.178:0/1613055198 wait complete. 2024-03-07T07:38:32.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:32.858+0000 7f14def44700 1 Processor -- start 2024-03-07T07:38:32.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:32.858+0000 7f14def44700 1 -- start start 2024-03-07T07:38:32.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:32.858+0000 7f14def44700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f14d8101070 0x7f14d81983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:32.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:32.858+0000 7f14def44700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f14d8102270 0x7f14d8198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:32.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:32.858+0000 7f14def44700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14d8102c10 0x7f14d819f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:32.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:32.858+0000 7f14def44700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d810cc60 con 0x7f14d8101070 2024-03-07T07:38:32.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:32.858+0000 7f14def44700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d810cae0 con 0x7f14d8102270 2024-03-07T07:38:32.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:32.858+0000 7f14def44700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d810cde0 con 0x7f14d8102c10 2024-03-07T07:38:32.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:32.858+0000 7f14dd4e1700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14d8102c10 0x7f14d819f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:32.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:32.858+0000 7f14dd4e1700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14d8102c10 0x7f14d819f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:56066/0 (socket says 172.21.15.178:56066) 2024-03-07T07:38:32.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:32.858+0000 7f14dd4e1700 1 -- 172.21.15.178:0/3871301456 learned_addr learned my addr 172.21.15.178:0/3871301456 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-03-07T07:38:32.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:32.858+0000 7f14dcce0700 1 --2- 172.21.15.178:0/3871301456 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f14d8101070 0x7f14d81983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:32.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:32.858+0000 7f14d7fff700 1 --2- 172.21.15.178:0/3871301456 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f14d8102270 0x7f14d8198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:32.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:32.859+0000 7f14dd4e1700 1 -- 172.21.15.178:0/3871301456 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f14d8102270 msgr2=0x7f14d8198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:32.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:32.859+0000 7f14dd4e1700 1 --2- 172.21.15.178:0/3871301456 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f14d8102270 0x7f14d8198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:32.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:32.859+0000 7f14dd4e1700 1 -- 172.21.15.178:0/3871301456 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f14d8101070 msgr2=0x7f14d81983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:32.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:32.859+0000 7f14dd4e1700 1 --2- 172.21.15.178:0/3871301456 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f14d8101070 0x7f14d81983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:32.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:32.859+0000 7f14dd4e1700 1 -- 172.21.15.178:0/3871301456 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d81a0050 con 0x7f14d8102c10 2024-03-07T07:38:32.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:32.859+0000 7f14dd4e1700 1 --2- 172.21.15.178:0/3871301456 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14d8102c10 0x7f14d819f9b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f14c000c9a0 tx=0x7f14c000ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:38:32.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:32.859+0000 7f14d5ffb700 1 -- 172.21.15.178:0/3871301456 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c0007ac0 con 0x7f14d8102c10 2024-03-07T07:38:32.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:32.859+0000 7f14def44700 1 -- 172.21.15.178:0/3871301456 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14d81a0340 con 0x7f14d8102c10 2024-03-07T07:38:32.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:32.859+0000 7f14d5ffb700 1 -- 172.21.15.178:0/3871301456 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f14c0004d10 con 0x7f14d8102c10 2024-03-07T07:38:32.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:32.859+0000 7f14d5ffb700 1 -- 172.21.15.178:0/3871301456 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c00055e0 con 0x7f14d8102c10 2024-03-07T07:38:32.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:32.859+0000 7f14def44700 1 -- 172.21.15.178:0/3871301456 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14d81a08d0 con 0x7f14d8102c10 2024-03-07T07:38:32.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:32.860+0000 7f14def44700 1 -- 172.21.15.178:0/3871301456 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14d8068e10 con 0x7f14d8102c10 2024-03-07T07:38:32.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:32.861+0000 7f14d5ffb700 1 -- 172.21.15.178:0/3871301456 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f14c0004750 con 0x7f14d8102c10 2024-03-07T07:38:32.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:32.862+0000 7f14d5ffb700 1 --2- 172.21.15.178:0/3871301456 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f14c40a1e10 0x7f14c40a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:32.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:32.862+0000 7f14d5ffb700 1 -- 172.21.15.178:0/3871301456 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f14c00cd420 con 0x7f14d8102c10 2024-03-07T07:38:32.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:32.862+0000 7f14dcce0700 1 --2- 172.21.15.178:0/3871301456 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f14c40a1e10 0x7f14c40a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:32.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:32.863+0000 7f14dcce0700 1 --2- 172.21.15.178:0/3871301456 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f14c40a1e10 0x7f14c40a42d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f14d0006070 tx=0x7f14d0006000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:38:32.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:32.867+0000 7f14d5ffb700 1 -- 172.21.15.178:0/3871301456 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f14c0014030 con 0x7f14d8102c10 2024-03-07T07:38:33.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:33.079+0000 7f14def44700 1 -- 172.21.15.178:0/3871301456 --> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi160=a;smithi160=d;smithi160=g;smithi160=j;smithi177=b;smithi177=e;smithi177=h;smithi177=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f14d806aea0 con 0x7f14c40a1e10 2024-03-07T07:38:33.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:33.087+0000 7f14d5ffb700 1 -- 172.21.15.178:0/3871301456 <== mgr.14152 v2:172.21.15.160:6800/419630619 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f14d806aea0 con 0x7f14c40a1e10 2024-03-07T07:38:33.088 INFO:teuthology.orchestra.run.smithi178.stdout:Scheduled mds.all update... 2024-03-07T07:38:33.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:33.090+0000 7f14def44700 1 -- 172.21.15.178:0/3871301456 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f14c40a1e10 msgr2=0x7f14c40a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:33.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:33.090+0000 7f14def44700 1 --2- 172.21.15.178:0/3871301456 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f14c40a1e10 0x7f14c40a42d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f14d0006070 tx=0x7f14d0006000 comp rx=0 tx=0).stop 2024-03-07T07:38:33.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:33.090+0000 7f14def44700 1 -- 172.21.15.178:0/3871301456 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14d8102c10 msgr2=0x7f14d819f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:33.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:33.090+0000 7f14def44700 1 --2- 172.21.15.178:0/3871301456 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14d8102c10 0x7f14d819f9b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f14c000c9a0 tx=0x7f14c000ccb0 comp rx=0 tx=0).stop 2024-03-07T07:38:33.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:33.090+0000 7f14def44700 1 -- 172.21.15.178:0/3871301456 shutdown_connections 2024-03-07T07:38:33.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:33.090+0000 7f14def44700 1 --2- 172.21.15.178:0/3871301456 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f14c40a1e10 0x7f14c40a42d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:33.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:33.090+0000 7f14def44700 1 --2- 172.21.15.178:0/3871301456 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14d8102c10 0x7f14d819f9b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:33.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:33.090+0000 7f14def44700 1 --2- 172.21.15.178:0/3871301456 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f14d8101070 0x7f14d81983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:33.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:33.090+0000 7f14def44700 1 --2- 172.21.15.178:0/3871301456 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f14d8102270 0x7f14d8198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:33.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:33.090+0000 7f14def44700 1 -- 172.21.15.178:0/3871301456 >> 172.21.15.178:0/3871301456 conn(0x7f14d80fc610 msgr2=0x7f14d806a690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:38:33.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:33.090+0000 7f14def44700 1 -- 172.21.15.178:0/3871301456 shutdown_connections 2024-03-07T07:38:33.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-03-07T07:38:33.090+0000 7f14def44700 1 -- 172.21.15.178:0/3871301456 wait complete. 2024-03-07T07:38:33.608 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:33 smithi177 ceph-mon[36578]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:33.608 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:33 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:33.608 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:33 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:38:33.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:33 smithi160 ceph-mon[31963]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:33.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:33 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:33.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:33 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:38:33.749 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:33 smithi178 ceph-mon[36650]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:33.750 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:33 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:33.750 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:33 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:38:33.780 DEBUG:teuthology.orchestra.run.smithi160:mds.a> sudo journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@mds.a.service 2024-03-07T07:38:33.784 DEBUG:teuthology.orchestra.run.smithi160:mds.d> sudo journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@mds.d.service 2024-03-07T07:38:33.828 DEBUG:teuthology.orchestra.run.smithi160:mds.g> sudo journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@mds.g.service 2024-03-07T07:38:33.872 DEBUG:teuthology.orchestra.run.smithi160:mds.j> sudo journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@mds.j.service 2024-03-07T07:38:33.915 DEBUG:teuthology.orchestra.run.smithi177:mds.b> sudo journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@mds.b.service 2024-03-07T07:38:33.919 DEBUG:teuthology.orchestra.run.smithi177:mds.e> sudo journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@mds.e.service 2024-03-07T07:38:33.962 DEBUG:teuthology.orchestra.run.smithi177:mds.h> sudo journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@mds.h.service 2024-03-07T07:38:34.005 DEBUG:teuthology.orchestra.run.smithi177:mds.k> sudo journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@mds.k.service 2024-03-07T07:38:34.048 DEBUG:teuthology.orchestra.run.smithi178:mds.c> sudo journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@mds.c.service 2024-03-07T07:38:34.051 DEBUG:teuthology.orchestra.run.smithi178:mds.f> sudo journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@mds.f.service 2024-03-07T07:38:34.094 DEBUG:teuthology.orchestra.run.smithi178:mds.i> sudo journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@mds.i.service 2024-03-07T07:38:34.137 DEBUG:teuthology.orchestra.run.smithi178:mds.l> sudo journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@mds.l.service 2024-03-07T07:38:34.180 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-03-07T07:38:34.180 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T07:38:34.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.295+0000 7fa2f07b8640 1 -- 172.21.15.160:0/281361397 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2e8100580 msgr2=0x7fa2e81009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:34.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.295+0000 7fa2f07b8640 1 --2- 172.21.15.160:0/281361397 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2e8100580 0x7fa2e81009a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4009900 tx=0x7fa2e402bc30 comp rx=0 tx=0).stop 2024-03-07T07:38:34.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.295+0000 7fa2f07b8640 1 -- 172.21.15.160:0/281361397 shutdown_connections 2024-03-07T07:38:34.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.295+0000 7fa2f07b8640 1 --2- 172.21.15.160:0/281361397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2e8102130 0x7fa2e81089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:34.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.295+0000 7fa2f07b8640 1 --2- 172.21.15.160:0/281361397 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2e8101770 0x7fa2e8101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:34.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.295+0000 7fa2f07b8640 1 --2- 172.21.15.160:0/281361397 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2e8100580 0x7fa2e81009a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:34.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.295+0000 7fa2f07b8640 1 -- 172.21.15.160:0/281361397 >> 172.21.15.160:0/281361397 conn(0x7fa2e80fbaf0 msgr2=0x7fa2e80fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:38:34.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.295+0000 7fa2f07b8640 1 -- 172.21.15.160:0/281361397 shutdown_connections 2024-03-07T07:38:34.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.295+0000 7fa2f07b8640 1 -- 172.21.15.160:0/281361397 wait complete. 2024-03-07T07:38:34.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.295+0000 7fa2f07b8640 1 Processor -- start 2024-03-07T07:38:34.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.295+0000 7fa2f07b8640 1 -- start start 2024-03-07T07:38:34.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.296+0000 7fa2f07b8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2e8100580 0x7fa2e8069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:34.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.296+0000 7fa2f07b8640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2e8101770 0x7fa2e819eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:34.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.296+0000 7fa2f07b8640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2e8102130 0x7fa2e81a1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:34.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.296+0000 7fa2f07b8640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e810c210 con 0x7fa2e8102130 2024-03-07T07:38:34.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.296+0000 7fa2f07b8640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e810c090 con 0x7fa2e8101770 2024-03-07T07:38:34.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.296+0000 7fa2f07b8640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e810c390 con 0x7fa2e8100580 2024-03-07T07:38:34.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.296+0000 7fa2eed55640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2e8102130 0x7fa2e81a1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:34.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.296+0000 7fa2ee554640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2e8100580 0x7fa2e8069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:34.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.296+0000 7fa2ee554640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2e8100580 0x7fa2e8069330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.160:40726/0 (socket says 172.21.15.160:40726) 2024-03-07T07:38:34.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.296+0000 7fa2eed55640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2e8102130 0x7fa2e81a1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44672/0 (socket says 172.21.15.160:44672) 2024-03-07T07:38:34.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.296+0000 7fa2ee554640 1 -- 172.21.15.160:0/3172641737 learned_addr learned my addr 172.21.15.160:0/3172641737 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:38:34.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.296+0000 7fa2edd53640 1 --2- 172.21.15.160:0/3172641737 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2e8101770 0x7fa2e819eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:34.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.296+0000 7fa2ee554640 1 -- 172.21.15.160:0/3172641737 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2e8101770 msgr2=0x7fa2e819eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:34.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.296+0000 7fa2ee554640 1 --2- 172.21.15.160:0/3172641737 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2e8101770 0x7fa2e819eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:34.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.296+0000 7fa2ee554640 1 -- 172.21.15.160:0/3172641737 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2e8102130 msgr2=0x7fa2e81a1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:34.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.296+0000 7fa2ee554640 1 --2- 172.21.15.160:0/3172641737 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2e8102130 0x7fa2e81a1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:34.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.296+0000 7fa2ee554640 1 -- 172.21.15.160:0/3172641737 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e81a1820 con 0x7fa2e8100580 2024-03-07T07:38:34.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.296+0000 7fa2eed55640 1 --2- 172.21.15.160:0/3172641737 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2e8102130 0x7fa2e81a1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:38:34.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.296+0000 7fa2ee554640 1 --2- 172.21.15.160:0/3172641737 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2e8100580 0x7fa2e8069330 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4000c00 tx=0x7fa2e402c640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:38:34.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.296+0000 7fa2df7fe640 1 -- 172.21.15.160:0/3172641737 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e4041070 con 0x7fa2e8100580 2024-03-07T07:38:34.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.296+0000 7fa2f07b8640 1 -- 172.21.15.160:0/3172641737 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2e81a1a50 con 0x7fa2e8100580 2024-03-07T07:38:34.301 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.297+0000 7fa2df7fe640 1 -- 172.21.15.160:0/3172641737 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa2e4004450 con 0x7fa2e8100580 2024-03-07T07:38:34.301 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.297+0000 7fa2f07b8640 1 -- 172.21.15.160:0/3172641737 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2e81a1f90 con 0x7fa2e8100580 2024-03-07T07:38:34.301 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.297+0000 7fa2df7fe640 1 -- 172.21.15.160:0/3172641737 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e4034440 con 0x7fa2e8100580 2024-03-07T07:38:34.301 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.297+0000 7fa2f07b8640 1 -- 172.21.15.160:0/3172641737 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2e8107920 con 0x7fa2e8100580 2024-03-07T07:38:34.301 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.299+0000 7fa2df7fe640 1 -- 172.21.15.160:0/3172641737 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7fa2e4044070 con 0x7fa2e8100580 2024-03-07T07:38:34.302 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.299+0000 7fa2df7fe640 1 --2- 172.21.15.160:0/3172641737 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa2d40a1db0 0x7fa2d40a4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:34.302 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.299+0000 7fa2edd53640 1 --2- 172.21.15.160:0/3172641737 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa2d40a1db0 0x7fa2d40a4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:34.302 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.299+0000 7fa2df7fe640 1 -- 172.21.15.160:0/3172641737 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa2e40ef5c0 con 0x7fa2e8100580 2024-03-07T07:38:34.302 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.300+0000 7fa2edd53640 1 --2- 172.21.15.160:0/3172641737 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa2d40a1db0 0x7fa2d40a4270 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa2e80690a0 tx=0x7fa2d8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:38:34.307 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.307+0000 7fa2df7fe640 1 -- 172.21.15.160:0/3172641737 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa2e40bc8d0 con 0x7fa2e8100580 2024-03-07T07:38:34.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.492+0000 7fa2f07b8640 1 -- 172.21.15.160:0/3172641737 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa2e802d1b0 con 0x7fa2e8100580 2024-03-07T07:38:34.493 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.493+0000 7fa2df7fe640 1 -- 172.21.15.160:0/3172641737 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18479 (secure 0 0 0) 0x7fa2e40c0970 con 0x7fa2e8100580 2024-03-07T07:38:34.494 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:38:34.494 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":81,"fsid":"826f6582-dc54-11ee-95c6-87774f69a715","created":"2024-03-07T07:31:14.229288+0000","modified":"2024-03-07T07:38:22.331696+0000","last_up_change":"2024-03-07T07:38:20.776225+0000","last_in_change":"2024-03-07T07:38:00.508917+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T07:34:34.529424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bee66c67-fb6e-4717-9685-ca62587d5f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6803","nonce":1368482926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6805","nonce":1368482926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6809","nonce":1368482926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6807","nonce":1368482926}]},"public_addr":"172.21.15.160:6803/1368482926","cluster_addr":"172.21.15.160:6805/1368482926","heartbeat_back_addr":"172.21.15.160:6809/1368482926","heartbeat_front_addr":"172.21.15.160:6807/1368482926","state":["exists","up"]},{"osd":1,"uuid":"49236251-f750-424e-992b-bddf55762556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6801","nonce":2922391884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6803","nonce":2922391884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6807","nonce":2922391884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6805","nonce":2922391884}]},"public_addr":"172.21.15.177:6801/2922391884","cluster_addr":"172.21.15.177:6803/2922391884","heartbeat_back_addr":"172.21.15.177:6807/2922391884","heartbeat_front_addr":"172.21.15.177:6805/2922391884","state":["exists","up"]},{"osd":2,"uuid":"7a432b28-f7ee-43fe-9196-7aaeccfecf77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6801","nonce":1217901551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6803","nonce":1217901551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6807","nonce":1217901551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6805","nonce":1217901551}]},"public_addr":"172.21.15.178:6801/1217901551","cluster_addr":"172.21.15.178:6803/1217901551","heartbeat_back_addr":"172.21.15.178:6807/1217901551","heartbeat_front_addr":"172.21.15.178:6805/1217901551","state":["exists","up"]},{"osd":3,"uuid":"4ad22a20-f1be-4002-a48d-ecd3953e63f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6811","nonce":42543989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6813","nonce":42543989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6817","nonce":42543989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6815","nonce":42543989}]},"public_addr":"172.21.15.160:6811/42543989","cluster_addr":"172.21.15.160:6813/42543989","heartbeat_back_addr":"172.21.15.160:6817/42543989","heartbeat_front_addr":"172.21.15.160:6815/42543989","state":["exists","up"]},{"osd":4,"uuid":"2b65b104-ac82-4ce3-b6c5-11a399103045","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6809","nonce":88178553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6811","nonce":88178553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6815","nonce":88178553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6813","nonce":88178553}]},"public_addr":"172.21.15.177:6809/88178553","cluster_addr":"172.21.15.177:6811/88178553","heartbeat_back_addr":"172.21.15.177:6815/88178553","heartbeat_front_addr":"172.21.15.177:6813/88178553","state":["exists","up"]},{"osd":5,"uuid":"30a1e714-08ff-44e8-9264-9ef164604b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6809","nonce":964732891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6811","nonce":964732891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6815","nonce":964732891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6813","nonce":964732891}]},"public_addr":"172.21.15.178:6809/964732891","cluster_addr":"172.21.15.178:6811/964732891","heartbeat_back_addr":"172.21.15.178:6815/964732891","heartbeat_front_addr":"172.21.15.178:6813/964732891","state":["exists","up"]},{"osd":6,"uuid":"b1e1646e-f8a9-42b8-b62d-9d7b5d6a1ff4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6819","nonce":3251327001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6821","nonce":3251327001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6825","nonce":3251327001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6823","nonce":3251327001}]},"public_addr":"172.21.15.160:6819/3251327001","cluster_addr":"172.21.15.160:6821/3251327001","heartbeat_back_addr":"172.21.15.160:6825/3251327001","heartbeat_front_addr":"172.21.15.160:6823/3251327001","state":["exists","up"]},{"osd":7,"uuid":"e66b400b-9747-4f6d-b05b-db22468e2750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6817","nonce":1175500850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6819","nonce":1175500850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6823","nonce":1175500850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6821","nonce":1175500850}]},"public_addr":"172.21.15.177:6817/1175500850","cluster_addr":"172.21.15.177:6819/1175500850","heartbeat_back_addr":"172.21.15.177:6823/1175500850","heartbeat_front_addr":"172.21.15.177:6821/1175500850","state":["exists","up"]},{"osd":8,"uuid":"3ee0f2c4-fca4-49fc-a98b-07aa0fcba0e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6817","nonce":688015531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6819","nonce":688015531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6823","nonce":688015531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6821","nonce":688015531}]},"public_addr":"172.21.15.178:6817/688015531","cluster_addr":"172.21.15.178:6819/688015531","heartbeat_back_addr":"172.21.15.178:6823/688015531","heartbeat_front_addr":"172.21.15.178:6821/688015531","state":["exists","up"]},{"osd":9,"uuid":"856039d7-f7ee-4dbd-b019-0073e705ce0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6827","nonce":3265797851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6829","nonce":3265797851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6833","nonce":3265797851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6831","nonce":3265797851}]},"public_addr":"172.21.15.160:6827/3265797851","cluster_addr":"172.21.15.160:6829/3265797851","heartbeat_back_addr":"172.21.15.160:6833/3265797851","heartbeat_front_addr":"172.21.15.160:6831/3265797851","state":["exists","up"]},{"osd":10,"uuid":"bdc9899d-d720-4429-ad53-4c388a4dbe75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6825","nonce":2867473412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6827","nonce":2867473412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6831","nonce":2867473412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6829","nonce":2867473412}]},"public_addr":"172.21.15.177:6825/2867473412","cluster_addr":"172.21.15.177:6827/2867473412","heartbeat_back_addr":"172.21.15.177:6831/2867473412","heartbeat_front_addr":"172.21.15.177:6829/2867473412","state":["exists","up"]},{"osd":11,"uuid":"cc9adb54-ef49-47c6-90e3-fc03bc0f5cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6825","nonce":4068724731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6827","nonce":4068724731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6831","nonce":4068724731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6829","nonce":4068724731}]},"public_addr":"172.21.15.178:6825/4068724731","cluster_addr":"172.21.15.178:6827/4068724731","heartbeat_back_addr":"172.21.15.178:6831/4068724731","heartbeat_front_addr":"172.21.15.178:6829/4068724731","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:33:44.141585+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:07.685585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:31.044386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:56.308925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:35:21.582097+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:35:45.983540+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:36:11.530242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:36:36.844681+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:02.303437+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:27.629867+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:53.289495+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:38:18.536169+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/3027450612":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/874709722":"2024-03-08T07:32:08.190837+0000","172.21.15.160:6800/1130675225":"2024-03-08T07:31:41.166858+0000","172.21.15.160:0/3857193980":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/3861968297":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6800/3859174315":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/3013534743":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6801/3859174315":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/1310937667":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6801/1130675225":"2024-03-08T07:31:41.166858+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T07:38:34.496 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.495+0000 7fa2f07b8640 1 -- 172.21.15.160:0/3172641737 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa2d40a1db0 msgr2=0x7fa2d40a4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:34.496 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.495+0000 7fa2f07b8640 1 --2- 172.21.15.160:0/3172641737 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa2d40a1db0 0x7fa2d40a4270 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa2e80690a0 tx=0x7fa2d8005f90 comp rx=0 tx=0).stop 2024-03-07T07:38:34.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.495+0000 7fa2f07b8640 1 -- 172.21.15.160:0/3172641737 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2e8100580 msgr2=0x7fa2e8069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:34.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.495+0000 7fa2f07b8640 1 --2- 172.21.15.160:0/3172641737 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2e8100580 0x7fa2e8069330 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4000c00 tx=0x7fa2e402c640 comp rx=0 tx=0).stop 2024-03-07T07:38:34.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.495+0000 7fa2f07b8640 1 -- 172.21.15.160:0/3172641737 shutdown_connections 2024-03-07T07:38:34.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.495+0000 7fa2f07b8640 1 --2- 172.21.15.160:0/3172641737 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa2d40a1db0 0x7fa2d40a4270 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:34.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.495+0000 7fa2f07b8640 1 --2- 172.21.15.160:0/3172641737 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2e8102130 0x7fa2e81a1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:34.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.495+0000 7fa2f07b8640 1 --2- 172.21.15.160:0/3172641737 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2e8101770 0x7fa2e819eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:34.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.495+0000 7fa2f07b8640 1 --2- 172.21.15.160:0/3172641737 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2e8100580 0x7fa2e8069330 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:34.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.495+0000 7fa2f07b8640 1 -- 172.21.15.160:0/3172641737 >> 172.21.15.160:0/3172641737 conn(0x7fa2e80fbaf0 msgr2=0x7fa2e8104af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:38:34.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.495+0000 7fa2f07b8640 1 -- 172.21.15.160:0/3172641737 shutdown_connections 2024-03-07T07:38:34.498 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.496+0000 7fa2f07b8640 1 -- 172.21.15.160:0/3172641737 wait complete. 2024-03-07T07:38:34.506 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:34 smithi160 ceph-mon[31963]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi160=a;smithi160=d;smithi160=g;smithi160=j;smithi177=b;smithi177=e;smithi177=h;smithi177=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:38:34.506 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:34 smithi160 ceph-mon[31963]: Saving service mds.all spec with placement smithi160=a;smithi160=d;smithi160=g;smithi160=j;smithi177=b;smithi177=e;smithi177=h;smithi177=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l;count:12 2024-03-07T07:38:34.506 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:34 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:38:34.506 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:34 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:38:34.506 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:34 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:34.507 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:34 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-07T07:38:34.507 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:34 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-07T07:38:34.507 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:34 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:38:34.508 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-07T07:34:34.529424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-07T07:38:34.508 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-07T07:38:34.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:34 smithi177 ceph-mon[36578]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi160=a;smithi160=d;smithi160=g;smithi160=j;smithi177=b;smithi177=e;smithi177=h;smithi177=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:38:34.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:34 smithi177 ceph-mon[36578]: Saving service mds.all spec with placement smithi160=a;smithi160=d;smithi160=g;smithi160=j;smithi177=b;smithi177=e;smithi177=h;smithi177=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l;count:12 2024-03-07T07:38:34.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:34 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:38:34.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:34 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:38:34.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:34 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:34.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:34 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-07T07:38:34.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:34 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-07T07:38:34.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:34 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:38:34.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.615+0000 7fdd366c7640 1 -- 172.21.15.160:0/1664519663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd301073a0 msgr2=0x7fdd30109790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:34.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.615+0000 7fdd366c7640 1 --2- 172.21.15.160:0/1664519663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd301073a0 0x7fdd30109790 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdd28009ca0 tx=0x7fdd2802c0f0 comp rx=0 tx=0).stop 2024-03-07T07:38:34.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.615+0000 7fdd366c7640 1 -- 172.21.15.160:0/1664519663 shutdown_connections 2024-03-07T07:38:34.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.615+0000 7fdd366c7640 1 --2- 172.21.15.160:0/1664519663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd301073a0 0x7fdd30109790 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:34.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.615+0000 7fdd366c7640 1 --2- 172.21.15.160:0/1664519663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd30104a70 0x7fdd30106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:34.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.615+0000 7fdd366c7640 1 --2- 172.21.15.160:0/1664519663 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdd30102140 0x7fdd30104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:34.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.615+0000 7fdd366c7640 1 -- 172.21.15.160:0/1664519663 >> 172.21.15.160:0/1664519663 conn(0x7fdd300fbaf0 msgr2=0x7fdd300fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:38:34.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.615+0000 7fdd366c7640 1 -- 172.21.15.160:0/1664519663 shutdown_connections 2024-03-07T07:38:34.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.615+0000 7fdd366c7640 1 -- 172.21.15.160:0/1664519663 wait complete. 2024-03-07T07:38:34.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.616+0000 7fdd366c7640 1 Processor -- start 2024-03-07T07:38:34.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.616+0000 7fdd366c7640 1 -- start start 2024-03-07T07:38:34.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.616+0000 7fdd366c7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdd30102140 0x7fdd30197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:34.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.616+0000 7fdd366c7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd30104a70 0x7fdd30197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:34.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.616+0000 7fdd366c7640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd301073a0 0x7fdd3019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:34.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.616+0000 7fdd366c7640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3010d2f0 con 0x7fdd301073a0 2024-03-07T07:38:34.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.616+0000 7fdd366c7640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3010d170 con 0x7fdd30102140 2024-03-07T07:38:34.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.616+0000 7fdd366c7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3010d470 con 0x7fdd30104a70 2024-03-07T07:38:34.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.616+0000 7fdd34c64640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd301073a0 0x7fdd3019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:34.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.616+0000 7fdd34c64640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd301073a0 0x7fdd3019ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44694/0 (socket says 172.21.15.160:44694) 2024-03-07T07:38:34.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.616+0000 7fdd34c64640 1 -- 172.21.15.160:0/1465890848 learned_addr learned my addr 172.21.15.160:0/1465890848 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:38:34.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.616+0000 7fdd2f7fe640 1 --2- 172.21.15.160:0/1465890848 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd30104a70 0x7fdd30197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:34.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.616+0000 7fdd2ffff640 1 --2- 172.21.15.160:0/1465890848 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdd30102140 0x7fdd30197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:34.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.616+0000 7fdd34c64640 1 -- 172.21.15.160:0/1465890848 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd30104a70 msgr2=0x7fdd30197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:34.622 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.616+0000 7fdd34c64640 1 --2- 172.21.15.160:0/1465890848 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd30104a70 0x7fdd30197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:34.622 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.616+0000 7fdd34c64640 1 -- 172.21.15.160:0/1465890848 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdd30102140 msgr2=0x7fdd30197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:34.622 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.616+0000 7fdd34c64640 1 --2- 172.21.15.160:0/1465890848 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdd30102140 0x7fdd30197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:34.622 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.616+0000 7fdd34c64640 1 -- 172.21.15.160:0/1465890848 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd3019f590 con 0x7fdd301073a0 2024-03-07T07:38:34.622 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.616+0000 7fdd2ffff640 1 --2- 172.21.15.160:0/1465890848 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdd30102140 0x7fdd30197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T07:38:34.622 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.617+0000 7fdd34c64640 1 --2- 172.21.15.160:0/1465890848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd301073a0 0x7fdd3019ef20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdd28037040 tx=0x7fdd28032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:38:34.622 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.617+0000 7fdd2d7fa640 1 -- 172.21.15.160:0/1465890848 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd28041020 con 0x7fdd301073a0 2024-03-07T07:38:34.622 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.617+0000 7fdd2d7fa640 1 -- 172.21.15.160:0/1465890848 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdd28034cd0 con 0x7fdd301073a0 2024-03-07T07:38:34.622 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.617+0000 7fdd2d7fa640 1 -- 172.21.15.160:0/1465890848 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd28039040 con 0x7fdd301073a0 2024-03-07T07:38:34.622 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.617+0000 7fdd366c7640 1 -- 172.21.15.160:0/1465890848 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd3019f820 con 0x7fdd301073a0 2024-03-07T07:38:34.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.617+0000 7fdd366c7640 1 -- 172.21.15.160:0/1465890848 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd3019fd60 con 0x7fdd301073a0 2024-03-07T07:38:34.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.618+0000 7fdd366c7640 1 -- 172.21.15.160:0/1465890848 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd300fd5e0 con 0x7fdd301073a0 2024-03-07T07:38:34.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.620+0000 7fdd2d7fa640 1 -- 172.21.15.160:0/1465890848 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7fdd28044020 con 0x7fdd301073a0 2024-03-07T07:38:34.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.620+0000 7fdd2d7fa640 1 --2- 172.21.15.160:0/1465890848 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fdd180a1e00 0x7fdd180a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:34.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.621+0000 7fdd2ffff640 1 --2- 172.21.15.160:0/1465890848 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fdd180a1e00 0x7fdd180a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:34.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.621+0000 7fdd2d7fa640 1 -- 172.21.15.160:0/1465890848 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdd280efc30 con 0x7fdd301073a0 2024-03-07T07:38:34.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.621+0000 7fdd2ffff640 1 --2- 172.21.15.160:0/1465890848 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fdd180a1e00 0x7fdd180a42c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdd24009b50 tx=0x7fdd24005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:38:34.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.624+0000 7fdd2d7fa640 1 -- 172.21.15.160:0/1465890848 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fdd2804b020 con 0x7fdd301073a0 2024-03-07T07:38:34.686 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:34 smithi178 ceph-mon[36650]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi160=a;smithi160=d;smithi160=g;smithi160=j;smithi177=b;smithi177=e;smithi177=h;smithi177=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:38:34.687 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:34 smithi178 ceph-mon[36650]: Saving service mds.all spec with placement smithi160=a;smithi160=d;smithi160=g;smithi160=j;smithi177=b;smithi177=e;smithi177=h;smithi177=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l;count:12 2024-03-07T07:38:34.687 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:34 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:38:34.687 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:34 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:38:34.687 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:34 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:34.687 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:34 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-07T07:38:34.687 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:34 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-07T07:38:34.687 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:34 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:38:34.805 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.804+0000 7fdd366c7640 1 -- 172.21.15.160:0/1465890848 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdd3002d030 con 0x7fdd301073a0 2024-03-07T07:38:34.805 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.805+0000 7fdd2d7fa640 1 -- 172.21.15.160:0/1465890848 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdd280bd4f0 con 0x7fdd301073a0 2024-03-07T07:38:34.806 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 1 2024-03-07T07:38:34.807 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.807+0000 7fdd366c7640 1 -- 172.21.15.160:0/1465890848 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fdd180a1e00 msgr2=0x7fdd180a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:34.807 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.807+0000 7fdd366c7640 1 --2- 172.21.15.160:0/1465890848 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fdd180a1e00 0x7fdd180a42c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdd24009b50 tx=0x7fdd24005f90 comp rx=0 tx=0).stop 2024-03-07T07:38:34.808 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.807+0000 7fdd366c7640 1 -- 172.21.15.160:0/1465890848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd301073a0 msgr2=0x7fdd3019ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:34.808 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.807+0000 7fdd366c7640 1 --2- 172.21.15.160:0/1465890848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd301073a0 0x7fdd3019ef20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdd28037040 tx=0x7fdd28032f40 comp rx=0 tx=0).stop 2024-03-07T07:38:34.808 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.807+0000 7fdd366c7640 1 -- 172.21.15.160:0/1465890848 shutdown_connections 2024-03-07T07:38:34.808 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.807+0000 7fdd366c7640 1 --2- 172.21.15.160:0/1465890848 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fdd180a1e00 0x7fdd180a42c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:34.808 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.807+0000 7fdd366c7640 1 --2- 172.21.15.160:0/1465890848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd301073a0 0x7fdd3019ef20 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:34.808 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.807+0000 7fdd366c7640 1 --2- 172.21.15.160:0/1465890848 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd30104a70 0x7fdd30197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:34.808 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.807+0000 7fdd366c7640 1 --2- 172.21.15.160:0/1465890848 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdd30102140 0x7fdd30197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:34.808 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.807+0000 7fdd366c7640 1 -- 172.21.15.160:0/1465890848 >> 172.21.15.160:0/1465890848 conn(0x7fdd300fbaf0 msgr2=0x7fdd30102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:38:34.808 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.807+0000 7fdd366c7640 1 -- 172.21.15.160:0/1465890848 shutdown_connections 2024-03-07T07:38:34.808 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.807+0000 7fdd366c7640 1 -- 172.21.15.160:0/1465890848 wait complete. 2024-03-07T07:38:34.820 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T07:38:34.927 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.924+0000 7fd581399640 1 -- 172.21.15.160:0/2821421280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd57c1021b0 msgr2=0x7fd57c108a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:34.927 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.924+0000 7fd581399640 1 --2- 172.21.15.160:0/2821421280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd57c1021b0 0x7fd57c108a40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd574009ca0 tx=0x7fd57402c0f0 comp rx=0 tx=0).stop 2024-03-07T07:38:34.927 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.924+0000 7fd581399640 1 -- 172.21.15.160:0/2821421280 shutdown_connections 2024-03-07T07:38:34.927 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.924+0000 7fd581399640 1 --2- 172.21.15.160:0/2821421280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd57c1021b0 0x7fd57c108a40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:34.928 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.924+0000 7fd581399640 1 --2- 172.21.15.160:0/2821421280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd57c1017f0 0x7fd57c101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:34.928 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.924+0000 7fd581399640 1 --2- 172.21.15.160:0/2821421280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd57c100600 0x7fd57c100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:34.928 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.924+0000 7fd581399640 1 -- 172.21.15.160:0/2821421280 >> 172.21.15.160:0/2821421280 conn(0x7fd57c0fbb90 msgr2=0x7fd57c0fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:38:34.928 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.924+0000 7fd581399640 1 -- 172.21.15.160:0/2821421280 shutdown_connections 2024-03-07T07:38:34.928 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.924+0000 7fd581399640 1 -- 172.21.15.160:0/2821421280 wait complete. 2024-03-07T07:38:34.929 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.925+0000 7fd581399640 1 Processor -- start 2024-03-07T07:38:34.929 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.925+0000 7fd581399640 1 -- start start 2024-03-07T07:38:34.929 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.925+0000 7fd581399640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd57c100600 0x7fd57c06a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:34.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.925+0000 7fd581399640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd57c1017f0 0x7fd57c19c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:34.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.925+0000 7fd581399640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd57c1021b0 0x7fd57c19ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:34.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.925+0000 7fd581399640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c10c220 con 0x7fd57c100600 2024-03-07T07:38:34.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.925+0000 7fd581399640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c10c0a0 con 0x7fd57c1017f0 2024-03-07T07:38:34.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.925+0000 7fd581399640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c10c3a0 con 0x7fd57c1021b0 2024-03-07T07:38:34.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.925+0000 7fd57affd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd57c100600 0x7fd57c06a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:34.931 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.925+0000 7fd57affd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd57c100600 0x7fd57c06a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44710/0 (socket says 172.21.15.160:44710) 2024-03-07T07:38:34.931 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.925+0000 7fd57affd640 1 -- 172.21.15.160:0/31603454 learned_addr learned my addr 172.21.15.160:0/31603454 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:38:34.931 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.925+0000 7fd57b7fe640 1 --2- 172.21.15.160:0/31603454 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd57c1021b0 0x7fd57c19ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:34.931 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.925+0000 7fd57a7fc640 1 --2- 172.21.15.160:0/31603454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd57c1017f0 0x7fd57c19c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:34.931 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.925+0000 7fd57affd640 1 -- 172.21.15.160:0/31603454 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd57c1021b0 msgr2=0x7fd57c19ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:34.932 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.925+0000 7fd57affd640 1 --2- 172.21.15.160:0/31603454 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd57c1021b0 0x7fd57c19ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:34.932 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.925+0000 7fd57affd640 1 -- 172.21.15.160:0/31603454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd57c1017f0 msgr2=0x7fd57c19c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:34.932 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.925+0000 7fd57affd640 1 --2- 172.21.15.160:0/31603454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd57c1017f0 0x7fd57c19c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:34.932 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.925+0000 7fd57affd640 1 -- 172.21.15.160:0/31603454 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd57c19f520 con 0x7fd57c100600 2024-03-07T07:38:34.932 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.926+0000 7fd57affd640 1 --2- 172.21.15.160:0/31603454 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd57c100600 0x7fd57c06a800 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd56c00d770 tx=0x7fd56c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:38:34.932 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.926+0000 7fd563fff640 1 -- 172.21.15.160:0/31603454 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56c017070 con 0x7fd57c100600 2024-03-07T07:38:34.932 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.926+0000 7fd581399640 1 -- 172.21.15.160:0/31603454 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd57c19f810 con 0x7fd57c100600 2024-03-07T07:38:34.932 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.926+0000 7fd563fff640 1 -- 172.21.15.160:0/31603454 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd56c009c30 con 0x7fd57c100600 2024-03-07T07:38:34.932 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.926+0000 7fd563fff640 1 -- 172.21.15.160:0/31603454 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56c003dd0 con 0x7fd57c100600 2024-03-07T07:38:34.932 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.926+0000 7fd581399640 1 -- 172.21.15.160:0/31603454 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd57c19fd60 con 0x7fd57c100600 2024-03-07T07:38:34.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.927+0000 7fd581399640 1 -- 172.21.15.160:0/31603454 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd57c0688a0 con 0x7fd57c100600 2024-03-07T07:38:34.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.928+0000 7fd563fff640 1 -- 172.21.15.160:0/31603454 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7fd56c003f70 con 0x7fd57c100600 2024-03-07T07:38:34.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.928+0000 7fd563fff640 1 --2- 172.21.15.160:0/31603454 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fd5640a1d80 0x7fd5640a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:34.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.929+0000 7fd563fff640 1 -- 172.21.15.160:0/31603454 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd56c0cd320 con 0x7fd57c100600 2024-03-07T07:38:34.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.929+0000 7fd57a7fc640 1 --2- 172.21.15.160:0/31603454 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fd5640a1d80 0x7fd5640a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:34.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.929+0000 7fd57a7fc640 1 --2- 172.21.15.160:0/31603454 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fd5640a1d80 0x7fd5640a4240 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd57c06a570 tx=0x7fd570005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:38:34.935 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:34.933+0000 7fd563fff640 1 -- 172.21.15.160:0/31603454 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd56c09abf0 con 0x7fd57c100600 2024-03-07T07:38:35.144 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:35.144+0000 7fd581399640 1 -- 172.21.15.160:0/31603454 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd57c02d100 con 0x7fd57c100600 2024-03-07T07:38:35.144 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:35.144+0000 7fd563fff640 1 -- 172.21.15.160:0/31603454 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7fd57c02d100 con 0x7fd57c100600 2024-03-07T07:38:35.145 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:38:35.145 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-03-07T07:38:35.145 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 1 2024-03-07T07:38:35.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:35.146+0000 7fd581399640 1 -- 172.21.15.160:0/31603454 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fd5640a1d80 msgr2=0x7fd5640a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:35.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:35.146+0000 7fd581399640 1 --2- 172.21.15.160:0/31603454 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fd5640a1d80 0x7fd5640a4240 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd57c06a570 tx=0x7fd570005f90 comp rx=0 tx=0).stop 2024-03-07T07:38:35.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:35.146+0000 7fd581399640 1 -- 172.21.15.160:0/31603454 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd57c100600 msgr2=0x7fd57c06a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:35.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:35.146+0000 7fd581399640 1 --2- 172.21.15.160:0/31603454 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd57c100600 0x7fd57c06a800 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd56c00d770 tx=0x7fd56c00dc30 comp rx=0 tx=0).stop 2024-03-07T07:38:35.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:35.146+0000 7fd581399640 1 -- 172.21.15.160:0/31603454 shutdown_connections 2024-03-07T07:38:35.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:35.146+0000 7fd581399640 1 --2- 172.21.15.160:0/31603454 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fd5640a1d80 0x7fd5640a4240 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:35.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:35.146+0000 7fd581399640 1 --2- 172.21.15.160:0/31603454 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd57c1021b0 0x7fd57c19ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:35.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:35.146+0000 7fd581399640 1 --2- 172.21.15.160:0/31603454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd57c1017f0 0x7fd57c19c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:35.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:35.146+0000 7fd581399640 1 --2- 172.21.15.160:0/31603454 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd57c100600 0x7fd57c06a800 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:35.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:35.146+0000 7fd581399640 1 -- 172.21.15.160:0/31603454 >> 172.21.15.160:0/31603454 conn(0x7fd57c0fbb90 msgr2=0x7fd57c104a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:38:35.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:35.146+0000 7fd581399640 1 -- 172.21.15.160:0/31603454 shutdown_connections 2024-03-07T07:38:35.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:35.146+0000 7fd581399640 1 -- 172.21.15.160:0/31603454 wait complete. 2024-03-07T07:38:35.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:35 smithi160 ceph-mon[31963]: Deploying daemon mds.l on smithi178 2024-03-07T07:38:35.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:35 smithi160 ceph-mon[31963]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:35.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:35 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/3172641737' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:38:35.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:35 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/1465890848' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-07T07:38:35.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:35 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/31603454' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:38:35.541 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:35 smithi178 ceph-mon[36650]: Deploying daemon mds.l on smithi178 2024-03-07T07:38:35.541 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:35 smithi178 ceph-mon[36650]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:35.541 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:35 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/3172641737' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:38:35.542 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:35 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/1465890848' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-07T07:38:35.542 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:35 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/31603454' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:38:35.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:35 smithi177 ceph-mon[36578]: Deploying daemon mds.l on smithi178 2024-03-07T07:38:35.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:35 smithi177 ceph-mon[36578]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:35.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:35 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/3172641737' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:38:35.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:35 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/1465890848' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-07T07:38:35.608 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:35 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/31603454' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:38:36.223 INFO:journalctl@ceph.mds.l.smithi178.stdout:Mar 07 07:38:36 smithi178 systemd[1]: Starting Ceph mds.l for 826f6582-dc54-11ee-95c6-87774f69a715... 2024-03-07T07:38:36.504 INFO:journalctl@ceph.mds.l.smithi178.stdout:Mar 07 07:38:36 smithi178 podman[67671]: 2024-03-07 07:38:36.125979164 +0000 UTC m=+0.018230792 image pull e362cd7cebfee51077fb90b7f01895811140c9a4272edd83ad9910b05e3a46f6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2 2024-03-07T07:38:36.504 INFO:journalctl@ceph.mds.l.smithi178.stdout:Mar 07 07:38:36 smithi178 podman[67671]: 2024-03-07 07:38:36.234947352 +0000 UTC m=+0.127199018 container create 0ca9fa5a8119f99bb0a18a465590b46c2948341877837da0d3e94aa096f5f4f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-l, ceph=True, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-8b896a9, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-07T07:38:36.806 INFO:journalctl@ceph.mds.l.smithi178.stdout:Mar 07 07:38:36 smithi178 podman[67671]: 2024-03-07 07:38:36.504422173 +0000 UTC m=+0.396673813 container init 0ca9fa5a8119f99bb0a18a465590b46c2948341877837da0d3e94aa096f5f4f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-l, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240222, RELEASE=wip-lusov-qdb-8b896a9, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-07T07:38:36.806 INFO:journalctl@ceph.mds.l.smithi178.stdout:Mar 07 07:38:36 smithi178 podman[67671]: 2024-03-07 07:38:36.507026827 +0000 UTC m=+0.399278442 container start 0ca9fa5a8119f99bb0a18a465590b46c2948341877837da0d3e94aa096f5f4f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-l, RELEASE=wip-lusov-qdb-8b896a9, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-07T07:38:36.806 INFO:journalctl@ceph.mds.l.smithi178.stdout:Mar 07 07:38:36 smithi178 ceph-mds[67689]: -- 172.21.15.178:0/2698107541 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a072e79a40 con 0x55a072e4d000 2024-03-07T07:38:36.806 INFO:journalctl@ceph.mds.l.smithi178.stdout:Mar 07 07:38:36 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-l[67685]: starting mds.l at 2024-03-07T07:38:36.806 INFO:journalctl@ceph.mds.l.smithi178.stdout:Mar 07 07:38:36 smithi178 bash[67671]: 0ca9fa5a8119f99bb0a18a465590b46c2948341877837da0d3e94aa096f5f4f3 2024-03-07T07:38:36.807 INFO:journalctl@ceph.mds.l.smithi178.stdout:Mar 07 07:38:36 smithi178 systemd[1]: Started Ceph mds.l for 826f6582-dc54-11ee-95c6-87774f69a715. 2024-03-07T07:38:37.158 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T07:38:37.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.273+0000 7fcefdba7640 1 -- 172.21.15.160:0/1354555508 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcef8102130 msgr2=0x7fcef81089c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:37.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.273+0000 7fcefdba7640 1 --2- 172.21.15.160:0/1354555508 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcef8102130 0x7fcef81089c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcef0009ca0 tx=0x7fcef002c0f0 comp rx=0 tx=0).stop 2024-03-07T07:38:37.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.273+0000 7fcefdba7640 1 -- 172.21.15.160:0/1354555508 shutdown_connections 2024-03-07T07:38:37.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.273+0000 7fcefdba7640 1 --2- 172.21.15.160:0/1354555508 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcef8102130 0x7fcef81089c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:37.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.273+0000 7fcefdba7640 1 --2- 172.21.15.160:0/1354555508 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcef8101770 0x7fcef8101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:37.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.273+0000 7fcefdba7640 1 --2- 172.21.15.160:0/1354555508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcef8100580 0x7fcef81009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:37.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.273+0000 7fcefdba7640 1 -- 172.21.15.160:0/1354555508 >> 172.21.15.160:0/1354555508 conn(0x7fcef80fbaf0 msgr2=0x7fcef80fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:38:37.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.273+0000 7fcefdba7640 1 -- 172.21.15.160:0/1354555508 shutdown_connections 2024-03-07T07:38:37.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.273+0000 7fcefdba7640 1 -- 172.21.15.160:0/1354555508 wait complete. 2024-03-07T07:38:37.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.274+0000 7fcefdba7640 1 Processor -- start 2024-03-07T07:38:37.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.274+0000 7fcefdba7640 1 -- start start 2024-03-07T07:38:37.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.274+0000 7fcefdba7640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcef8100580 0x7fcef810af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:37.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.274+0000 7fcefdba7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcef8101770 0x7fcef8109540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:37.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.274+0000 7fcefdba7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcef8102130 0x7fcef8109a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:37.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.274+0000 7fcefdba7640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef810d2e0 con 0x7fcef8100580 2024-03-07T07:38:37.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.274+0000 7fcefdba7640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef810d160 con 0x7fcef8102130 2024-03-07T07:38:37.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.274+0000 7fcefdba7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef810d460 con 0x7fcef8101770 2024-03-07T07:38:37.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.274+0000 7fcef77fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcef8100580 0x7fcef810af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:37.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.274+0000 7fcef77fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcef8100580 0x7fcef810af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44738/0 (socket says 172.21.15.160:44738) 2024-03-07T07:38:37.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.274+0000 7fcef77fe640 1 -- 172.21.15.160:0/3692205310 learned_addr learned my addr 172.21.15.160:0/3692205310 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:38:37.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.274+0000 7fcef7fff640 1 --2- 172.21.15.160:0/3692205310 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcef8102130 0x7fcef8109a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:37.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.274+0000 7fcef6ffd640 1 --2- 172.21.15.160:0/3692205310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcef8101770 0x7fcef8109540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:37.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.274+0000 7fcef77fe640 1 -- 172.21.15.160:0/3692205310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcef8101770 msgr2=0x7fcef8109540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:37.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.274+0000 7fcef77fe640 1 --2- 172.21.15.160:0/3692205310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcef8101770 0x7fcef8109540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:37.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.274+0000 7fcef77fe640 1 -- 172.21.15.160:0/3692205310 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcef8102130 msgr2=0x7fcef8109a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:37.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.274+0000 7fcef77fe640 1 --2- 172.21.15.160:0/3692205310 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcef8102130 0x7fcef8109a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:37.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.274+0000 7fcef77fe640 1 -- 172.21.15.160:0/3692205310 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef810a310 con 0x7fcef8100580 2024-03-07T07:38:37.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.274+0000 7fcef7fff640 1 --2- 172.21.15.160:0/3692205310 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcef8102130 0x7fcef8109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T07:38:37.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.275+0000 7fcef6ffd640 1 --2- 172.21.15.160:0/3692205310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcef8101770 0x7fcef8109540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T07:38:37.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.275+0000 7fcef77fe640 1 --2- 172.21.15.160:0/3692205310 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcef8100580 0x7fcef810af00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcee800d770 tx=0x7fcee800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:38:37.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.275+0000 7fcef4ff9640 1 -- 172.21.15.160:0/3692205310 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee8017070 con 0x7fcef8100580 2024-03-07T07:38:37.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.275+0000 7fcefdba7640 1 -- 172.21.15.160:0/3692205310 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcef81a1a10 con 0x7fcef8100580 2024-03-07T07:38:37.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.275+0000 7fcef4ff9640 1 -- 172.21.15.160:0/3692205310 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcee8009c30 con 0x7fcef8100580 2024-03-07T07:38:37.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.275+0000 7fcef4ff9640 1 -- 172.21.15.160:0/3692205310 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee8003dd0 con 0x7fcef8100580 2024-03-07T07:38:37.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.275+0000 7fcefdba7640 1 -- 172.21.15.160:0/3692205310 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcef81a1f60 con 0x7fcef8100580 2024-03-07T07:38:37.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.276+0000 7fcefdba7640 1 -- 172.21.15.160:0/3692205310 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcef80688a0 con 0x7fcef8100580 2024-03-07T07:38:37.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.277+0000 7fcef4ff9640 1 -- 172.21.15.160:0/3692205310 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7fcee8003f70 con 0x7fcef8100580 2024-03-07T07:38:37.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.277+0000 7fcef4ff9640 1 --2- 172.21.15.160:0/3692205310 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fcee00a1d80 0x7fcee00a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:37.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.278+0000 7fcef6ffd640 1 --2- 172.21.15.160:0/3692205310 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fcee00a1d80 0x7fcee00a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:37.280 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.278+0000 7fcef4ff9640 1 -- 172.21.15.160:0/3692205310 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fcee80cd320 con 0x7fcef8100580 2024-03-07T07:38:37.280 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.278+0000 7fcef6ffd640 1 --2- 172.21.15.160:0/3692205310 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fcee00a1d80 0x7fcee00a4240 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcef810ac70 tx=0x7fceec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:38:37.282 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.282+0000 7fcef4ff9640 1 -- 172.21.15.160:0/3692205310 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fcee809abf0 con 0x7fcef8100580 2024-03-07T07:38:37.494 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.494+0000 7fcefdba7640 1 -- 172.21.15.160:0/3692205310 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcef81078a0 con 0x7fcef8100580 2024-03-07T07:38:37.495 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.494+0000 7fcef4ff9640 1 -- 172.21.15.160:0/3692205310 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1325 (secure 0 0 0) 0x7fcee8009de0 con 0x7fcef8100580 2024-03-07T07:38:37.495 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:38:37.495 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/645365846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":645365846},{"type":"v1","addr":"172.21.15.178:6833","nonce":645365846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-07T07:38:37.495 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 2 2024-03-07T07:38:37.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.497+0000 7fcefdba7640 1 -- 172.21.15.160:0/3692205310 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fcee00a1d80 msgr2=0x7fcee00a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:37.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.497+0000 7fcefdba7640 1 --2- 172.21.15.160:0/3692205310 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fcee00a1d80 0x7fcee00a4240 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcef810ac70 tx=0x7fceec005f90 comp rx=0 tx=0).stop 2024-03-07T07:38:37.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.497+0000 7fcefdba7640 1 -- 172.21.15.160:0/3692205310 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcef8100580 msgr2=0x7fcef810af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:37.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.497+0000 7fcefdba7640 1 --2- 172.21.15.160:0/3692205310 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcef8100580 0x7fcef810af00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcee800d770 tx=0x7fcee800dc30 comp rx=0 tx=0).stop 2024-03-07T07:38:37.498 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.497+0000 7fcefdba7640 1 -- 172.21.15.160:0/3692205310 shutdown_connections 2024-03-07T07:38:37.498 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.497+0000 7fcefdba7640 1 --2- 172.21.15.160:0/3692205310 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fcee00a1d80 0x7fcee00a4240 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:37.498 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.497+0000 7fcefdba7640 1 --2- 172.21.15.160:0/3692205310 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcef8102130 0x7fcef8109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:37.498 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.497+0000 7fcefdba7640 1 --2- 172.21.15.160:0/3692205310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcef8101770 0x7fcef8109540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:37.498 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.497+0000 7fcefdba7640 1 --2- 172.21.15.160:0/3692205310 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcef8100580 0x7fcef810af00 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:37.498 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.497+0000 7fcefdba7640 1 -- 172.21.15.160:0/3692205310 >> 172.21.15.160:0/3692205310 conn(0x7fcef80fbaf0 msgr2=0x7fcef8104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:38:37.498 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.497+0000 7fcefdba7640 1 -- 172.21.15.160:0/3692205310 shutdown_connections 2024-03-07T07:38:37.498 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:37.497+0000 7fcefdba7640 1 -- 172.21.15.160:0/3692205310 wait complete. 2024-03-07T07:38:37.507 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:37 smithi160 ceph-mon[31963]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:37.507 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:37 smithi160 ceph-mon[31963]: mds.? [v2:172.21.15.178:6832/645365846,v1:172.21.15.178:6833/645365846] up:boot 2024-03-07T07:38:37.507 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:37 smithi160 ceph-mon[31963]: fsmap 1 up:standby 2024-03-07T07:38:37.507 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:37 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-07T07:38:37.507 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:37 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:37.507 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:37 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:37.507 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:37 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:37.508 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:37 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-07T07:38:37.508 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:37 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-07T07:38:37.508 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:37 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:38:37.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:37 smithi177 ceph-mon[36578]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:37.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:37 smithi177 ceph-mon[36578]: mds.? [v2:172.21.15.178:6832/645365846,v1:172.21.15.178:6833/645365846] up:boot 2024-03-07T07:38:37.608 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:37 smithi177 ceph-mon[36578]: fsmap 1 up:standby 2024-03-07T07:38:37.608 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:37 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-07T07:38:37.608 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:37 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:37.608 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:37 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:37.608 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:37 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:37.608 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:37 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-07T07:38:37.608 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:37 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-07T07:38:37.608 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:37 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:38:37.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:37 smithi178 ceph-mon[36650]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:37.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:37 smithi178 ceph-mon[36650]: mds.? [v2:172.21.15.178:6832/645365846,v1:172.21.15.178:6833/645365846] up:boot 2024-03-07T07:38:37.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:37 smithi178 ceph-mon[36650]: fsmap 1 up:standby 2024-03-07T07:38:37.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:37 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-07T07:38:37.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:37 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:37.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:37 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:37.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:37 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:37.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:37 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-07T07:38:37.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:37 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-07T07:38:37.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:37 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:38:38.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:38 smithi177 ceph-mon[36578]: Deploying daemon mds.k on smithi177 2024-03-07T07:38:38.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:38 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/3692205310' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:38:38.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:38 smithi160 ceph-mon[31963]: Deploying daemon mds.k on smithi177 2024-03-07T07:38:38.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:38 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/3692205310' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:38:38.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:38 smithi178 ceph-mon[36650]: Deploying daemon mds.k on smithi177 2024-03-07T07:38:38.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:38 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/3692205310' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:38:39.107 INFO:journalctl@ceph.mds.k.smithi177.stdout:Mar 07 07:38:38 smithi177 systemd[1]: Starting Ceph mds.k for 826f6582-dc54-11ee-95c6-87774f69a715... 2024-03-07T07:38:39.386 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:39 smithi177 ceph-mon[36578]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:39.386 INFO:journalctl@ceph.mds.k.smithi177.stdout:Mar 07 07:38:39 smithi177 podman[66055]: 2024-03-07 07:38:39.015521752 +0000 UTC m=+0.017164750 image pull e362cd7cebfee51077fb90b7f01895811140c9a4272edd83ad9910b05e3a46f6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2 2024-03-07T07:38:39.386 INFO:journalctl@ceph.mds.k.smithi177.stdout:Mar 07 07:38:39 smithi177 podman[66055]: 2024-03-07 07:38:39.150041409 +0000 UTC m=+0.151684397 container create 3a9e5ab2868658995fc4b33fc24267e794d7e50c334174c482a6e5b87d72ea41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-k, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-07T07:38:39.386 INFO:journalctl@ceph.mds.k.smithi177.stdout:Mar 07 07:38:39 smithi177 podman[66055]: 2024-03-07 07:38:39.386525693 +0000 UTC m=+0.388168686 container init 3a9e5ab2868658995fc4b33fc24267e794d7e50c334174c482a6e5b87d72ea41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-k, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-8b896a9, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-07T07:38:39.510 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T07:38:39.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.625+0000 7f32833eb640 1 -- 172.21.15.160:0/1123365427 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f327c103250 msgr2=0x7f327c109ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:39.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.625+0000 7f32833eb640 1 --2- 172.21.15.160:0/1123365427 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f327c103250 0x7f327c109ae0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3274009ca0 tx=0x7f327402c0f0 comp rx=0 tx=0).stop 2024-03-07T07:38:39.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.625+0000 7f32833eb640 1 -- 172.21.15.160:0/1123365427 shutdown_connections 2024-03-07T07:38:39.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.625+0000 7f32833eb640 1 --2- 172.21.15.160:0/1123365427 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f327c103250 0x7f327c109ae0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:39.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.625+0000 7f32833eb640 1 --2- 172.21.15.160:0/1123365427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f327c102890 0x7f327c102d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:39.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.625+0000 7f32833eb640 1 --2- 172.21.15.160:0/1123365427 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f327c1016a0 0x7f327c101ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:39.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.625+0000 7f32833eb640 1 -- 172.21.15.160:0/1123365427 >> 172.21.15.160:0/1123365427 conn(0x7f327c0fcc10 msgr2=0x7f327c0ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:38:39.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.626+0000 7f32833eb640 1 -- 172.21.15.160:0/1123365427 shutdown_connections 2024-03-07T07:38:39.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.626+0000 7f32833eb640 1 -- 172.21.15.160:0/1123365427 wait complete. 2024-03-07T07:38:39.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.626+0000 7f32833eb640 1 Processor -- start 2024-03-07T07:38:39.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.626+0000 7f32833eb640 1 -- start start 2024-03-07T07:38:39.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.626+0000 7f32833eb640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f327c1016a0 0x7f327c199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:39.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.626+0000 7f32833eb640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f327c102890 0x7f327c19a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:39.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.626+0000 7f32833eb640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f327c103250 0x7f327c1a1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:39.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.626+0000 7f32833eb640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327c10d320 con 0x7f327c102890 2024-03-07T07:38:39.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.626+0000 7f32833eb640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327c10d1a0 con 0x7f327c103250 2024-03-07T07:38:39.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.626+0000 7f32833eb640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327c10d4a0 con 0x7f327c1016a0 2024-03-07T07:38:39.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.626+0000 7f3280986640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f327c102890 0x7f327c19a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:39.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.626+0000 7f3280986640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f327c102890 0x7f327c19a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:56284/0 (socket says 172.21.15.160:56284) 2024-03-07T07:38:39.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.626+0000 7f3280986640 1 -- 172.21.15.160:0/2903884760 learned_addr learned my addr 172.21.15.160:0/2903884760 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:38:39.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.626+0000 7f3281187640 1 --2- 172.21.15.160:0/2903884760 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f327c1016a0 0x7f327c199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:39.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.627+0000 7f3281988640 1 --2- 172.21.15.160:0/2903884760 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f327c103250 0x7f327c1a1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:39.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.627+0000 7f3280986640 1 -- 172.21.15.160:0/2903884760 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f327c1016a0 msgr2=0x7f327c199c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:39.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.627+0000 7f3280986640 1 --2- 172.21.15.160:0/2903884760 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f327c1016a0 0x7f327c199c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:39.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.627+0000 7f3280986640 1 -- 172.21.15.160:0/2903884760 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f327c103250 msgr2=0x7f327c1a1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:39.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.627+0000 7f3280986640 1 --2- 172.21.15.160:0/2903884760 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f327c103250 0x7f327c1a1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:39.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.627+0000 7f3280986640 1 -- 172.21.15.160:0/2903884760 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f327c1a1780 con 0x7f327c102890 2024-03-07T07:38:39.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.627+0000 7f3281187640 1 --2- 172.21.15.160:0/2903884760 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f327c1016a0 0x7f327c199c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:38:39.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.627+0000 7f3280986640 1 --2- 172.21.15.160:0/2903884760 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f327c102890 0x7f327c19a140 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f326c00e9f0 tx=0x7f326c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:38:39.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.628+0000 7f32727fc640 1 -- 172.21.15.160:0/2903884760 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326c00cd80 con 0x7f327c102890 2024-03-07T07:38:39.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.628+0000 7f32833eb640 1 -- 172.21.15.160:0/2903884760 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f327c1a1a70 con 0x7f327c102890 2024-03-07T07:38:39.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.628+0000 7f32833eb640 1 -- 172.21.15.160:0/2903884760 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f327c1a1fc0 con 0x7f327c102890 2024-03-07T07:38:39.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.628+0000 7f32727fc640 1 -- 172.21.15.160:0/2903884760 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f326c005500 con 0x7f327c102890 2024-03-07T07:38:39.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.628+0000 7f32727fc640 1 -- 172.21.15.160:0/2903884760 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326c010690 con 0x7f327c102890 2024-03-07T07:38:39.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.629+0000 7f32833eb640 1 -- 172.21.15.160:0/2903884760 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32600052c0 con 0x7f327c102890 2024-03-07T07:38:39.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.630+0000 7f32727fc640 1 -- 172.21.15.160:0/2903884760 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f326c0046c0 con 0x7f327c102890 2024-03-07T07:38:39.632 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.630+0000 7f32727fc640 1 --2- 172.21.15.160:0/2903884760 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f32680a1d80 0x7f32680a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:39.632 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.631+0000 7f3281187640 1 --2- 172.21.15.160:0/2903884760 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f32680a1d80 0x7f32680a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:39.632 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.631+0000 7f32727fc640 1 -- 172.21.15.160:0/2903884760 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f326c0cdde0 con 0x7f327c102890 2024-03-07T07:38:39.632 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.631+0000 7f3281187640 1 --2- 172.21.15.160:0/2903884760 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f32680a1d80 0x7f32680a4240 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f327c1026f0 tx=0x7f3278005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:38:39.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.635+0000 7f32727fc640 1 -- 172.21.15.160:0/2903884760 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f326c09b6b0 con 0x7f327c102890 2024-03-07T07:38:39.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:39 smithi160 ceph-mon[31963]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:39.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:39 smithi178 ceph-mon[36650]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:39.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.854+0000 7f32833eb640 1 -- 172.21.15.160:0/2903884760 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3260005880 con 0x7f327c102890 2024-03-07T07:38:39.855 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.855+0000 7f32727fc640 1 -- 172.21.15.160:0/2903884760 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1325 (secure 0 0 0) 0x7f326c09afb0 con 0x7f327c102890 2024-03-07T07:38:39.855 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:38:39.855 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/645365846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":645365846},{"type":"v1","addr":"172.21.15.178:6833","nonce":645365846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-07T07:38:39.855 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 2 2024-03-07T07:38:39.856 INFO:journalctl@ceph.mds.k.smithi177.stdout:Mar 07 07:38:39 smithi177 podman[66055]: 2024-03-07 07:38:39.390138619 +0000 UTC m=+0.391781599 container start 3a9e5ab2868658995fc4b33fc24267e794d7e50c334174c482a6e5b87d72ea41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-k, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-8b896a9, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-03-07T07:38:39.856 INFO:journalctl@ceph.mds.k.smithi177.stdout:Mar 07 07:38:39 smithi177 ceph-mds[66074]: -- 172.21.15.177:0/2009136423 <== mon.2 v2:172.21.15.177:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d811cbba40 con 0x55d811c8f000 2024-03-07T07:38:39.856 INFO:journalctl@ceph.mds.k.smithi177.stdout:Mar 07 07:38:39 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-k[66070]: starting mds.k at 2024-03-07T07:38:39.857 INFO:journalctl@ceph.mds.k.smithi177.stdout:Mar 07 07:38:39 smithi177 bash[66055]: 3a9e5ab2868658995fc4b33fc24267e794d7e50c334174c482a6e5b87d72ea41 2024-03-07T07:38:39.857 INFO:journalctl@ceph.mds.k.smithi177.stdout:Mar 07 07:38:39 smithi177 systemd[1]: Started Ceph mds.k for 826f6582-dc54-11ee-95c6-87774f69a715. 2024-03-07T07:38:39.857 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.857+0000 7f32833eb640 1 -- 172.21.15.160:0/2903884760 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f32680a1d80 msgr2=0x7f32680a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:39.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.857+0000 7f32833eb640 1 --2- 172.21.15.160:0/2903884760 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f32680a1d80 0x7f32680a4240 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f327c1026f0 tx=0x7f3278005f90 comp rx=0 tx=0).stop 2024-03-07T07:38:39.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.857+0000 7f32833eb640 1 -- 172.21.15.160:0/2903884760 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f327c102890 msgr2=0x7f327c19a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:39.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.857+0000 7f32833eb640 1 --2- 172.21.15.160:0/2903884760 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f327c102890 0x7f327c19a140 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f326c00e9f0 tx=0x7f326c00edf0 comp rx=0 tx=0).stop 2024-03-07T07:38:39.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.857+0000 7f32833eb640 1 -- 172.21.15.160:0/2903884760 shutdown_connections 2024-03-07T07:38:39.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.857+0000 7f32833eb640 1 --2- 172.21.15.160:0/2903884760 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f32680a1d80 0x7f32680a4240 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:39.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.857+0000 7f32833eb640 1 --2- 172.21.15.160:0/2903884760 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f327c103250 0x7f327c1a1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:39.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.857+0000 7f32833eb640 1 --2- 172.21.15.160:0/2903884760 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f327c102890 0x7f327c19a140 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:39.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.857+0000 7f32833eb640 1 --2- 172.21.15.160:0/2903884760 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f327c1016a0 0x7f327c199c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:39.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.857+0000 7f32833eb640 1 -- 172.21.15.160:0/2903884760 >> 172.21.15.160:0/2903884760 conn(0x7f327c0fcc10 msgr2=0x7f327c073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:38:39.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.857+0000 7f32833eb640 1 -- 172.21.15.160:0/2903884760 shutdown_connections 2024-03-07T07:38:39.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:39.857+0000 7f32833eb640 1 -- 172.21.15.160:0/2903884760 wait complete. 2024-03-07T07:38:40.842 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:40 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:40.842 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:40 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:40.843 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:40 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:40.843 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:40 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-07T07:38:40.843 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:40 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-07T07:38:40.843 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:40 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:38:40.843 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:40 smithi177 ceph-mon[36578]: Deploying daemon mds.j on smithi160 2024-03-07T07:38:40.843 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:40 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/2903884760' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:38:40.843 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:40 smithi177 ceph-mon[36578]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:40.843 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:40 smithi177 ceph-mon[36578]: mds.? [v2:172.21.15.177:6832/1457364761,v1:172.21.15.177:6833/1457364761] up:boot 2024-03-07T07:38:40.843 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:40 smithi177 ceph-mon[36578]: fsmap 2 up:standby 2024-03-07T07:38:40.843 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:40 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-07T07:38:40.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:40 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:40.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:40 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:40.929 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:40 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:40.929 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:40 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-07T07:38:40.929 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:40 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-07T07:38:40.929 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:40 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:38:40.929 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:40 smithi160 ceph-mon[31963]: Deploying daemon mds.j on smithi160 2024-03-07T07:38:40.929 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:40 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/2903884760' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:38:40.929 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:40 smithi160 ceph-mon[31963]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:40.929 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:40 smithi160 ceph-mon[31963]: mds.? [v2:172.21.15.177:6832/1457364761,v1:172.21.15.177:6833/1457364761] up:boot 2024-03-07T07:38:40.929 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:40 smithi160 ceph-mon[31963]: fsmap 2 up:standby 2024-03-07T07:38:40.929 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:40 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-07T07:38:41.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:40 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:41.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:40 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:41.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:40 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:41.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:40 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-07T07:38:41.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:40 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-07T07:38:41.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:40 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:38:41.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:40 smithi178 ceph-mon[36650]: Deploying daemon mds.j on smithi160 2024-03-07T07:38:41.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:40 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/2903884760' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:38:41.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:40 smithi178 ceph-mon[36650]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:41.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:40 smithi178 ceph-mon[36650]: mds.? [v2:172.21.15.177:6832/1457364761,v1:172.21.15.177:6833/1457364761] up:boot 2024-03-07T07:38:41.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:40 smithi178 ceph-mon[36650]: fsmap 2 up:standby 2024-03-07T07:38:41.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:40 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-07T07:38:41.869 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T07:38:41.898 INFO:journalctl@ceph.mds.j.smithi160.stdout:Mar 07 07:38:41 smithi160 systemd[1]: Starting Ceph mds.j for 826f6582-dc54-11ee-95c6-87774f69a715... 2024-03-07T07:38:41.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:41.982+0000 7fc4adfde640 1 -- 172.21.15.160:0/3463385083 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc4a8101770 msgr2=0x7fc4a8101bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:41.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:41.982+0000 7fc4adfde640 1 --2- 172.21.15.160:0/3463385083 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc4a8101770 0x7fc4a8101bf0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc490009900 tx=0x7fc49002bc30 comp rx=0 tx=0).stop 2024-03-07T07:38:41.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:41.982+0000 7fc4adfde640 1 -- 172.21.15.160:0/3463385083 shutdown_connections 2024-03-07T07:38:41.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:41.982+0000 7fc4adfde640 1 --2- 172.21.15.160:0/3463385083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4a8102130 0x7fc4a81089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:41.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:41.982+0000 7fc4adfde640 1 --2- 172.21.15.160:0/3463385083 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc4a8101770 0x7fc4a8101bf0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:41.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:41.982+0000 7fc4adfde640 1 --2- 172.21.15.160:0/3463385083 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc4a8100580 0x7fc4a81009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:41.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:41.982+0000 7fc4adfde640 1 -- 172.21.15.160:0/3463385083 >> 172.21.15.160:0/3463385083 conn(0x7fc4a80fbaf0 msgr2=0x7fc4a80fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:38:41.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:41.982+0000 7fc4adfde640 1 -- 172.21.15.160:0/3463385083 shutdown_connections 2024-03-07T07:38:41.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:41.982+0000 7fc4adfde640 1 -- 172.21.15.160:0/3463385083 wait complete. 2024-03-07T07:38:41.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:41.982+0000 7fc4adfde640 1 Processor -- start 2024-03-07T07:38:41.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:41.982+0000 7fc4adfde640 1 -- start start 2024-03-07T07:38:41.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:41.983+0000 7fc4adfde640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc4a8100580 0x7fc4a8104510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:41.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:41.983+0000 7fc4adfde640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc4a8101770 0x7fc4a8102b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:41.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:41.983+0000 7fc4adfde640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4a8102130 0x7fc4a8103090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:41.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:41.983+0000 7fc4adfde640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a810c210 con 0x7fc4a8101770 2024-03-07T07:38:41.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:41.983+0000 7fc4adfde640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a810c090 con 0x7fc4a8100580 2024-03-07T07:38:41.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:41.983+0000 7fc4adfde640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a810c390 con 0x7fc4a8102130 2024-03-07T07:38:41.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:41.983+0000 7fc4a6ffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc4a8101770 0x7fc4a8102b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:41.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:41.983+0000 7fc4a6ffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc4a8101770 0x7fc4a8102b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:56308/0 (socket says 172.21.15.160:56308) 2024-03-07T07:38:41.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:41.983+0000 7fc4a6ffd640 1 -- 172.21.15.160:0/2655255394 learned_addr learned my addr 172.21.15.160:0/2655255394 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:38:41.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:41.983+0000 7fc4a77fe640 1 --2- 172.21.15.160:0/2655255394 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc4a8100580 0x7fc4a8104510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:41.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:41.983+0000 7fc4a7fff640 1 --2- 172.21.15.160:0/2655255394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4a8102130 0x7fc4a8103090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:41.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:41.983+0000 7fc4a6ffd640 1 -- 172.21.15.160:0/2655255394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4a8102130 msgr2=0x7fc4a8103090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:41.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:41.983+0000 7fc4a6ffd640 1 --2- 172.21.15.160:0/2655255394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4a8102130 0x7fc4a8103090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:41.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:41.983+0000 7fc4a6ffd640 1 -- 172.21.15.160:0/2655255394 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc4a8100580 msgr2=0x7fc4a8104510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:41.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:41.983+0000 7fc4a6ffd640 1 --2- 172.21.15.160:0/2655255394 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc4a8100580 0x7fc4a8104510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:41.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:41.983+0000 7fc4a6ffd640 1 -- 172.21.15.160:0/2655255394 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4a8103920 con 0x7fc4a8101770 2024-03-07T07:38:41.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:41.983+0000 7fc4a6ffd640 1 --2- 172.21.15.160:0/2655255394 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc4a8101770 0x7fc4a8102b50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc490009230 tx=0x7fc490004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:38:41.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:41.983+0000 7fc4a4ff9640 1 -- 172.21.15.160:0/2655255394 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc490041070 con 0x7fc4a8101770 2024-03-07T07:38:41.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:41.983+0000 7fc4adfde640 1 -- 172.21.15.160:0/2655255394 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4a81a1940 con 0x7fc4a8101770 2024-03-07T07:38:41.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:41.983+0000 7fc4a4ff9640 1 -- 172.21.15.160:0/2655255394 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc4900054e0 con 0x7fc4a8101770 2024-03-07T07:38:41.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:41.983+0000 7fc4a4ff9640 1 -- 172.21.15.160:0/2655255394 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4900346a0 con 0x7fc4a8101770 2024-03-07T07:38:41.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:41.983+0000 7fc4adfde640 1 -- 172.21.15.160:0/2655255394 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4a81a1e90 con 0x7fc4a8101770 2024-03-07T07:38:41.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:41.984+0000 7fc4adfde640 1 -- 172.21.15.160:0/2655255394 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4a81a2060 con 0x7fc4a8101770 2024-03-07T07:38:41.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:41.985+0000 7fc4a4ff9640 1 -- 172.21.15.160:0/2655255394 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7fc490044070 con 0x7fc4a8101770 2024-03-07T07:38:41.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:41.986+0000 7fc4a4ff9640 1 --2- 172.21.15.160:0/2655255394 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fc4940a1d80 0x7fc4940a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:41.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:41.986+0000 7fc4a77fe640 1 --2- 172.21.15.160:0/2655255394 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fc4940a1d80 0x7fc4940a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:41.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:41.986+0000 7fc4a4ff9640 1 -- 172.21.15.160:0/2655255394 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc4900ef620 con 0x7fc4a8101770 2024-03-07T07:38:41.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:41.986+0000 7fc4a77fe640 1 --2- 172.21.15.160:0/2655255394 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fc4940a1d80 0x7fc4940a4240 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc4a81015d0 tx=0x7fc4a0007600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:38:41.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:41.990+0000 7fc4a4ff9640 1 -- 172.21.15.160:0/2655255394 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc4900bcef0 con 0x7fc4a8101770 2024-03-07T07:38:42.176 INFO:journalctl@ceph.mds.j.smithi160.stdout:Mar 07 07:38:42 smithi160 podman[72687]: 2024-03-07 07:38:41.939822069 +0000 UTC m=+0.018903997 image pull e362cd7cebfee51077fb90b7f01895811140c9a4272edd83ad9910b05e3a46f6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2 2024-03-07T07:38:42.176 INFO:journalctl@ceph.mds.j.smithi160.stdout:Mar 07 07:38:42 smithi160 podman[72687]: 2024-03-07 07:38:42.053735766 +0000 UTC m=+0.132817692 container create 219cb8715400277ef18909512abbd20ce27453cda0ff0ab1d498d9491875bf7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-j, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-07T07:38:42.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:42.210+0000 7fc4adfde640 1 -- 172.21.15.160:0/2655255394 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4a81a23f0 con 0x7fc4a8101770 2024-03-07T07:38:42.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:42.210+0000 7fc4a4ff9640 1 -- 172.21.15.160:0/2655255394 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2119 (secure 0 0 0) 0x7fc4900bc7f0 con 0x7fc4a8101770 2024-03-07T07:38:42.211 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:38:42.211 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1457364761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1457364761},{"type":"v1","addr":"172.21.15.177:6833","nonce":1457364761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/645365846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":645365846},{"type":"v1","addr":"172.21.15.178:6833","nonce":645365846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-07T07:38:42.211 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 3 2024-03-07T07:38:42.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:42.212+0000 7fc4adfde640 1 -- 172.21.15.160:0/2655255394 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fc4940a1d80 msgr2=0x7fc4940a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:42.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:42.212+0000 7fc4adfde640 1 --2- 172.21.15.160:0/2655255394 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fc4940a1d80 0x7fc4940a4240 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc4a81015d0 tx=0x7fc4a0007600 comp rx=0 tx=0).stop 2024-03-07T07:38:42.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:42.212+0000 7fc4adfde640 1 -- 172.21.15.160:0/2655255394 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc4a8101770 msgr2=0x7fc4a8102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:42.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:42.212+0000 7fc4adfde640 1 --2- 172.21.15.160:0/2655255394 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc4a8101770 0x7fc4a8102b50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc490009230 tx=0x7fc490004040 comp rx=0 tx=0).stop 2024-03-07T07:38:42.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:42.212+0000 7fc4adfde640 1 -- 172.21.15.160:0/2655255394 shutdown_connections 2024-03-07T07:38:42.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:42.212+0000 7fc4adfde640 1 --2- 172.21.15.160:0/2655255394 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fc4940a1d80 0x7fc4940a4240 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:42.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:42.212+0000 7fc4adfde640 1 --2- 172.21.15.160:0/2655255394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4a8102130 0x7fc4a8103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:42.214 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:42.212+0000 7fc4adfde640 1 --2- 172.21.15.160:0/2655255394 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc4a8101770 0x7fc4a8102b50 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:42.214 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:42.212+0000 7fc4adfde640 1 --2- 172.21.15.160:0/2655255394 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc4a8100580 0x7fc4a8104510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:42.214 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:42.212+0000 7fc4adfde640 1 -- 172.21.15.160:0/2655255394 >> 172.21.15.160:0/2655255394 conn(0x7fc4a80fbaf0 msgr2=0x7fc4a80fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:38:42.214 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:42.213+0000 7fc4adfde640 1 -- 172.21.15.160:0/2655255394 shutdown_connections 2024-03-07T07:38:42.214 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:42.213+0000 7fc4adfde640 1 -- 172.21.15.160:0/2655255394 wait complete. 2024-03-07T07:38:42.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:42 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/2655255394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:38:42.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:42 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/2655255394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:38:42.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:42 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/2655255394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:38:42.678 INFO:journalctl@ceph.mds.j.smithi160.stdout:Mar 07 07:38:42 smithi160 podman[72687]: 2024-03-07 07:38:42.290062724 +0000 UTC m=+0.369144648 container init 219cb8715400277ef18909512abbd20ce27453cda0ff0ab1d498d9491875bf7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-j, org.label-schema.build-date=20240222, ceph=True, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.vendor=CentOS, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-07T07:38:42.678 INFO:journalctl@ceph.mds.j.smithi160.stdout:Mar 07 07:38:42 smithi160 podman[72687]: 2024-03-07 07:38:42.292736009 +0000 UTC m=+0.371817933 container start 219cb8715400277ef18909512abbd20ce27453cda0ff0ab1d498d9491875bf7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-j, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-07T07:38:42.678 INFO:journalctl@ceph.mds.j.smithi160.stdout:Mar 07 07:38:42 smithi160 ceph-mds[72727]: -- 172.21.15.160:0/3696614139 <== mon.0 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fe050e1a40 con 0x55fe050b5000 2024-03-07T07:38:42.678 INFO:journalctl@ceph.mds.j.smithi160.stdout:Mar 07 07:38:42 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-j[72721]: starting mds.j at 2024-03-07T07:38:42.678 INFO:journalctl@ceph.mds.j.smithi160.stdout:Mar 07 07:38:42 smithi160 bash[72687]: 219cb8715400277ef18909512abbd20ce27453cda0ff0ab1d498d9491875bf7e 2024-03-07T07:38:42.678 INFO:journalctl@ceph.mds.j.smithi160.stdout:Mar 07 07:38:42 smithi160 systemd[1]: Started Ceph mds.j for 826f6582-dc54-11ee-95c6-87774f69a715. 2024-03-07T07:38:43.319 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:43 smithi160 ceph-mon[31963]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:43.319 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:43 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:43.319 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:43 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:43.319 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:43 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:43.319 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:43 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-07T07:38:43.319 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:43 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-07T07:38:43.319 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:43 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:38:43.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:43 smithi178 ceph-mon[36650]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:43.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:43 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:43.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:43 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:43.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:43 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:43.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:43 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-07T07:38:43.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:43 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-07T07:38:43.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:43 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:38:43.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:43 smithi177 ceph-mon[36578]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:43.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:43 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:43.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:43 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:43.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:43 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:43.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:43 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-07T07:38:43.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:43 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-07T07:38:43.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:43 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:38:44.225 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T07:38:44.363 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.361+0000 7efda891d640 1 -- 172.21.15.160:0/2637857725 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efda4102160 msgr2=0x7efda4104550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:44.363 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.361+0000 7efda891d640 1 --2- 172.21.15.160:0/2637857725 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efda4102160 0x7efda4104550 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7efd94009900 tx=0x7efd9402bc30 comp rx=0 tx=0).stop 2024-03-07T07:38:44.363 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.361+0000 7efda891d640 1 -- 172.21.15.160:0/2637857725 shutdown_connections 2024-03-07T07:38:44.363 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.361+0000 7efda891d640 1 --2- 172.21.15.160:0/2637857725 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efda41073c0 0x7efda41097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:44.364 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.361+0000 7efda891d640 1 --2- 172.21.15.160:0/2637857725 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efda4104a90 0x7efda4106e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:44.364 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.361+0000 7efda891d640 1 --2- 172.21.15.160:0/2637857725 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efda4102160 0x7efda4104550 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:44.364 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.361+0000 7efda891d640 1 -- 172.21.15.160:0/2637857725 >> 172.21.15.160:0/2637857725 conn(0x7efda40fbaf0 msgr2=0x7efda40fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:38:44.364 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.361+0000 7efda891d640 1 -- 172.21.15.160:0/2637857725 shutdown_connections 2024-03-07T07:38:44.364 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.361+0000 7efda891d640 1 -- 172.21.15.160:0/2637857725 wait complete. 2024-03-07T07:38:44.364 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.361+0000 7efda891d640 1 Processor -- start 2024-03-07T07:38:44.364 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.362+0000 7efda891d640 1 -- start start 2024-03-07T07:38:44.364 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.362+0000 7efda891d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efda4102160 0x7efda41979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:44.364 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.362+0000 7efda891d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efda4104a90 0x7efda4197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:44.364 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.362+0000 7efda891d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efda41073c0 0x7efda419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:44.364 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.362+0000 7efda891d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda410d320 con 0x7efda4104a90 2024-03-07T07:38:44.364 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.362+0000 7efda891d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda410d1a0 con 0x7efda41073c0 2024-03-07T07:38:44.365 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.362+0000 7efda891d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda410d4a0 con 0x7efda4102160 2024-03-07T07:38:44.365 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.362+0000 7efd9bfff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efda4104a90 0x7efda4197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:44.365 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.362+0000 7efd9bfff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efda4104a90 0x7efda4197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:56354/0 (socket says 172.21.15.160:56354) 2024-03-07T07:38:44.365 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.362+0000 7efd9bfff640 1 -- 172.21.15.160:0/3537397641 learned_addr learned my addr 172.21.15.160:0/3537397641 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:38:44.365 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.362+0000 7efda259c640 1 --2- 172.21.15.160:0/3537397641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efda4102160 0x7efda41979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:44.365 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.362+0000 7efda2d9d640 1 --2- 172.21.15.160:0/3537397641 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efda41073c0 0x7efda419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:44.365 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.362+0000 7efd9bfff640 1 -- 172.21.15.160:0/3537397641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efda4102160 msgr2=0x7efda41979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:44.365 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.362+0000 7efd9bfff640 1 --2- 172.21.15.160:0/3537397641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efda4102160 0x7efda41979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:44.365 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.362+0000 7efd9bfff640 1 -- 172.21.15.160:0/3537397641 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efda41073c0 msgr2=0x7efda419ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:44.365 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.362+0000 7efd9bfff640 1 --2- 172.21.15.160:0/3537397641 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efda41073c0 0x7efda419ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:44.365 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.362+0000 7efd9bfff640 1 -- 172.21.15.160:0/3537397641 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda419f5e0 con 0x7efda4104a90 2024-03-07T07:38:44.365 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.362+0000 7efd9bfff640 1 --2- 172.21.15.160:0/3537397641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efda4104a90 0x7efda4197f00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7efd8c00c970 tx=0x7efd8c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:38:44.365 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.362+0000 7efd9b7fe640 1 -- 172.21.15.160:0/3537397641 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8c007bb0 con 0x7efda4104a90 2024-03-07T07:38:44.365 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.362+0000 7efd9b7fe640 1 -- 172.21.15.160:0/3537397641 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efd8c007d50 con 0x7efda4104a90 2024-03-07T07:38:44.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.362+0000 7efd9b7fe640 1 -- 172.21.15.160:0/3537397641 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8c00f450 con 0x7efda4104a90 2024-03-07T07:38:44.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.362+0000 7efda891d640 1 -- 172.21.15.160:0/3537397641 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efda419f8d0 con 0x7efda4104a90 2024-03-07T07:38:44.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.363+0000 7efda891d640 1 -- 172.21.15.160:0/3537397641 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efda419fe70 con 0x7efda4104a90 2024-03-07T07:38:44.367 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.364+0000 7efda891d640 1 -- 172.21.15.160:0/3537397641 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efda40fd5e0 con 0x7efda4104a90 2024-03-07T07:38:44.367 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.365+0000 7efd9b7fe640 1 -- 172.21.15.160:0/3537397641 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7efd8c0046c0 con 0x7efda4104a90 2024-03-07T07:38:44.367 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.365+0000 7efd9b7fe640 1 --2- 172.21.15.160:0/3537397641 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7efd900a1e00 0x7efd900a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:44.367 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.365+0000 7efda259c640 1 --2- 172.21.15.160:0/3537397641 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7efd900a1e00 0x7efd900a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:44.367 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.365+0000 7efd9b7fe640 1 -- 172.21.15.160:0/3537397641 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7efd8c0cd8d0 con 0x7efda4104a90 2024-03-07T07:38:44.367 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.366+0000 7efda259c640 1 --2- 172.21.15.160:0/3537397641 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7efd900a1e00 0x7efd900a42c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7efd9402c7b0 tx=0x7efd94005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:38:44.370 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.370+0000 7efd9b7fe640 1 -- 172.21.15.160:0/3537397641 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7efd8c014030 con 0x7efda4104a90 2024-03-07T07:38:44.471 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:44 smithi160 ceph-mon[31963]: Deploying daemon mds.d on smithi160 2024-03-07T07:38:44.471 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:44 smithi160 ceph-mon[31963]: mds.? [v2:172.21.15.160:6834/2615961083,v1:172.21.15.160:6835/2615961083] up:boot 2024-03-07T07:38:44.471 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:44 smithi160 ceph-mon[31963]: fsmap 3 up:standby 2024-03-07T07:38:44.471 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:44 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-07T07:38:44.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:44 smithi178 ceph-mon[36650]: Deploying daemon mds.d on smithi160 2024-03-07T07:38:44.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:44 smithi178 ceph-mon[36650]: mds.? [v2:172.21.15.160:6834/2615961083,v1:172.21.15.160:6835/2615961083] up:boot 2024-03-07T07:38:44.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:44 smithi178 ceph-mon[36650]: fsmap 3 up:standby 2024-03-07T07:38:44.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:44 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-07T07:38:44.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:44 smithi177 ceph-mon[36578]: Deploying daemon mds.d on smithi160 2024-03-07T07:38:44.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:44 smithi177 ceph-mon[36578]: mds.? [v2:172.21.15.160:6834/2615961083,v1:172.21.15.160:6835/2615961083] up:boot 2024-03-07T07:38:44.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:44 smithi177 ceph-mon[36578]: fsmap 3 up:standby 2024-03-07T07:38:44.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:44 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-07T07:38:44.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.616+0000 7efda891d640 1 -- 172.21.15.160:0/3537397641 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efda402d200 con 0x7efda4104a90 2024-03-07T07:38:44.617 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.616+0000 7efd9b7fe640 1 -- 172.21.15.160:0/3537397641 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2913 (secure 0 0 0) 0x7efd8c09b1a0 con 0x7efda4104a90 2024-03-07T07:38:44.617 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:38:44.617 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/2615961083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2615961083},{"type":"v1","addr":"172.21.15.160:6835","nonce":2615961083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1457364761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1457364761},{"type":"v1","addr":"172.21.15.177:6833","nonce":1457364761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/645365846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":645365846},{"type":"v1","addr":"172.21.15.178:6833","nonce":645365846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-07T07:38:44.617 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 4 2024-03-07T07:38:44.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.619+0000 7efda891d640 1 -- 172.21.15.160:0/3537397641 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7efd900a1e00 msgr2=0x7efd900a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:44.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.619+0000 7efda891d640 1 --2- 172.21.15.160:0/3537397641 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7efd900a1e00 0x7efd900a42c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7efd9402c7b0 tx=0x7efd94005fd0 comp rx=0 tx=0).stop 2024-03-07T07:38:44.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.619+0000 7efda891d640 1 -- 172.21.15.160:0/3537397641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efda4104a90 msgr2=0x7efda4197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:44.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.619+0000 7efda891d640 1 --2- 172.21.15.160:0/3537397641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efda4104a90 0x7efda4197f00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7efd8c00c970 tx=0x7efd8c00ce30 comp rx=0 tx=0).stop 2024-03-07T07:38:44.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.619+0000 7efda891d640 1 -- 172.21.15.160:0/3537397641 shutdown_connections 2024-03-07T07:38:44.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.619+0000 7efda891d640 1 --2- 172.21.15.160:0/3537397641 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7efd900a1e00 0x7efd900a42c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:44.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.619+0000 7efda891d640 1 --2- 172.21.15.160:0/3537397641 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efda41073c0 0x7efda419ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:44.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.619+0000 7efda891d640 1 --2- 172.21.15.160:0/3537397641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efda4104a90 0x7efda4197f00 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:44.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.619+0000 7efda891d640 1 --2- 172.21.15.160:0/3537397641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efda4102160 0x7efda41979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:44.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.619+0000 7efda891d640 1 -- 172.21.15.160:0/3537397641 >> 172.21.15.160:0/3537397641 conn(0x7efda40fbaf0 msgr2=0x7efda4102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:38:44.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.619+0000 7efda891d640 1 -- 172.21.15.160:0/3537397641 shutdown_connections 2024-03-07T07:38:44.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:44.619+0000 7efda891d640 1 -- 172.21.15.160:0/3537397641 wait complete. 2024-03-07T07:38:44.901 INFO:journalctl@ceph.mds.d.smithi160.stdout:Mar 07 07:38:44 smithi160 systemd[1]: Starting Ceph mds.d for 826f6582-dc54-11ee-95c6-87774f69a715... 2024-03-07T07:38:45.164 INFO:journalctl@ceph.mds.d.smithi160.stdout:Mar 07 07:38:44 smithi160 podman[73055]: 2024-03-07 07:38:44.804324686 +0000 UTC m=+0.018195866 image pull e362cd7cebfee51077fb90b7f01895811140c9a4272edd83ad9910b05e3a46f6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2 2024-03-07T07:38:45.164 INFO:journalctl@ceph.mds.d.smithi160.stdout:Mar 07 07:38:44 smithi160 podman[73055]: 2024-03-07 07:38:44.919078059 +0000 UTC m=+0.132949270 container create b876bcd750c29b84c7ed3821f7f7245fe229d4da46e7189bcbc6d085b91ee911 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-d, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-8b896a9, GIT_CLEAN=True, ceph=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-07T07:38:45.165 INFO:journalctl@ceph.mds.d.smithi160.stdout:Mar 07 07:38:45 smithi160 podman[73055]: 2024-03-07 07:38:45.163879261 +0000 UTC m=+0.377750435 container init b876bcd750c29b84c7ed3821f7f7245fe229d4da46e7189bcbc6d085b91ee911 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-d, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, RELEASE=wip-lusov-qdb-8b896a9, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-07T07:38:45.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:45 smithi160 ceph-mon[31963]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:45.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:45 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/3537397641' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:38:45.428 INFO:journalctl@ceph.mds.d.smithi160.stdout:Mar 07 07:38:45 smithi160 podman[73055]: 2024-03-07 07:38:45.166831232 +0000 UTC m=+0.380702402 container start b876bcd750c29b84c7ed3821f7f7245fe229d4da46e7189bcbc6d085b91ee911 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-lusov-qdb-8b896a9, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-07T07:38:45.428 INFO:journalctl@ceph.mds.d.smithi160.stdout:Mar 07 07:38:45 smithi160 ceph-mds[73075]: -- 172.21.15.160:0/2795659431 <== mon.0 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55eccfddfa40 con 0x55eccfdb3000 2024-03-07T07:38:45.428 INFO:journalctl@ceph.mds.d.smithi160.stdout:Mar 07 07:38:45 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-d[73071]: starting mds.d at 2024-03-07T07:38:45.429 INFO:journalctl@ceph.mds.d.smithi160.stdout:Mar 07 07:38:45 smithi160 bash[73055]: b876bcd750c29b84c7ed3821f7f7245fe229d4da46e7189bcbc6d085b91ee911 2024-03-07T07:38:45.429 INFO:journalctl@ceph.mds.d.smithi160.stdout:Mar 07 07:38:45 smithi160 systemd[1]: Started Ceph mds.d for 826f6582-dc54-11ee-95c6-87774f69a715. 2024-03-07T07:38:45.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:45 smithi178 ceph-mon[36650]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:45.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:45 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/3537397641' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:38:45.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:45 smithi177 ceph-mon[36578]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:45.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:45 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/3537397641' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:38:46.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:46 smithi178 ceph-mon[36650]: mds.? [v2:172.21.15.160:6836/1759006033,v1:172.21.15.160:6837/1759006033] up:boot 2024-03-07T07:38:46.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:46 smithi178 ceph-mon[36650]: fsmap 4 up:standby 2024-03-07T07:38:46.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:46 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-07T07:38:46.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:46 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:46.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:46 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:46.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:46 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:46.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:46 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-07T07:38:46.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:46 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-07T07:38:46.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:46 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:38:46.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:46 smithi178 ceph-mon[36650]: Deploying daemon mds.e on smithi177 2024-03-07T07:38:46.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:46 smithi177 ceph-mon[36578]: mds.? [v2:172.21.15.160:6836/1759006033,v1:172.21.15.160:6837/1759006033] up:boot 2024-03-07T07:38:46.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:46 smithi177 ceph-mon[36578]: fsmap 4 up:standby 2024-03-07T07:38:46.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:46 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-07T07:38:46.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:46 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:46.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:46 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:46.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:46 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:46.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:46 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-07T07:38:46.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:46 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-07T07:38:46.608 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:46 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:38:46.608 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:46 smithi177 ceph-mon[36578]: Deploying daemon mds.e on smithi177 2024-03-07T07:38:46.632 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T07:38:46.663 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:46 smithi160 ceph-mon[31963]: mds.? [v2:172.21.15.160:6836/1759006033,v1:172.21.15.160:6837/1759006033] up:boot 2024-03-07T07:38:46.664 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:46 smithi160 ceph-mon[31963]: fsmap 4 up:standby 2024-03-07T07:38:46.664 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:46 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-07T07:38:46.664 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:46 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:46.664 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:46 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:46.664 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:46 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:46.664 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:46 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-07T07:38:46.664 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:46 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-07T07:38:46.664 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:46 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:38:46.664 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:46 smithi160 ceph-mon[31963]: Deploying daemon mds.e on smithi177 2024-03-07T07:38:46.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.748+0000 7f28babee640 1 -- 172.21.15.160:0/2377002251 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28b4107400 msgr2=0x7f28b41097f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:46.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.748+0000 7f28babee640 1 --2- 172.21.15.160:0/2377002251 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28b4107400 0x7f28b41097f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f28ac009ca0 tx=0x7f28ac02c0f0 comp rx=0 tx=0).stop 2024-03-07T07:38:46.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.748+0000 7f28babee640 1 -- 172.21.15.160:0/2377002251 shutdown_connections 2024-03-07T07:38:46.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.748+0000 7f28babee640 1 --2- 172.21.15.160:0/2377002251 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28b4107400 0x7f28b41097f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:46.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.748+0000 7f28babee640 1 --2- 172.21.15.160:0/2377002251 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28b4104ad0 0x7f28b4106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:46.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.748+0000 7f28babee640 1 --2- 172.21.15.160:0/2377002251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28b41021a0 0x7f28b4104590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:46.752 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.748+0000 7f28babee640 1 -- 172.21.15.160:0/2377002251 >> 172.21.15.160:0/2377002251 conn(0x7f28b40fbaf0 msgr2=0x7f28b40fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:38:46.752 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.748+0000 7f28babee640 1 -- 172.21.15.160:0/2377002251 shutdown_connections 2024-03-07T07:38:46.752 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.748+0000 7f28babee640 1 -- 172.21.15.160:0/2377002251 wait complete. 2024-03-07T07:38:46.752 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.749+0000 7f28babee640 1 Processor -- start 2024-03-07T07:38:46.752 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.749+0000 7f28babee640 1 -- start start 2024-03-07T07:38:46.752 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.749+0000 7f28babee640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28b41021a0 0x7f28b4101a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:46.752 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.749+0000 7f28babee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28b4104ad0 0x7f28b4100150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:46.752 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.749+0000 7f28babee640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28b4107400 0x7f28b4100690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:46.752 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.749+0000 7f28babee640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b410d320 con 0x7f28b4107400 2024-03-07T07:38:46.752 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.749+0000 7f28babee640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b410d1a0 con 0x7f28b41021a0 2024-03-07T07:38:46.752 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.749+0000 7f28babee640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b410d4a0 con 0x7f28b4104ad0 2024-03-07T07:38:46.752 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.749+0000 7f28b918b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28b4107400 0x7f28b4100690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:46.753 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.749+0000 7f28b918b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28b4107400 0x7f28b4100690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:56386/0 (socket says 172.21.15.160:56386) 2024-03-07T07:38:46.753 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.749+0000 7f28b918b640 1 -- 172.21.15.160:0/2987352506 learned_addr learned my addr 172.21.15.160:0/2987352506 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:38:46.753 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.749+0000 7f28b898a640 1 --2- 172.21.15.160:0/2987352506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28b41021a0 0x7f28b4101a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:46.753 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.750+0000 7f28b3fff640 1 --2- 172.21.15.160:0/2987352506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28b4104ad0 0x7f28b4100150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:46.753 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.750+0000 7f28b918b640 1 -- 172.21.15.160:0/2987352506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28b4104ad0 msgr2=0x7f28b4100150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:46.753 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.750+0000 7f28b918b640 1 --2- 172.21.15.160:0/2987352506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28b4104ad0 0x7f28b4100150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:46.753 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.750+0000 7f28b918b640 1 -- 172.21.15.160:0/2987352506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28b41021a0 msgr2=0x7f28b4101a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:46.753 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.750+0000 7f28b918b640 1 --2- 172.21.15.160:0/2987352506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28b41021a0 0x7f28b4101a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:46.753 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.750+0000 7f28b918b640 1 -- 172.21.15.160:0/2987352506 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28b4100e60 con 0x7f28b4107400 2024-03-07T07:38:46.753 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.750+0000 7f28b898a640 1 --2- 172.21.15.160:0/2987352506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28b41021a0 0x7f28b4101a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:38:46.753 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.750+0000 7f28b918b640 1 --2- 172.21.15.160:0/2987352506 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28b4107400 0x7f28b4100690 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f28ac02c700 tx=0x7f28ac032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:38:46.753 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.750+0000 7f28b1ffb640 1 -- 172.21.15.160:0/2987352506 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ac039040 con 0x7f28b4107400 2024-03-07T07:38:46.753 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.750+0000 7f28b1ffb640 1 -- 172.21.15.160:0/2987352506 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f28ac044070 con 0x7f28b4107400 2024-03-07T07:38:46.753 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.750+0000 7f28babee640 1 -- 172.21.15.160:0/2987352506 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28b41a19f0 con 0x7f28b4107400 2024-03-07T07:38:46.753 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.750+0000 7f28b1ffb640 1 -- 172.21.15.160:0/2987352506 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ac0042a0 con 0x7f28b4107400 2024-03-07T07:38:46.754 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.750+0000 7f28babee640 1 -- 172.21.15.160:0/2987352506 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28b41a1eb0 con 0x7f28b4107400 2024-03-07T07:38:46.754 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.751+0000 7f28babee640 1 -- 172.21.15.160:0/2987352506 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28b4068480 con 0x7f28b4107400 2024-03-07T07:38:46.755 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.752+0000 7f28b1ffb640 1 -- 172.21.15.160:0/2987352506 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f28ac003970 con 0x7f28b4107400 2024-03-07T07:38:46.755 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.753+0000 7f28b1ffb640 1 --2- 172.21.15.160:0/2987352506 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f28a40a1d80 0x7f28a40a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:46.755 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.753+0000 7f28b898a640 1 --2- 172.21.15.160:0/2987352506 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f28a40a1d80 0x7f28a40a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:46.755 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.753+0000 7f28b1ffb640 1 -- 172.21.15.160:0/2987352506 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f28ac0efef0 con 0x7f28b4107400 2024-03-07T07:38:46.755 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.753+0000 7f28b898a640 1 --2- 172.21.15.160:0/2987352506 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f28a40a1d80 0x7f28a40a4240 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f28a80058c0 tx=0x7f28a8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:38:46.757 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.757+0000 7f28b1ffb640 1 -- 172.21.15.160:0/2987352506 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f28b4068480 con 0x7f28b4107400 2024-03-07T07:38:46.968 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.968+0000 7f28babee640 1 -- 172.21.15.160:0/2987352506 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28b402d0b0 con 0x7f28b4107400 2024-03-07T07:38:46.968 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.968+0000 7f28b1ffb640 1 -- 172.21.15.160:0/2987352506 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3707 (secure 0 0 0) 0x7f28ac0333a0 con 0x7f28b4107400 2024-03-07T07:38:46.969 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:38:46.969 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/2615961083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2615961083},{"type":"v1","addr":"172.21.15.160:6835","nonce":2615961083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1759006033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1759006033},{"type":"v1","addr":"172.21.15.160:6837","nonce":1759006033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1457364761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1457364761},{"type":"v1","addr":"172.21.15.177:6833","nonce":1457364761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/645365846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":645365846},{"type":"v1","addr":"172.21.15.178:6833","nonce":645365846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-07T07:38:46.969 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 5 2024-03-07T07:38:46.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.970+0000 7f28babee640 1 -- 172.21.15.160:0/2987352506 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f28a40a1d80 msgr2=0x7f28a40a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:46.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.970+0000 7f28babee640 1 --2- 172.21.15.160:0/2987352506 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f28a40a1d80 0x7f28a40a4240 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f28a80058c0 tx=0x7f28a8005850 comp rx=0 tx=0).stop 2024-03-07T07:38:46.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.970+0000 7f28babee640 1 -- 172.21.15.160:0/2987352506 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28b4107400 msgr2=0x7f28b4100690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:46.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.970+0000 7f28babee640 1 --2- 172.21.15.160:0/2987352506 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28b4107400 0x7f28b4100690 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f28ac02c700 tx=0x7f28ac032f20 comp rx=0 tx=0).stop 2024-03-07T07:38:46.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.970+0000 7f28babee640 1 -- 172.21.15.160:0/2987352506 shutdown_connections 2024-03-07T07:38:46.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.970+0000 7f28babee640 1 --2- 172.21.15.160:0/2987352506 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f28a40a1d80 0x7f28a40a4240 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:46.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.970+0000 7f28babee640 1 --2- 172.21.15.160:0/2987352506 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28b4107400 0x7f28b4100690 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:46.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.970+0000 7f28babee640 1 --2- 172.21.15.160:0/2987352506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28b4104ad0 0x7f28b4100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:46.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.970+0000 7f28babee640 1 --2- 172.21.15.160:0/2987352506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28b41021a0 0x7f28b4101a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:46.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.970+0000 7f28babee640 1 -- 172.21.15.160:0/2987352506 >> 172.21.15.160:0/2987352506 conn(0x7f28b40fbaf0 msgr2=0x7f28b4102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:38:46.972 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.970+0000 7f28babee640 1 -- 172.21.15.160:0/2987352506 shutdown_connections 2024-03-07T07:38:46.972 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:46.970+0000 7f28babee640 1 -- 172.21.15.160:0/2987352506 wait complete. 2024-03-07T07:38:47.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:47 smithi177 ceph-mon[36578]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:47.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:47 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/2987352506' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:38:47.607 INFO:journalctl@ceph.mds.e.smithi177.stdout:Mar 07 07:38:47 smithi177 systemd[1]: Starting Ceph mds.e for 826f6582-dc54-11ee-95c6-87774f69a715... 2024-03-07T07:38:47.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:47 smithi160 ceph-mon[31963]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:47.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:47 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/2987352506' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:38:47.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:47 smithi178 ceph-mon[36650]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:47.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:47 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/2987352506' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:38:48.025 INFO:journalctl@ceph.mds.e.smithi177.stdout:Mar 07 07:38:47 smithi177 podman[66376]: 2024-03-07 07:38:47.669260411 +0000 UTC m=+0.017353691 image pull e362cd7cebfee51077fb90b7f01895811140c9a4272edd83ad9910b05e3a46f6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2 2024-03-07T07:38:48.025 INFO:journalctl@ceph.mds.e.smithi177.stdout:Mar 07 07:38:47 smithi177 podman[66376]: 2024-03-07 07:38:47.781114121 +0000 UTC m=+0.129207401 container create f8fb2c249ecdc7ccd4ad9c35f83d050e5456abd2edb55de0b22e777d3f3bfe0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-e, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-8b896a9, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, ceph=True, GIT_CLEAN=True) 2024-03-07T07:38:48.026 INFO:journalctl@ceph.mds.e.smithi177.stdout:Mar 07 07:38:48 smithi177 podman[66376]: 2024-03-07 07:38:48.000499843 +0000 UTC m=+0.348593141 container init f8fb2c249ecdc7ccd4ad9c35f83d050e5456abd2edb55de0b22e777d3f3bfe0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-e, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, ceph=True, RELEASE=wip-lusov-qdb-8b896a9, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-03-07T07:38:48.026 INFO:journalctl@ceph.mds.e.smithi177.stdout:Mar 07 07:38:48 smithi177 podman[66376]: 2024-03-07 07:38:48.003223526 +0000 UTC m=+0.351316801 container start f8fb2c249ecdc7ccd4ad9c35f83d050e5456abd2edb55de0b22e777d3f3bfe0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-e, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b) 2024-03-07T07:38:48.357 INFO:journalctl@ceph.mds.e.smithi177.stdout:Mar 07 07:38:48 smithi177 ceph-mds[66395]: -- 172.21.15.177:0/507172807 <== mon.2 v2:172.21.15.177:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d6a47d9a40 con 0x55d6a47acc00 2024-03-07T07:38:48.357 INFO:journalctl@ceph.mds.e.smithi177.stdout:Mar 07 07:38:48 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-e[66391]: starting mds.e at 2024-03-07T07:38:48.357 INFO:journalctl@ceph.mds.e.smithi177.stdout:Mar 07 07:38:48 smithi177 bash[66376]: f8fb2c249ecdc7ccd4ad9c35f83d050e5456abd2edb55de0b22e777d3f3bfe0f 2024-03-07T07:38:48.357 INFO:journalctl@ceph.mds.e.smithi177.stdout:Mar 07 07:38:48 smithi177 systemd[1]: Started Ceph mds.e for 826f6582-dc54-11ee-95c6-87774f69a715. 2024-03-07T07:38:48.983 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T07:38:49.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.097+0000 7fdf737a7640 1 -- 172.21.15.160:0/3956112978 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf6c104a30 msgr2=0x7fdf6c106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:49.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.097+0000 7fdf737a7640 1 --2- 172.21.15.160:0/3956112978 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf6c104a30 0x7fdf6c106e20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c009900 tx=0x7fdf5c02bc30 comp rx=0 tx=0).stop 2024-03-07T07:38:49.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.097+0000 7fdf737a7640 1 -- 172.21.15.160:0/3956112978 shutdown_connections 2024-03-07T07:38:49.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.097+0000 7fdf737a7640 1 --2- 172.21.15.160:0/3956112978 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf6c107360 0x7fdf6c1097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:49.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.097+0000 7fdf737a7640 1 --2- 172.21.15.160:0/3956112978 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf6c104a30 0x7fdf6c106e20 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:49.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.097+0000 7fdf737a7640 1 --2- 172.21.15.160:0/3956112978 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf6c102100 0x7fdf6c1044f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:49.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.097+0000 7fdf737a7640 1 -- 172.21.15.160:0/3956112978 >> 172.21.15.160:0/3956112978 conn(0x7fdf6c0fbaf0 msgr2=0x7fdf6c0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:38:49.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.097+0000 7fdf737a7640 1 -- 172.21.15.160:0/3956112978 shutdown_connections 2024-03-07T07:38:49.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.098+0000 7fdf737a7640 1 -- 172.21.15.160:0/3956112978 wait complete. 2024-03-07T07:38:49.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.098+0000 7fdf737a7640 1 Processor -- start 2024-03-07T07:38:49.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.098+0000 7fdf737a7640 1 -- start start 2024-03-07T07:38:49.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.098+0000 7fdf737a7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf6c102100 0x7fdf6c1979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:49.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.098+0000 7fdf737a7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf6c104a30 0x7fdf6c197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:49.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.098+0000 7fdf737a7640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf6c107360 0x7fdf6c19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:49.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.098+0000 7fdf737a7640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6c10d360 con 0x7fdf6c107360 2024-03-07T07:38:49.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.098+0000 7fdf737a7640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6c10d1e0 con 0x7fdf6c102100 2024-03-07T07:38:49.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.098+0000 7fdf737a7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6c10d4e0 con 0x7fdf6c104a30 2024-03-07T07:38:49.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.098+0000 7fdf70d42640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf6c104a30 0x7fdf6c197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:49.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.098+0000 7fdf71d44640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf6c107360 0x7fdf6c19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:49.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.098+0000 7fdf70d42640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf6c104a30 0x7fdf6c197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.160:53784/0 (socket says 172.21.15.160:53784) 2024-03-07T07:38:49.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.098+0000 7fdf70d42640 1 -- 172.21.15.160:0/102139839 learned_addr learned my addr 172.21.15.160:0/102139839 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:38:49.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.099+0000 7fdf71543640 1 --2- 172.21.15.160:0/102139839 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf6c102100 0x7fdf6c1979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:49.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.099+0000 7fdf71d44640 1 -- 172.21.15.160:0/102139839 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf6c104a30 msgr2=0x7fdf6c197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:49.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.099+0000 7fdf71d44640 1 --2- 172.21.15.160:0/102139839 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf6c104a30 0x7fdf6c197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:49.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.099+0000 7fdf71d44640 1 -- 172.21.15.160:0/102139839 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf6c102100 msgr2=0x7fdf6c1979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:49.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.099+0000 7fdf71d44640 1 --2- 172.21.15.160:0/102139839 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf6c102100 0x7fdf6c1979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:49.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.099+0000 7fdf71d44640 1 -- 172.21.15.160:0/102139839 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf6c19f5f0 con 0x7fdf6c107360 2024-03-07T07:38:49.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.099+0000 7fdf70d42640 1 --2- 172.21.15.160:0/102139839 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf6c104a30 0x7fdf6c197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:38:49.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.099+0000 7fdf71d44640 1 --2- 172.21.15.160:0/102139839 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf6c107360 0x7fdf6c19ef80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdf6400ca30 tx=0x7fdf6400cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:38:49.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.099+0000 7fdf627fc640 1 -- 172.21.15.160:0/102139839 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf64007d90 con 0x7fdf6c107360 2024-03-07T07:38:49.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.099+0000 7fdf627fc640 1 -- 172.21.15.160:0/102139839 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf64005020 con 0x7fdf6c107360 2024-03-07T07:38:49.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.099+0000 7fdf737a7640 1 -- 172.21.15.160:0/102139839 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf6c19f8e0 con 0x7fdf6c107360 2024-03-07T07:38:49.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.099+0000 7fdf627fc640 1 -- 172.21.15.160:0/102139839 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6400f780 con 0x7fdf6c107360 2024-03-07T07:38:49.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.099+0000 7fdf737a7640 1 -- 172.21.15.160:0/102139839 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf6c19fdb0 con 0x7fdf6c107360 2024-03-07T07:38:49.104 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.100+0000 7fdf737a7640 1 -- 172.21.15.160:0/102139839 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf6c0fd5e0 con 0x7fdf6c107360 2024-03-07T07:38:49.104 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.101+0000 7fdf627fc640 1 -- 172.21.15.160:0/102139839 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7fdf640051d0 con 0x7fdf6c107360 2024-03-07T07:38:49.104 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.102+0000 7fdf627fc640 1 --2- 172.21.15.160:0/102139839 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fdf580a1d80 0x7fdf580a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:49.104 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.102+0000 7fdf71543640 1 --2- 172.21.15.160:0/102139839 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fdf580a1d80 0x7fdf580a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:49.104 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.102+0000 7fdf627fc640 1 -- 172.21.15.160:0/102139839 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdf640cdb10 con 0x7fdf6c107360 2024-03-07T07:38:49.104 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.102+0000 7fdf71543640 1 --2- 172.21.15.160:0/102139839 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fdf580a1d80 0x7fdf580a4240 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdf680058c0 tx=0x7fdf6800a250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:38:49.106 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.106+0000 7fdf627fc640 1 -- 172.21.15.160:0/102139839 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fdf64014070 con 0x7fdf6c107360 2024-03-07T07:38:49.319 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.319+0000 7fdf737a7640 1 -- 172.21.15.160:0/102139839 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf6c02d030 con 0x7fdf6c107360 2024-03-07T07:38:49.320 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.319+0000 7fdf627fc640 1 -- 172.21.15.160:0/102139839 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4498 (secure 0 0 0) 0x7fdf6409ad90 con 0x7fdf6c107360 2024-03-07T07:38:49.320 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:38:49.320 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/2615961083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2615961083},{"type":"v1","addr":"172.21.15.160:6835","nonce":2615961083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1759006033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1759006033},{"type":"v1","addr":"172.21.15.160:6837","nonce":1759006033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1457364761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1457364761},{"type":"v1","addr":"172.21.15.177:6833","nonce":1457364761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/170428292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":170428292},{"type":"v1","addr":"172.21.15.177:6835","nonce":170428292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/645365846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":645365846},{"type":"v1","addr":"172.21.15.178:6833","nonce":645365846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-07T07:38:49.320 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 6 2024-03-07T07:38:49.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.321+0000 7fdf737a7640 1 -- 172.21.15.160:0/102139839 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fdf580a1d80 msgr2=0x7fdf580a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:49.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.321+0000 7fdf737a7640 1 --2- 172.21.15.160:0/102139839 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fdf580a1d80 0x7fdf580a4240 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdf680058c0 tx=0x7fdf6800a250 comp rx=0 tx=0).stop 2024-03-07T07:38:49.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.321+0000 7fdf737a7640 1 -- 172.21.15.160:0/102139839 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf6c107360 msgr2=0x7fdf6c19ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:49.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.321+0000 7fdf737a7640 1 --2- 172.21.15.160:0/102139839 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf6c107360 0x7fdf6c19ef80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdf6400ca30 tx=0x7fdf6400cef0 comp rx=0 tx=0).stop 2024-03-07T07:38:49.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.321+0000 7fdf737a7640 1 -- 172.21.15.160:0/102139839 shutdown_connections 2024-03-07T07:38:49.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.321+0000 7fdf737a7640 1 --2- 172.21.15.160:0/102139839 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fdf580a1d80 0x7fdf580a4240 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:49.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.321+0000 7fdf737a7640 1 --2- 172.21.15.160:0/102139839 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf6c107360 0x7fdf6c19ef80 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:49.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.321+0000 7fdf737a7640 1 --2- 172.21.15.160:0/102139839 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf6c104a30 0x7fdf6c197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:49.323 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.321+0000 7fdf737a7640 1 --2- 172.21.15.160:0/102139839 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf6c102100 0x7fdf6c1979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:49.323 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.321+0000 7fdf737a7640 1 -- 172.21.15.160:0/102139839 >> 172.21.15.160:0/102139839 conn(0x7fdf6c0fbaf0 msgr2=0x7fdf6c102a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:38:49.323 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.321+0000 7fdf737a7640 1 -- 172.21.15.160:0/102139839 shutdown_connections 2024-03-07T07:38:49.323 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:49.321+0000 7fdf737a7640 1 -- 172.21.15.160:0/102139839 wait complete. 2024-03-07T07:38:49.332 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:49 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:49.332 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:49 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:49.332 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:49 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:49.332 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:49 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-07T07:38:49.332 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:49 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-07T07:38:49.332 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:49 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:38:49.332 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:49 smithi160 ceph-mon[31963]: Deploying daemon mds.f on smithi178 2024-03-07T07:38:49.333 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:49 smithi160 ceph-mon[31963]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:49.333 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:49 smithi160 ceph-mon[31963]: mds.? [v2:172.21.15.177:6834/170428292,v1:172.21.15.177:6835/170428292] up:boot 2024-03-07T07:38:49.333 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:49 smithi160 ceph-mon[31963]: fsmap 5 up:standby 2024-03-07T07:38:49.333 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:49 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-07T07:38:49.376 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:49 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:49.376 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:49 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:49.376 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:49 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:49.376 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:49 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-07T07:38:49.376 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:49 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-07T07:38:49.376 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:49 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:38:49.376 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:49 smithi178 ceph-mon[36650]: Deploying daemon mds.f on smithi178 2024-03-07T07:38:49.376 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:49 smithi178 ceph-mon[36650]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:49.377 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:49 smithi178 ceph-mon[36650]: mds.? [v2:172.21.15.177:6834/170428292,v1:172.21.15.177:6835/170428292] up:boot 2024-03-07T07:38:49.377 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:49 smithi178 ceph-mon[36650]: fsmap 5 up:standby 2024-03-07T07:38:49.377 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:49 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-07T07:38:49.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:49 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:49.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:49 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:49.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:49 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:49.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:49 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-07T07:38:49.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:49 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-07T07:38:49.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:49 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:38:49.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:49 smithi177 ceph-mon[36578]: Deploying daemon mds.f on smithi178 2024-03-07T07:38:49.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:49 smithi177 ceph-mon[36578]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:49.608 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:49 smithi177 ceph-mon[36578]: mds.? [v2:172.21.15.177:6834/170428292,v1:172.21.15.177:6835/170428292] up:boot 2024-03-07T07:38:49.608 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:49 smithi177 ceph-mon[36578]: fsmap 5 up:standby 2024-03-07T07:38:49.608 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:49 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-07T07:38:50.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:50 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/102139839' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:38:50.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:50 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/102139839' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:38:50.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:50 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/102139839' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:38:51.220 INFO:journalctl@ceph.mds.f.smithi178.stdout:Mar 07 07:38:50 smithi178 systemd[1]: Started Ceph mds.f for 826f6582-dc54-11ee-95c6-87774f69a715. 2024-03-07T07:38:51.335 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T07:38:51.451 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.448+0000 7fa98f274640 1 -- 172.21.15.160:0/935239222 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa988100580 msgr2=0x7fa9881009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:51.452 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.448+0000 7fa98f274640 1 --2- 172.21.15.160:0/935239222 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa988100580 0x7fa9881009a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa978009900 tx=0x7fa97802bc30 comp rx=0 tx=0).stop 2024-03-07T07:38:51.452 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.448+0000 7fa98f274640 1 -- 172.21.15.160:0/935239222 shutdown_connections 2024-03-07T07:38:51.452 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.448+0000 7fa98f274640 1 --2- 172.21.15.160:0/935239222 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa988102130 0x7fa9881089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:51.452 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.448+0000 7fa98f274640 1 --2- 172.21.15.160:0/935239222 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa988101770 0x7fa988101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:51.452 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.448+0000 7fa98f274640 1 --2- 172.21.15.160:0/935239222 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa988100580 0x7fa9881009a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:51.452 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.448+0000 7fa98f274640 1 -- 172.21.15.160:0/935239222 >> 172.21.15.160:0/935239222 conn(0x7fa9880fbaf0 msgr2=0x7fa9880fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:38:51.453 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.448+0000 7fa98f274640 1 -- 172.21.15.160:0/935239222 shutdown_connections 2024-03-07T07:38:51.453 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.448+0000 7fa98f274640 1 -- 172.21.15.160:0/935239222 wait complete. 2024-03-07T07:38:51.453 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.449+0000 7fa98f274640 1 Processor -- start 2024-03-07T07:38:51.453 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.449+0000 7fa98f274640 1 -- start start 2024-03-07T07:38:51.453 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.449+0000 7fa98f274640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa988100580 0x7fa988197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:51.453 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.449+0000 7fa98f274640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa988101770 0x7fa988197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:51.453 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.449+0000 7fa98f274640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa988102130 0x7fa98819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:51.453 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.449+0000 7fa98f274640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98810c1e0 con 0x7fa988101770 2024-03-07T07:38:51.453 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.449+0000 7fa98f274640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98810c060 con 0x7fa988100580 2024-03-07T07:38:51.453 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.449+0000 7fa98f274640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98810c360 con 0x7fa988102130 2024-03-07T07:38:51.453 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.449+0000 7fa98c80f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa988101770 0x7fa988197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:51.453 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.449+0000 7fa98c80f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa988101770 0x7fa988197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:48146/0 (socket says 172.21.15.160:48146) 2024-03-07T07:38:51.453 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.449+0000 7fa98c80f640 1 -- 172.21.15.160:0/2437630758 learned_addr learned my addr 172.21.15.160:0/2437630758 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:38:51.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.449+0000 7fa98d811640 1 --2- 172.21.15.160:0/2437630758 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa988102130 0x7fa98819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:51.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.449+0000 7fa98d010640 1 --2- 172.21.15.160:0/2437630758 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa988100580 0x7fa988197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:51.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.449+0000 7fa98c80f640 1 -- 172.21.15.160:0/2437630758 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa988102130 msgr2=0x7fa98819efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:51.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.449+0000 7fa98c80f640 1 --2- 172.21.15.160:0/2437630758 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa988102130 0x7fa98819efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:51.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.449+0000 7fa98c80f640 1 -- 172.21.15.160:0/2437630758 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa988100580 msgr2=0x7fa988197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:51.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.449+0000 7fa98c80f640 1 --2- 172.21.15.160:0/2437630758 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa988100580 0x7fa988197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:51.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.449+0000 7fa98c80f640 1 -- 172.21.15.160:0/2437630758 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa98819f620 con 0x7fa988101770 2024-03-07T07:38:51.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.449+0000 7fa98d010640 1 --2- 172.21.15.160:0/2437630758 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa988100580 0x7fa988197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:38:51.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.449+0000 7fa98c80f640 1 --2- 172.21.15.160:0/2437630758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa988101770 0x7fa988197f40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa98400e9f0 tx=0x7fa98400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:38:51.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.450+0000 7fa97e7fc640 1 -- 172.21.15.160:0/2437630758 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98400cd80 con 0x7fa988101770 2024-03-07T07:38:51.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.450+0000 7fa97e7fc640 1 -- 172.21.15.160:0/2437630758 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa984010910 con 0x7fa988101770 2024-03-07T07:38:51.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.450+0000 7fa97e7fc640 1 -- 172.21.15.160:0/2437630758 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa984018990 con 0x7fa988101770 2024-03-07T07:38:51.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.450+0000 7fa98f274640 1 -- 172.21.15.160:0/2437630758 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa98819f910 con 0x7fa988101770 2024-03-07T07:38:51.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.450+0000 7fa98f274640 1 -- 172.21.15.160:0/2437630758 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa98819feb0 con 0x7fa988101770 2024-03-07T07:38:51.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.451+0000 7fa98f274640 1 -- 172.21.15.160:0/2437630758 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9881a00e0 con 0x7fa988101770 2024-03-07T07:38:51.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.452+0000 7fa97e7fc640 1 -- 172.21.15.160:0/2437630758 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7fa984010ac0 con 0x7fa988101770 2024-03-07T07:38:51.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.453+0000 7fa97e7fc640 1 --2- 172.21.15.160:0/2437630758 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa9740a1e00 0x7fa9740a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:51.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.453+0000 7fa98d010640 1 --2- 172.21.15.160:0/2437630758 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa9740a1e00 0x7fa9740a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:51.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.453+0000 7fa97e7fc640 1 -- 172.21.15.160:0/2437630758 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa9840cdc60 con 0x7fa988101770 2024-03-07T07:38:51.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.453+0000 7fa98d010640 1 --2- 172.21.15.160:0/2437630758 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa9740a1e00 0x7fa9740a42c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa97802c7b0 tx=0x7fa978005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:38:51.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.457+0000 7fa97e7fc640 1 -- 172.21.15.160:0/2437630758 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa98409b530 con 0x7fa988101770 2024-03-07T07:38:51.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:51 smithi178 ceph-mon[36650]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:51.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:51 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:51.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:51 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:51.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:51 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:51.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:51 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-07T07:38:51.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:51 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-07T07:38:51.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:51 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:38:51.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:51 smithi177 ceph-mon[36578]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:51.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:51 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:51.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:51 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:51.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:51 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:51.608 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:51 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-07T07:38:51.608 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:51 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-07T07:38:51.608 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:51 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:38:51.669 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.669+0000 7fa98f274640 1 -- 172.21.15.160:0/2437630758 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa9881a02b0 con 0x7fa988101770 2024-03-07T07:38:51.669 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.669+0000 7fa97e7fc640 1 -- 172.21.15.160:0/2437630758 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5289 (secure 0 0 0) 0x7fa98409ae30 con 0x7fa988101770 2024-03-07T07:38:51.670 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:38:51.670 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/2615961083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2615961083},{"type":"v1","addr":"172.21.15.160:6835","nonce":2615961083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1759006033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1759006033},{"type":"v1","addr":"172.21.15.160:6837","nonce":1759006033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1457364761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1457364761},{"type":"v1","addr":"172.21.15.177:6833","nonce":1457364761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/170428292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":170428292},{"type":"v1","addr":"172.21.15.177:6835","nonce":170428292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/645365846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":645365846},{"type":"v1","addr":"172.21.15.178:6833","nonce":645365846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/647840547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":647840547},{"type":"v1","addr":"172.21.15.178:6835","nonce":647840547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2024-03-07T07:38:51.670 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 7 2024-03-07T07:38:51.671 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.671+0000 7fa98f274640 1 -- 172.21.15.160:0/2437630758 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa9740a1e00 msgr2=0x7fa9740a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:51.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.671+0000 7fa98f274640 1 --2- 172.21.15.160:0/2437630758 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa9740a1e00 0x7fa9740a42c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa97802c7b0 tx=0x7fa978005fd0 comp rx=0 tx=0).stop 2024-03-07T07:38:51.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.671+0000 7fa98f274640 1 -- 172.21.15.160:0/2437630758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa988101770 msgr2=0x7fa988197f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:51.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.671+0000 7fa98f274640 1 --2- 172.21.15.160:0/2437630758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa988101770 0x7fa988197f40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa98400e9f0 tx=0x7fa98400eeb0 comp rx=0 tx=0).stop 2024-03-07T07:38:51.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.671+0000 7fa98f274640 1 -- 172.21.15.160:0/2437630758 shutdown_connections 2024-03-07T07:38:51.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.671+0000 7fa98f274640 1 --2- 172.21.15.160:0/2437630758 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa9740a1e00 0x7fa9740a42c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:51.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.671+0000 7fa98f274640 1 --2- 172.21.15.160:0/2437630758 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa988102130 0x7fa98819efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:51.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.671+0000 7fa98f274640 1 --2- 172.21.15.160:0/2437630758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa988101770 0x7fa988197f40 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:51.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.671+0000 7fa98f274640 1 --2- 172.21.15.160:0/2437630758 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa988100580 0x7fa988197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:51.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.671+0000 7fa98f274640 1 -- 172.21.15.160:0/2437630758 >> 172.21.15.160:0/2437630758 conn(0x7fa9880fbaf0 msgr2=0x7fa9880fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:38:51.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.671+0000 7fa98f274640 1 -- 172.21.15.160:0/2437630758 shutdown_connections 2024-03-07T07:38:51.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:51.671+0000 7fa98f274640 1 -- 172.21.15.160:0/2437630758 wait complete. 2024-03-07T07:38:51.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:51 smithi160 ceph-mon[31963]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:51.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:51 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:51.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:51 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:51.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:51 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:51.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:51 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-07T07:38:51.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:51 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-07T07:38:51.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:51 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:38:52.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:52 smithi178 ceph-mon[36650]: Deploying daemon mds.c on smithi178 2024-03-07T07:38:52.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:52 smithi178 ceph-mon[36650]: mds.? [v2:172.21.15.178:6834/647840547,v1:172.21.15.178:6835/647840547] up:boot 2024-03-07T07:38:52.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:52 smithi178 ceph-mon[36650]: fsmap 6 up:standby 2024-03-07T07:38:52.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:52 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-07T07:38:52.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:52 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/2437630758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:38:52.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:52 smithi177 ceph-mon[36578]: Deploying daemon mds.c on smithi178 2024-03-07T07:38:52.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:52 smithi177 ceph-mon[36578]: mds.? [v2:172.21.15.178:6834/647840547,v1:172.21.15.178:6835/647840547] up:boot 2024-03-07T07:38:52.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:52 smithi177 ceph-mon[36578]: fsmap 6 up:standby 2024-03-07T07:38:52.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:52 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-07T07:38:52.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:52 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/2437630758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:38:52.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:52 smithi160 ceph-mon[31963]: Deploying daemon mds.c on smithi178 2024-03-07T07:38:52.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:52 smithi160 ceph-mon[31963]: mds.? [v2:172.21.15.178:6834/647840547,v1:172.21.15.178:6835/647840547] up:boot 2024-03-07T07:38:52.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:52 smithi160 ceph-mon[31963]: fsmap 6 up:standby 2024-03-07T07:38:52.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:52 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-07T07:38:52.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:52 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/2437630758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:38:53.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:53 smithi178 ceph-mon[36650]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:53.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:53 smithi177 ceph-mon[36578]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:53.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:53 smithi160 ceph-mon[31963]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:53.684 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T07:38:53.799 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:53.797+0000 7f2e8fffc640 1 -- 172.21.15.160:0/2153029176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e88101760 msgr2=0x7f2e88101be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:53.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:53.797+0000 7f2e8fffc640 1 --2- 172.21.15.160:0/2153029176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e88101760 0x7f2e88101be0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2e84009900 tx=0x7f2e8402bc30 comp rx=0 tx=0).stop 2024-03-07T07:38:53.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:53.797+0000 7f2e8fffc640 1 -- 172.21.15.160:0/2153029176 shutdown_connections 2024-03-07T07:38:53.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:53.797+0000 7f2e8fffc640 1 --2- 172.21.15.160:0/2153029176 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e88102120 0x7f2e881089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:53.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:53.797+0000 7f2e8fffc640 1 --2- 172.21.15.160:0/2153029176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e88101760 0x7f2e88101be0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:53.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:53.797+0000 7f2e8fffc640 1 --2- 172.21.15.160:0/2153029176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e88100570 0x7f2e88100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:53.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:53.797+0000 7f2e8fffc640 1 -- 172.21.15.160:0/2153029176 >> 172.21.15.160:0/2153029176 conn(0x7f2e880fbb00 msgr2=0x7f2e880fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:38:53.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:53.797+0000 7f2e8fffc640 1 -- 172.21.15.160:0/2153029176 shutdown_connections 2024-03-07T07:38:53.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:53.797+0000 7f2e8fffc640 1 -- 172.21.15.160:0/2153029176 wait complete. 2024-03-07T07:38:53.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:53.797+0000 7f2e8fffc640 1 Processor -- start 2024-03-07T07:38:53.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:53.797+0000 7f2e8fffc640 1 -- start start 2024-03-07T07:38:53.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:53.798+0000 7f2e8fffc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e88100570 0x7f2e88197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:53.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:53.798+0000 7f2e8fffc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e88101760 0x7f2e88197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:53.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:53.798+0000 7f2e8fffc640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e88102120 0x7f2e8819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:53.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:53.798+0000 7f2e8fffc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8810c200 con 0x7f2e88100570 2024-03-07T07:38:53.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:53.798+0000 7f2e8fffc640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8810c080 con 0x7f2e88102120 2024-03-07T07:38:53.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:53.798+0000 7f2e8fffc640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8810c380 con 0x7f2e88101760 2024-03-07T07:38:53.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:53.798+0000 7f2e8dd98640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e88100570 0x7f2e88197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:53.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:53.798+0000 7f2e8dd98640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e88100570 0x7f2e88197960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:48160/0 (socket says 172.21.15.160:48160) 2024-03-07T07:38:53.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:53.798+0000 7f2e8dd98640 1 -- 172.21.15.160:0/4064557483 learned_addr learned my addr 172.21.15.160:0/4064557483 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:38:53.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:53.798+0000 7f2e8d597640 1 --2- 172.21.15.160:0/4064557483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e88101760 0x7f2e88197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:53.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:53.798+0000 7f2e8e599640 1 --2- 172.21.15.160:0/4064557483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e88102120 0x7f2e8819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:53.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:53.798+0000 7f2e8dd98640 1 -- 172.21.15.160:0/4064557483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e88101760 msgr2=0x7f2e88197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:53.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:53.798+0000 7f2e8dd98640 1 --2- 172.21.15.160:0/4064557483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e88101760 0x7f2e88197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:53.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:53.798+0000 7f2e8dd98640 1 -- 172.21.15.160:0/4064557483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e88102120 msgr2=0x7f2e8819ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:53.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:53.798+0000 7f2e8dd98640 1 --2- 172.21.15.160:0/4064557483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e88102120 0x7f2e8819ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:53.802 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:53.798+0000 7f2e8dd98640 1 -- 172.21.15.160:0/4064557483 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e8819f580 con 0x7f2e88100570 2024-03-07T07:38:53.802 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:53.798+0000 7f2e8dd98640 1 --2- 172.21.15.160:0/4064557483 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e88100570 0x7f2e88197960 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2e7800d950 tx=0x7f2e7800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:38:53.802 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:53.798+0000 7f2e7effd640 1 -- 172.21.15.160:0/4064557483 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e78017020 con 0x7f2e88100570 2024-03-07T07:38:53.802 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:53.798+0000 7f2e8fffc640 1 -- 172.21.15.160:0/4064557483 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e8819f870 con 0x7f2e88100570 2024-03-07T07:38:53.802 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:53.798+0000 7f2e7effd640 1 -- 172.21.15.160:0/4064557483 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2e78009c30 con 0x7f2e88100570 2024-03-07T07:38:53.802 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:53.798+0000 7f2e7effd640 1 -- 172.21.15.160:0/4064557483 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7800f740 con 0x7f2e88100570 2024-03-07T07:38:53.802 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:53.798+0000 7f2e8fffc640 1 -- 172.21.15.160:0/4064557483 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e8819fd90 con 0x7f2e88100570 2024-03-07T07:38:53.802 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:53.799+0000 7f2e8fffc640 1 -- 172.21.15.160:0/4064557483 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e8819fff0 con 0x7f2e88100570 2024-03-07T07:38:53.803 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:53.800+0000 7f2e7effd640 1 -- 172.21.15.160:0/4064557483 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f2e7801a020 con 0x7f2e88100570 2024-03-07T07:38:53.803 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:53.801+0000 7f2e7effd640 1 --2- 172.21.15.160:0/4064557483 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2e740a1d80 0x7f2e740a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:53.803 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:53.801+0000 7f2e8d597640 1 --2- 172.21.15.160:0/4064557483 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2e740a1d80 0x7f2e740a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:53.803 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:53.801+0000 7f2e7effd640 1 -- 172.21.15.160:0/4064557483 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2e780cd5c0 con 0x7f2e88100570 2024-03-07T07:38:53.803 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:53.801+0000 7f2e8d597640 1 --2- 172.21.15.160:0/4064557483 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2e740a1d80 0x7f2e740a4240 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2e8402c7b0 tx=0x7f2e84005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:38:53.805 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:53.805+0000 7f2e7effd640 1 -- 172.21.15.160:0/4064557483 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f2e78021070 con 0x7f2e88100570 2024-03-07T07:38:54.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:54.021+0000 7f2e8fffc640 1 -- 172.21.15.160:0/4064557483 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e881a01c0 con 0x7f2e88100570 2024-03-07T07:38:54.022 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:54.021+0000 7f2e7effd640 1 -- 172.21.15.160:0/4064557483 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5289 (secure 0 0 0) 0x7f2e78009de0 con 0x7f2e88100570 2024-03-07T07:38:54.022 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:38:54.022 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/2615961083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2615961083},{"type":"v1","addr":"172.21.15.160:6835","nonce":2615961083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1759006033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1759006033},{"type":"v1","addr":"172.21.15.160:6837","nonce":1759006033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1457364761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1457364761},{"type":"v1","addr":"172.21.15.177:6833","nonce":1457364761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/170428292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":170428292},{"type":"v1","addr":"172.21.15.177:6835","nonce":170428292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/645365846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":645365846},{"type":"v1","addr":"172.21.15.178:6833","nonce":645365846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/647840547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":647840547},{"type":"v1","addr":"172.21.15.178:6835","nonce":647840547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2024-03-07T07:38:54.022 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 7 2024-03-07T07:38:54.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:54.023+0000 7f2e8fffc640 1 -- 172.21.15.160:0/4064557483 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2e740a1d80 msgr2=0x7f2e740a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:54.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:54.023+0000 7f2e8fffc640 1 --2- 172.21.15.160:0/4064557483 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2e740a1d80 0x7f2e740a4240 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2e8402c7b0 tx=0x7f2e84005fd0 comp rx=0 tx=0).stop 2024-03-07T07:38:54.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:54.023+0000 7f2e8fffc640 1 -- 172.21.15.160:0/4064557483 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e88100570 msgr2=0x7f2e88197960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:54.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:54.023+0000 7f2e8fffc640 1 --2- 172.21.15.160:0/4064557483 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e88100570 0x7f2e88197960 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2e7800d950 tx=0x7f2e7800de10 comp rx=0 tx=0).stop 2024-03-07T07:38:54.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:54.023+0000 7f2e8fffc640 1 -- 172.21.15.160:0/4064557483 shutdown_connections 2024-03-07T07:38:54.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:54.023+0000 7f2e8fffc640 1 --2- 172.21.15.160:0/4064557483 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2e740a1d80 0x7f2e740a4240 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:54.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:54.023+0000 7f2e8fffc640 1 --2- 172.21.15.160:0/4064557483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e88102120 0x7f2e8819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:54.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:54.023+0000 7f2e8fffc640 1 --2- 172.21.15.160:0/4064557483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e88101760 0x7f2e88197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:54.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:54.023+0000 7f2e8fffc640 1 --2- 172.21.15.160:0/4064557483 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e88100570 0x7f2e88197960 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:54.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:54.023+0000 7f2e8fffc640 1 -- 172.21.15.160:0/4064557483 >> 172.21.15.160:0/4064557483 conn(0x7f2e880fbb00 msgr2=0x7f2e88069cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:38:54.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:54.023+0000 7f2e8fffc640 1 -- 172.21.15.160:0/4064557483 shutdown_connections 2024-03-07T07:38:54.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:54.024+0000 7f2e8fffc640 1 -- 172.21.15.160:0/4064557483 wait complete. 2024-03-07T07:38:54.056 INFO:journalctl@ceph.mds.c.smithi178.stdout:Mar 07 07:38:53 smithi178 systemd[1]: Started Ceph mds.c for 826f6582-dc54-11ee-95c6-87774f69a715. 2024-03-07T07:38:55.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:54 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:55.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:54 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:55.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:54 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:55.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:54 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-07T07:38:55.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:54 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-07T07:38:55.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:54 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:38:55.108 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:54 smithi177 ceph-mon[36578]: Deploying daemon mds.h on smithi177 2024-03-07T07:38:55.108 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:54 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/4064557483' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:38:55.108 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:54 smithi177 ceph-mon[36578]: mds.? [v2:172.21.15.178:6836/1247829154,v1:172.21.15.178:6837/1247829154] up:boot 2024-03-07T07:38:55.108 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:54 smithi177 ceph-mon[36578]: fsmap 7 up:standby 2024-03-07T07:38:55.108 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:54 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-07T07:38:55.108 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:54 smithi177 ceph-mon[36578]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:55.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:54 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:55.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:54 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:55.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:54 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:55.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:54 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-07T07:38:55.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:54 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-07T07:38:55.179 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:54 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:38:55.179 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:54 smithi160 ceph-mon[31963]: Deploying daemon mds.h on smithi177 2024-03-07T07:38:55.179 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:54 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/4064557483' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:38:55.179 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:54 smithi160 ceph-mon[31963]: mds.? [v2:172.21.15.178:6836/1247829154,v1:172.21.15.178:6837/1247829154] up:boot 2024-03-07T07:38:55.179 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:54 smithi160 ceph-mon[31963]: fsmap 7 up:standby 2024-03-07T07:38:55.179 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:54 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-07T07:38:55.179 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:54 smithi160 ceph-mon[31963]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:55.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:54 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:55.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:54 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:55.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:54 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:55.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:54 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-07T07:38:55.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:54 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-07T07:38:55.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:54 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:38:55.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:54 smithi178 ceph-mon[36650]: Deploying daemon mds.h on smithi177 2024-03-07T07:38:55.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:54 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/4064557483' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:38:55.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:54 smithi178 ceph-mon[36650]: mds.? [v2:172.21.15.178:6836/1247829154,v1:172.21.15.178:6837/1247829154] up:boot 2024-03-07T07:38:55.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:54 smithi178 ceph-mon[36650]: fsmap 7 up:standby 2024-03-07T07:38:55.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:54 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-07T07:38:55.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:54 smithi178 ceph-mon[36650]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:56.036 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T07:38:56.149 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.149+0000 7f31ad35d640 1 -- 172.21.15.160:0/4208901256 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31a8100580 msgr2=0x7f31a81009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:56.150 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.149+0000 7f31ad35d640 1 --2- 172.21.15.160:0/4208901256 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31a8100580 0x7f31a81009a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f31a0009900 tx=0x7f31a002bc30 comp rx=0 tx=0).stop 2024-03-07T07:38:56.150 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.149+0000 7f31ad35d640 1 -- 172.21.15.160:0/4208901256 shutdown_connections 2024-03-07T07:38:56.150 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.149+0000 7f31ad35d640 1 --2- 172.21.15.160:0/4208901256 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31a8102130 0x7f31a81089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:56.150 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.149+0000 7f31ad35d640 1 --2- 172.21.15.160:0/4208901256 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31a8101770 0x7f31a8101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:56.150 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.149+0000 7f31ad35d640 1 --2- 172.21.15.160:0/4208901256 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31a8100580 0x7f31a81009a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:56.150 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.149+0000 7f31ad35d640 1 -- 172.21.15.160:0/4208901256 >> 172.21.15.160:0/4208901256 conn(0x7f31a80fbb10 msgr2=0x7f31a80fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:38:56.150 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.149+0000 7f31ad35d640 1 -- 172.21.15.160:0/4208901256 shutdown_connections 2024-03-07T07:38:56.150 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.149+0000 7f31ad35d640 1 -- 172.21.15.160:0/4208901256 wait complete. 2024-03-07T07:38:56.150 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.149+0000 7f31ad35d640 1 Processor -- start 2024-03-07T07:38:56.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.150+0000 7f31ad35d640 1 -- start start 2024-03-07T07:38:56.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.150+0000 7f31ad35d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31a8100580 0x7f31a81979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:56.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.150+0000 7f31ad35d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31a8101770 0x7f31a8197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:56.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.150+0000 7f31ad35d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31a8102130 0x7f31a819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:56.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.150+0000 7f31ad35d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a810c210 con 0x7f31a8102130 2024-03-07T07:38:56.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.150+0000 7f31ad35d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a810c090 con 0x7f31a8101770 2024-03-07T07:38:56.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.150+0000 7f31ad35d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a810c390 con 0x7f31a8100580 2024-03-07T07:38:56.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.150+0000 7f31a77fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31a8102130 0x7f31a819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:56.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.150+0000 7f31a77fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31a8102130 0x7f31a819ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:48186/0 (socket says 172.21.15.160:48186) 2024-03-07T07:38:56.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.150+0000 7f31a77fe640 1 -- 172.21.15.160:0/2920887135 learned_addr learned my addr 172.21.15.160:0/2920887135 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:38:56.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.150+0000 7f31a67fc640 1 --2- 172.21.15.160:0/2920887135 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31a8101770 0x7f31a8197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:56.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.150+0000 7f31a6ffd640 1 --2- 172.21.15.160:0/2920887135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31a8100580 0x7f31a81979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:56.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.150+0000 7f31a77fe640 1 -- 172.21.15.160:0/2920887135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31a8100580 msgr2=0x7f31a81979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:56.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.150+0000 7f31a77fe640 1 --2- 172.21.15.160:0/2920887135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31a8100580 0x7f31a81979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:56.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.150+0000 7f31a77fe640 1 -- 172.21.15.160:0/2920887135 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31a8101770 msgr2=0x7f31a8197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:56.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.150+0000 7f31a77fe640 1 --2- 172.21.15.160:0/2920887135 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31a8101770 0x7f31a8197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:56.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.150+0000 7f31a77fe640 1 -- 172.21.15.160:0/2920887135 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a819f5d0 con 0x7f31a8102130 2024-03-07T07:38:56.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.151+0000 7f31a77fe640 1 --2- 172.21.15.160:0/2920887135 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31a8102130 0x7f31a819ef60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f319800e970 tx=0x7f319800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:38:56.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.151+0000 7f318ffff640 1 -- 172.21.15.160:0/2920887135 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319800ccc0 con 0x7f31a8102130 2024-03-07T07:38:56.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.151+0000 7f318ffff640 1 -- 172.21.15.160:0/2920887135 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f319800ce60 con 0x7f31a8102130 2024-03-07T07:38:56.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.151+0000 7f31ad35d640 1 -- 172.21.15.160:0/2920887135 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31a819f8c0 con 0x7f31a8102130 2024-03-07T07:38:56.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.151+0000 7f318ffff640 1 -- 172.21.15.160:0/2920887135 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3198010490 con 0x7f31a8102130 2024-03-07T07:38:56.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.151+0000 7f31ad35d640 1 -- 172.21.15.160:0/2920887135 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31a819fdc0 con 0x7f31a8102130 2024-03-07T07:38:56.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.152+0000 7f31ad35d640 1 -- 172.21.15.160:0/2920887135 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31a81a0080 con 0x7f31a8102130 2024-03-07T07:38:56.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.153+0000 7f318ffff640 1 -- 172.21.15.160:0/2920887135 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f3198005020 con 0x7f31a8102130 2024-03-07T07:38:56.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.153+0000 7f318ffff640 1 --2- 172.21.15.160:0/2920887135 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f31900a1d80 0x7f31900a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:56.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.153+0000 7f31a6ffd640 1 --2- 172.21.15.160:0/2920887135 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f31900a1d80 0x7f31900a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:56.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.153+0000 7f318ffff640 1 -- 172.21.15.160:0/2920887135 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f31980cdd70 con 0x7f31a8102130 2024-03-07T07:38:56.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.154+0000 7f31a6ffd640 1 --2- 172.21.15.160:0/2920887135 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f31900a1d80 0x7f31900a4240 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f31a0000c00 tx=0x7f31a0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:38:56.158 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.158+0000 7f318ffff640 1 -- 172.21.15.160:0/2920887135 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f319809b640 con 0x7f31a8102130 2024-03-07T07:38:56.283 INFO:journalctl@ceph.mds.h.smithi177.stdout:Mar 07 07:38:56 smithi177 systemd[1]: Starting Ceph mds.h for 826f6582-dc54-11ee-95c6-87774f69a715... 2024-03-07T07:38:56.370 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.370+0000 7f31ad35d640 1 -- 172.21.15.160:0/2920887135 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f31a81a0250 con 0x7f31a8102130 2024-03-07T07:38:56.370 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.370+0000 7f318ffff640 1 -- 172.21.15.160:0/2920887135 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6083 (secure 0 0 0) 0x7f319809af40 con 0x7f31a8102130 2024-03-07T07:38:56.371 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:38:56.371 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/2615961083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2615961083},{"type":"v1","addr":"172.21.15.160:6835","nonce":2615961083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1759006033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1759006033},{"type":"v1","addr":"172.21.15.160:6837","nonce":1759006033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1457364761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1457364761},{"type":"v1","addr":"172.21.15.177:6833","nonce":1457364761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/170428292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":170428292},{"type":"v1","addr":"172.21.15.177:6835","nonce":170428292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/645365846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":645365846},{"type":"v1","addr":"172.21.15.178:6833","nonce":645365846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/647840547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":647840547},{"type":"v1","addr":"172.21.15.178:6835","nonce":647840547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1247829154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1247829154},{"type":"v1","addr":"172.21.15.178:6837","nonce":1247829154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-03-07T07:38:56.371 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 8 2024-03-07T07:38:56.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.372+0000 7f31ad35d640 1 -- 172.21.15.160:0/2920887135 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f31900a1d80 msgr2=0x7f31900a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:56.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.372+0000 7f31ad35d640 1 --2- 172.21.15.160:0/2920887135 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f31900a1d80 0x7f31900a4240 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f31a0000c00 tx=0x7f31a0005fd0 comp rx=0 tx=0).stop 2024-03-07T07:38:56.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.372+0000 7f31ad35d640 1 -- 172.21.15.160:0/2920887135 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31a8102130 msgr2=0x7f31a819ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:56.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.372+0000 7f31ad35d640 1 --2- 172.21.15.160:0/2920887135 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31a8102130 0x7f31a819ef60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f319800e970 tx=0x7f319800ed70 comp rx=0 tx=0).stop 2024-03-07T07:38:56.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.372+0000 7f31ad35d640 1 -- 172.21.15.160:0/2920887135 shutdown_connections 2024-03-07T07:38:56.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.372+0000 7f31ad35d640 1 --2- 172.21.15.160:0/2920887135 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f31900a1d80 0x7f31900a4240 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:56.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.372+0000 7f31ad35d640 1 --2- 172.21.15.160:0/2920887135 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31a8102130 0x7f31a819ef60 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:56.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.372+0000 7f31ad35d640 1 --2- 172.21.15.160:0/2920887135 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31a8101770 0x7f31a8197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:56.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.372+0000 7f31ad35d640 1 --2- 172.21.15.160:0/2920887135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31a8100580 0x7f31a81979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:56.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.372+0000 7f31ad35d640 1 -- 172.21.15.160:0/2920887135 >> 172.21.15.160:0/2920887135 conn(0x7f31a80fbb10 msgr2=0x7f31a8069c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:38:56.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.372+0000 7f31ad35d640 1 -- 172.21.15.160:0/2920887135 shutdown_connections 2024-03-07T07:38:56.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:56.372+0000 7f31ad35d640 1 -- 172.21.15.160:0/2920887135 wait complete. 2024-03-07T07:38:56.565 INFO:journalctl@ceph.mds.h.smithi177.stdout:Mar 07 07:38:56 smithi177 podman[66690]: 2024-03-07 07:38:56.185673009 +0000 UTC m=+0.017563772 image pull e362cd7cebfee51077fb90b7f01895811140c9a4272edd83ad9910b05e3a46f6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2 2024-03-07T07:38:56.565 INFO:journalctl@ceph.mds.h.smithi177.stdout:Mar 07 07:38:56 smithi177 podman[66690]: 2024-03-07 07:38:56.314468624 +0000 UTC m=+0.146359373 container create 41cf76184494cc9ec4064b6b59fd0441e86f8e43bcafb45c0677b07ff04a8396 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-h, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222) 2024-03-07T07:38:56.565 INFO:journalctl@ceph.mds.h.smithi177.stdout:Mar 07 07:38:56 smithi177 podman[66690]: 2024-03-07 07:38:56.56524607 +0000 UTC m=+0.397136823 container init 41cf76184494cc9ec4064b6b59fd0441e86f8e43bcafb45c0677b07ff04a8396 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-h, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-8b896a9, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2) 2024-03-07T07:38:56.857 INFO:journalctl@ceph.mds.h.smithi177.stdout:Mar 07 07:38:56 smithi177 podman[66690]: 2024-03-07 07:38:56.567999213 +0000 UTC m=+0.399889965 container start 41cf76184494cc9ec4064b6b59fd0441e86f8e43bcafb45c0677b07ff04a8396 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-h, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b) 2024-03-07T07:38:56.857 INFO:journalctl@ceph.mds.h.smithi177.stdout:Mar 07 07:38:56 smithi177 ceph-mds[66709]: -- 172.21.15.177:0/1154935521 <== mon.2 v2:172.21.15.177:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55dbe4d07a40 con 0x55dbe4cdb000 2024-03-07T07:38:56.857 INFO:journalctl@ceph.mds.h.smithi177.stdout:Mar 07 07:38:56 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-h[66705]: starting mds.h at 2024-03-07T07:38:56.857 INFO:journalctl@ceph.mds.h.smithi177.stdout:Mar 07 07:38:56 smithi177 bash[66690]: 41cf76184494cc9ec4064b6b59fd0441e86f8e43bcafb45c0677b07ff04a8396 2024-03-07T07:38:56.857 INFO:journalctl@ceph.mds.h.smithi177.stdout:Mar 07 07:38:56 smithi177 systemd[1]: Started Ceph mds.h for 826f6582-dc54-11ee-95c6-87774f69a715. 2024-03-07T07:38:57.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:57 smithi177 ceph-mon[36578]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:57.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:57 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/2920887135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:38:57.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:57 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:57.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:57 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:57.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:57 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:57.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:57 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-07T07:38:57.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:57 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-07T07:38:57.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:57 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:38:57.616 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:57 smithi178 ceph-mon[36650]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:57.616 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:57 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/2920887135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:38:57.616 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:57 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:57.617 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:57 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:57.617 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:57 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:57.617 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:57 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-07T07:38:57.617 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:57 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-07T07:38:57.617 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:57 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:38:57.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:57 smithi160 ceph-mon[31963]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:57.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:57 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/2920887135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:38:57.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:57 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:57.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:57 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:57.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:57 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:38:57.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:57 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-07T07:38:57.679 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:57 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-07T07:38:57.679 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:57 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:38:58.385 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T07:38:58.502 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.499+0000 7f6d33add640 1 -- 172.21.15.160:0/3977399488 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d2c104ab0 msgr2=0x7f6d2c106ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:58.502 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.499+0000 7f6d33add640 1 --2- 172.21.15.160:0/3977399488 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d2c104ab0 0x7f6d2c106ea0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c009900 tx=0x7f6d1c02bc30 comp rx=0 tx=0).stop 2024-03-07T07:38:58.502 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.499+0000 7f6d33add640 1 -- 172.21.15.160:0/3977399488 shutdown_connections 2024-03-07T07:38:58.502 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.499+0000 7f6d33add640 1 --2- 172.21.15.160:0/3977399488 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d2c1073e0 0x7f6d2c109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:58.502 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.499+0000 7f6d33add640 1 --2- 172.21.15.160:0/3977399488 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d2c104ab0 0x7f6d2c106ea0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:58.502 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.499+0000 7f6d33add640 1 --2- 172.21.15.160:0/3977399488 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d2c102180 0x7f6d2c104570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:58.502 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.499+0000 7f6d33add640 1 -- 172.21.15.160:0/3977399488 >> 172.21.15.160:0/3977399488 conn(0x7f6d2c0fbb90 msgr2=0x7f6d2c0fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:38:58.502 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.499+0000 7f6d33add640 1 -- 172.21.15.160:0/3977399488 shutdown_connections 2024-03-07T07:38:58.502 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.499+0000 7f6d33add640 1 -- 172.21.15.160:0/3977399488 wait complete. 2024-03-07T07:38:58.502 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.499+0000 7f6d33add640 1 Processor -- start 2024-03-07T07:38:58.502 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.499+0000 7f6d33add640 1 -- start start 2024-03-07T07:38:58.502 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.500+0000 7f6d33add640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d2c102180 0x7f6d2c101af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:58.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.500+0000 7f6d33add640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d2c104ab0 0x7f6d2c100130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:58.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.500+0000 7f6d33add640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d2c1073e0 0x7f6d2c100670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:58.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.500+0000 7f6d33add640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d2c10d2f0 con 0x7f6d2c104ab0 2024-03-07T07:38:58.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.500+0000 7f6d33add640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d2c10d170 con 0x7f6d2c1073e0 2024-03-07T07:38:58.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.500+0000 7f6d33add640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d2c10d470 con 0x7f6d2c102180 2024-03-07T07:38:58.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.500+0000 7f6d31078640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d2c104ab0 0x7f6d2c100130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:58.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.500+0000 7f6d31078640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d2c104ab0 0x7f6d2c100130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:48208/0 (socket says 172.21.15.160:48208) 2024-03-07T07:38:58.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.500+0000 7f6d31078640 1 -- 172.21.15.160:0/1960930585 learned_addr learned my addr 172.21.15.160:0/1960930585 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:38:58.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.500+0000 7f6d31879640 1 --2- 172.21.15.160:0/1960930585 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d2c102180 0x7f6d2c101af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:58.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.500+0000 7f6d3207a640 1 --2- 172.21.15.160:0/1960930585 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d2c1073e0 0x7f6d2c100670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:58.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.500+0000 7f6d31078640 1 -- 172.21.15.160:0/1960930585 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d2c102180 msgr2=0x7f6d2c101af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:58.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.500+0000 7f6d31078640 1 --2- 172.21.15.160:0/1960930585 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d2c102180 0x7f6d2c101af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:58.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.500+0000 7f6d31078640 1 -- 172.21.15.160:0/1960930585 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d2c1073e0 msgr2=0x7f6d2c100670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:58.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.500+0000 7f6d31078640 1 --2- 172.21.15.160:0/1960930585 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d2c1073e0 0x7f6d2c100670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:58.504 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.500+0000 7f6d31078640 1 -- 172.21.15.160:0/1960930585 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d2c100f00 con 0x7f6d2c104ab0 2024-03-07T07:38:58.504 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.500+0000 7f6d31078640 1 --2- 172.21.15.160:0/1960930585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d2c104ab0 0x7f6d2c100130 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c000c00 tx=0x7f6d1c005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:38:58.504 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.500+0000 7f6d22ffd640 1 -- 172.21.15.160:0/1960930585 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c041070 con 0x7f6d2c104ab0 2024-03-07T07:38:58.504 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.500+0000 7f6d22ffd640 1 -- 172.21.15.160:0/1960930585 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6d1c0044d0 con 0x7f6d2c104ab0 2024-03-07T07:38:58.504 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.500+0000 7f6d33add640 1 -- 172.21.15.160:0/1960930585 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d2c1a1a60 con 0x7f6d2c104ab0 2024-03-07T07:38:58.504 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.500+0000 7f6d22ffd640 1 -- 172.21.15.160:0/1960930585 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c034440 con 0x7f6d2c104ab0 2024-03-07T07:38:58.504 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.500+0000 7f6d33add640 1 -- 172.21.15.160:0/1960930585 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d2c1a1fb0 con 0x7f6d2c104ab0 2024-03-07T07:38:58.504 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.501+0000 7f6d33add640 1 -- 172.21.15.160:0/1960930585 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d2c0fd680 con 0x7f6d2c104ab0 2024-03-07T07:38:58.505 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.502+0000 7f6d22ffd640 1 -- 172.21.15.160:0/1960930585 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f6d1c044070 con 0x7f6d2c104ab0 2024-03-07T07:38:58.505 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.503+0000 7f6d22ffd640 1 --2- 172.21.15.160:0/1960930585 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f6d180a1d80 0x7f6d180a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:38:58.505 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.503+0000 7f6d31879640 1 --2- 172.21.15.160:0/1960930585 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f6d180a1d80 0x7f6d180a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:38:58.505 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.503+0000 7f6d22ffd640 1 -- 172.21.15.160:0/1960930585 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6d1c0ef5e0 con 0x7f6d2c104ab0 2024-03-07T07:38:58.505 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.503+0000 7f6d31879640 1 --2- 172.21.15.160:0/1960930585 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f6d180a1d80 0x7f6d180a4240 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6d28009bd0 tx=0x7f6d28005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:38:58.507 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.507+0000 7f6d22ffd640 1 -- 172.21.15.160:0/1960930585 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6d1c0bc860 con 0x7f6d2c104ab0 2024-03-07T07:38:58.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:58 smithi177 ceph-mon[36578]: Deploying daemon mds.i on smithi178 2024-03-07T07:38:58.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:58 smithi177 ceph-mon[36578]: mds.? [v2:172.21.15.177:6836/4088036516,v1:172.21.15.177:6837/4088036516] up:boot 2024-03-07T07:38:58.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:58 smithi177 ceph-mon[36578]: fsmap 8 up:standby 2024-03-07T07:38:58.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:58 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-07T07:38:58.648 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:58 smithi178 ceph-mon[36650]: Deploying daemon mds.i on smithi178 2024-03-07T07:38:58.648 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:58 smithi178 ceph-mon[36650]: mds.? [v2:172.21.15.177:6836/4088036516,v1:172.21.15.177:6837/4088036516] up:boot 2024-03-07T07:38:58.648 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:58 smithi178 ceph-mon[36650]: fsmap 8 up:standby 2024-03-07T07:38:58.649 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:58 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-07T07:38:58.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:58 smithi160 ceph-mon[31963]: Deploying daemon mds.i on smithi178 2024-03-07T07:38:58.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:58 smithi160 ceph-mon[31963]: mds.? [v2:172.21.15.177:6836/4088036516,v1:172.21.15.177:6837/4088036516] up:boot 2024-03-07T07:38:58.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:58 smithi160 ceph-mon[31963]: fsmap 8 up:standby 2024-03-07T07:38:58.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:58 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-07T07:38:58.718 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.718+0000 7f6d33add640 1 -- 172.21.15.160:0/1960930585 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6d2c02d030 con 0x7f6d2c104ab0 2024-03-07T07:38:58.719 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.718+0000 7f6d22ffd640 1 -- 172.21.15.160:0/1960930585 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6877 (secure 0 0 0) 0x7f6d1c0c0900 con 0x7f6d2c104ab0 2024-03-07T07:38:58.719 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:38:58.719 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/2615961083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2615961083},{"type":"v1","addr":"172.21.15.160:6835","nonce":2615961083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1759006033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1759006033},{"type":"v1","addr":"172.21.15.160:6837","nonce":1759006033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1457364761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1457364761},{"type":"v1","addr":"172.21.15.177:6833","nonce":1457364761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/170428292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":170428292},{"type":"v1","addr":"172.21.15.177:6835","nonce":170428292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/645365846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":645365846},{"type":"v1","addr":"172.21.15.178:6833","nonce":645365846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/4088036516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":4088036516},{"type":"v1","addr":"172.21.15.177:6837","nonce":4088036516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/647840547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":647840547},{"type":"v1","addr":"172.21.15.178:6835","nonce":647840547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1247829154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1247829154},{"type":"v1","addr":"172.21.15.178:6837","nonce":1247829154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-03-07T07:38:58.719 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 9 2024-03-07T07:38:58.721 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.720+0000 7f6d33add640 1 -- 172.21.15.160:0/1960930585 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f6d180a1d80 msgr2=0x7f6d180a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:58.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.720+0000 7f6d33add640 1 --2- 172.21.15.160:0/1960930585 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f6d180a1d80 0x7f6d180a4240 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6d28009bd0 tx=0x7f6d28005f90 comp rx=0 tx=0).stop 2024-03-07T07:38:58.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.720+0000 7f6d33add640 1 -- 172.21.15.160:0/1960930585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d2c104ab0 msgr2=0x7f6d2c100130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:38:58.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.720+0000 7f6d33add640 1 --2- 172.21.15.160:0/1960930585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d2c104ab0 0x7f6d2c100130 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c000c00 tx=0x7f6d1c005d50 comp rx=0 tx=0).stop 2024-03-07T07:38:58.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.720+0000 7f6d33add640 1 -- 172.21.15.160:0/1960930585 shutdown_connections 2024-03-07T07:38:58.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.720+0000 7f6d33add640 1 --2- 172.21.15.160:0/1960930585 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f6d180a1d80 0x7f6d180a4240 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:58.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.721+0000 7f6d33add640 1 --2- 172.21.15.160:0/1960930585 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d2c1073e0 0x7f6d2c100670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:58.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.721+0000 7f6d33add640 1 --2- 172.21.15.160:0/1960930585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d2c104ab0 0x7f6d2c100130 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:58.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.721+0000 7f6d33add640 1 --2- 172.21.15.160:0/1960930585 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d2c102180 0x7f6d2c101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:38:58.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.721+0000 7f6d33add640 1 -- 172.21.15.160:0/1960930585 >> 172.21.15.160:0/1960930585 conn(0x7f6d2c0fbb90 msgr2=0x7f6d2c102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:38:58.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.721+0000 7f6d33add640 1 -- 172.21.15.160:0/1960930585 shutdown_connections 2024-03-07T07:38:58.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:38:58.721+0000 7f6d33add640 1 -- 172.21.15.160:0/1960930585 wait complete. 2024-03-07T07:38:59.051 INFO:journalctl@ceph.mds.i.smithi178.stdout:Mar 07 07:38:58 smithi178 systemd[1]: Starting Ceph mds.i for 826f6582-dc54-11ee-95c6-87774f69a715... 2024-03-07T07:38:59.306 INFO:journalctl@ceph.mds.i.smithi178.stdout:Mar 07 07:38:59 smithi178 podman[68603]: 2024-03-07 07:38:58.954305841 +0000 UTC m=+0.017281561 image pull e362cd7cebfee51077fb90b7f01895811140c9a4272edd83ad9910b05e3a46f6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2 2024-03-07T07:38:59.306 INFO:journalctl@ceph.mds.i.smithi178.stdout:Mar 07 07:38:59 smithi178 podman[68603]: 2024-03-07 07:38:59.065257626 +0000 UTC m=+0.128233344 container create cc48c1537dafd0a324d5024673cdd99b77589c1df9fa14c8a900ef818a05642e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-i, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-8b896a9) 2024-03-07T07:38:59.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:59 smithi177 ceph-mon[36578]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:59.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:38:59 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/1960930585' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:38:59.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:59 smithi160 ceph-mon[31963]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:59.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:38:59 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/1960930585' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:38:59.797 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:59 smithi178 ceph-mon[36650]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:38:59.797 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:38:59 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/1960930585' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:38:59.797 INFO:journalctl@ceph.mds.i.smithi178.stdout:Mar 07 07:38:59 smithi178 podman[68603]: 2024-03-07 07:38:59.310188447 +0000 UTC m=+0.373164180 container init cc48c1537dafd0a324d5024673cdd99b77589c1df9fa14c8a900ef818a05642e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-8b896a9, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240222) 2024-03-07T07:38:59.797 INFO:journalctl@ceph.mds.i.smithi178.stdout:Mar 07 07:38:59 smithi178 podman[68603]: 2024-03-07 07:38:59.313005163 +0000 UTC m=+0.375980882 container start cc48c1537dafd0a324d5024673cdd99b77589c1df9fa14c8a900ef818a05642e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-i, GIT_CLEAN=True, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, RELEASE=wip-lusov-qdb-8b896a9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD) 2024-03-07T07:38:59.797 INFO:journalctl@ceph.mds.i.smithi178.stdout:Mar 07 07:38:59 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-i[68618]: starting mds.i at 2024-03-07T07:38:59.797 INFO:journalctl@ceph.mds.i.smithi178.stdout:Mar 07 07:38:59 smithi178 bash[68603]: cc48c1537dafd0a324d5024673cdd99b77589c1df9fa14c8a900ef818a05642e 2024-03-07T07:38:59.798 INFO:journalctl@ceph.mds.i.smithi178.stdout:Mar 07 07:38:59 smithi178 systemd[1]: Started Ceph mds.i for 826f6582-dc54-11ee-95c6-87774f69a715. 2024-03-07T07:39:00.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:00 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:00.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:00 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:00.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:00 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:00.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:00 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-07T07:39:00.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:00 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-07T07:39:00.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:00 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:39:00.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:00 smithi160 ceph-mon[31963]: Deploying daemon mds.g on smithi160 2024-03-07T07:39:00.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:00 smithi160 ceph-mon[31963]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:39:00.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:00 smithi160 ceph-mon[31963]: mds.? [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] up:boot 2024-03-07T07:39:00.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:00 smithi160 ceph-mon[31963]: fsmap 9 up:standby 2024-03-07T07:39:00.679 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:00 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-07T07:39:00.733 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T07:39:00.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:00 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:00.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:00 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:00.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:00 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:00.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:00 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-07T07:39:00.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:00 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-07T07:39:00.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:00 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:39:00.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:00 smithi178 ceph-mon[36650]: Deploying daemon mds.g on smithi160 2024-03-07T07:39:00.808 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:00 smithi178 ceph-mon[36650]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:39:00.808 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:00 smithi178 ceph-mon[36650]: mds.? [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] up:boot 2024-03-07T07:39:00.808 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:00 smithi178 ceph-mon[36650]: fsmap 9 up:standby 2024-03-07T07:39:00.808 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:00 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-07T07:39:00.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:00.840+0000 7f95e70d0640 1 -- 172.21.15.160:0/3452957319 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95e0101bf0 msgr2=0x7f95e0102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:00.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:00.840+0000 7f95e70d0640 1 --2- 172.21.15.160:0/3452957319 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95e0101bf0 0x7f95e0102010 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f95d4009900 tx=0x7f95d402bc80 comp rx=0 tx=0).stop 2024-03-07T07:39:00.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:00.840+0000 7f95e70d0640 1 -- 172.21.15.160:0/3452957319 shutdown_connections 2024-03-07T07:39:00.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:00.840+0000 7f95e70d0640 1 --2- 172.21.15.160:0/3452957319 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95e0101170 0x7f95e0108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:00.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:00.840+0000 7f95e70d0640 1 --2- 172.21.15.160:0/3452957319 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95e01007b0 0x7f95e0100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:00.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:00.840+0000 7f95e70d0640 1 --2- 172.21.15.160:0/3452957319 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95e0101bf0 0x7f95e0102010 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:00.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:00.840+0000 7f95e70d0640 1 -- 172.21.15.160:0/3452957319 >> 172.21.15.160:0/3452957319 conn(0x7f95e00fb260 msgr2=0x7f95e00fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:00.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:00.840+0000 7f95e70d0640 1 -- 172.21.15.160:0/3452957319 shutdown_connections 2024-03-07T07:39:00.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:00.840+0000 7f95e70d0640 1 -- 172.21.15.160:0/3452957319 wait complete. 2024-03-07T07:39:00.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:00.840+0000 7f95e70d0640 1 Processor -- start 2024-03-07T07:39:00.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:00.840+0000 7f95e70d0640 1 -- start start 2024-03-07T07:39:00.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:00.841+0000 7f95e70d0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95e01007b0 0x7f95e0197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:00.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:00.841+0000 7f95e70d0640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95e0101170 0x7f95e0197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:00.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:00.841+0000 7f95e70d0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95e0101bf0 0x7f95e019eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:00.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:00.841+0000 7f95e70d0640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e010c1f0 con 0x7f95e01007b0 2024-03-07T07:39:00.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:00.841+0000 7f95e70d0640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e010c070 con 0x7f95e0101170 2024-03-07T07:39:00.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:00.841+0000 7f95e70d0640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e010c370 con 0x7f95e0101bf0 2024-03-07T07:39:00.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:00.841+0000 7f95e4e6c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95e01007b0 0x7f95e0197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:00.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:00.841+0000 7f95e4e6c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95e01007b0 0x7f95e0197930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:36088/0 (socket says 172.21.15.160:36088) 2024-03-07T07:39:00.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:00.841+0000 7f95e4e6c640 1 -- 172.21.15.160:0/509027892 learned_addr learned my addr 172.21.15.160:0/509027892 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:39:00.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:00.841+0000 7f95dffff640 1 --2- 172.21.15.160:0/509027892 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95e0101170 0x7f95e0197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:00.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:00.841+0000 7f95e4e6c640 1 -- 172.21.15.160:0/509027892 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95e0101bf0 msgr2=0x7f95e019eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:00.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:00.841+0000 7f95e4e6c640 1 --2- 172.21.15.160:0/509027892 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95e0101bf0 0x7f95e019eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:00.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:00.841+0000 7f95e4e6c640 1 -- 172.21.15.160:0/509027892 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95e0101170 msgr2=0x7f95e0197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:00.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:00.841+0000 7f95e4e6c640 1 --2- 172.21.15.160:0/509027892 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95e0101170 0x7f95e0197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:00.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:00.841+0000 7f95e4e6c640 1 -- 172.21.15.160:0/509027892 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e019f550 con 0x7f95e01007b0 2024-03-07T07:39:00.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:00.841+0000 7f95e4e6c640 1 --2- 172.21.15.160:0/509027892 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95e01007b0 0x7f95e0197930 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f95d4005ab0 tx=0x7f95d4005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:00.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:00.841+0000 7f95ddffb640 1 -- 172.21.15.160:0/509027892 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d4041070 con 0x7f95e01007b0 2024-03-07T07:39:00.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:00.841+0000 7f95e70d0640 1 -- 172.21.15.160:0/509027892 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95e019f7e0 con 0x7f95e01007b0 2024-03-07T07:39:00.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:00.841+0000 7f95ddffb640 1 -- 172.21.15.160:0/509027892 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f95d40054e0 con 0x7f95e01007b0 2024-03-07T07:39:00.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:00.841+0000 7f95ddffb640 1 -- 172.21.15.160:0/509027892 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d4034440 con 0x7f95e01007b0 2024-03-07T07:39:00.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:00.842+0000 7f95e70d0640 1 -- 172.21.15.160:0/509027892 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e019fd20 con 0x7f95e01007b0 2024-03-07T07:39:00.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:00.842+0000 7f95e70d0640 1 -- 172.21.15.160:0/509027892 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95e019ffe0 con 0x7f95e01007b0 2024-03-07T07:39:00.849 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:00.843+0000 7f95ddffb640 1 -- 172.21.15.160:0/509027892 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f95d4044070 con 0x7f95e01007b0 2024-03-07T07:39:00.849 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:00.844+0000 7f95ddffb640 1 --2- 172.21.15.160:0/509027892 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f95d00a1d80 0x7f95d00a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:00.849 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:00.844+0000 7f95dffff640 1 --2- 172.21.15.160:0/509027892 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f95d00a1d80 0x7f95d00a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:00.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:00.844+0000 7f95ddffb640 1 -- 172.21.15.160:0/509027892 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f95d40ef560 con 0x7f95e01007b0 2024-03-07T07:39:00.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:00.845+0000 7f95dffff640 1 --2- 172.21.15.160:0/509027892 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f95d00a1d80 0x7f95d00a4240 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f95e0198e40 tx=0x7f95cc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:00.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:00.849+0000 7f95ddffb640 1 -- 172.21.15.160:0/509027892 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f95d40bce30 con 0x7f95e01007b0 2024-03-07T07:39:00.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:00 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:00.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:00 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:00.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:00 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:00.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:00 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-07T07:39:00.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:00 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-07T07:39:00.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:00 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:39:00.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:00 smithi177 ceph-mon[36578]: Deploying daemon mds.g on smithi160 2024-03-07T07:39:00.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:00 smithi177 ceph-mon[36578]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:39:00.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:00 smithi177 ceph-mon[36578]: mds.? [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] up:boot 2024-03-07T07:39:00.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:00 smithi177 ceph-mon[36578]: fsmap 9 up:standby 2024-03-07T07:39:00.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:00 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-07T07:39:01.061 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:01.060+0000 7f95e70d0640 1 -- 172.21.15.160:0/509027892 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95e01a0370 con 0x7f95e01007b0 2024-03-07T07:39:01.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:01.061+0000 7f95ddffb640 1 -- 172.21.15.160:0/509027892 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7673 (secure 0 0 0) 0x7f95d40bc730 con 0x7f95e01007b0 2024-03-07T07:39:01.062 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:39:01.062 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/2615961083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2615961083},{"type":"v1","addr":"172.21.15.160:6835","nonce":2615961083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1759006033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1759006033},{"type":"v1","addr":"172.21.15.160:6837","nonce":1759006033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1457364761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1457364761},{"type":"v1","addr":"172.21.15.177:6833","nonce":1457364761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/170428292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":170428292},{"type":"v1","addr":"172.21.15.177:6835","nonce":170428292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/645365846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":645365846},{"type":"v1","addr":"172.21.15.178:6833","nonce":645365846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/4088036516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":4088036516},{"type":"v1","addr":"172.21.15.177:6837","nonce":4088036516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/647840547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":647840547},{"type":"v1","addr":"172.21.15.178:6835","nonce":647840547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1247829154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1247829154},{"type":"v1","addr":"172.21.15.178:6837","nonce":1247829154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/2947354024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":2947354024},{"type":"v1","addr":"172.21.15.178:6839","nonce":2947354024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-07T07:39:01.063 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 10 2024-03-07T07:39:01.065 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:01.063+0000 7f95e70d0640 1 -- 172.21.15.160:0/509027892 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f95d00a1d80 msgr2=0x7f95d00a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:01.065 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:01.063+0000 7f95e70d0640 1 --2- 172.21.15.160:0/509027892 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f95d00a1d80 0x7f95d00a4240 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f95e0198e40 tx=0x7f95cc00b390 comp rx=0 tx=0).stop 2024-03-07T07:39:01.065 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:01.063+0000 7f95e70d0640 1 -- 172.21.15.160:0/509027892 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95e01007b0 msgr2=0x7f95e0197930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:01.065 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:01.063+0000 7f95e70d0640 1 --2- 172.21.15.160:0/509027892 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95e01007b0 0x7f95e0197930 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f95d4005ab0 tx=0x7f95d4005de0 comp rx=0 tx=0).stop 2024-03-07T07:39:01.065 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:01.063+0000 7f95e70d0640 1 -- 172.21.15.160:0/509027892 shutdown_connections 2024-03-07T07:39:01.065 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:01.063+0000 7f95e70d0640 1 --2- 172.21.15.160:0/509027892 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f95d00a1d80 0x7f95d00a4240 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:01.065 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:01.063+0000 7f95e70d0640 1 --2- 172.21.15.160:0/509027892 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95e0101bf0 0x7f95e019eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:01.065 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:01.063+0000 7f95e70d0640 1 --2- 172.21.15.160:0/509027892 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95e0101170 0x7f95e0197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:01.065 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:01.063+0000 7f95e70d0640 1 --2- 172.21.15.160:0/509027892 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95e01007b0 0x7f95e0197930 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:01.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:01.063+0000 7f95e70d0640 1 -- 172.21.15.160:0/509027892 >> 172.21.15.160:0/509027892 conn(0x7f95e00fb260 msgr2=0x7f95e00fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:01.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:01.063+0000 7f95e70d0640 1 -- 172.21.15.160:0/509027892 shutdown_connections 2024-03-07T07:39:01.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:01.063+0000 7f95e70d0640 1 -- 172.21.15.160:0/509027892 wait complete. 2024-03-07T07:39:01.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:01 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/509027892' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:39:01.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:01 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/509027892' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:39:01.888 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:01 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/509027892' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:39:01.889 INFO:journalctl@ceph.mds.g.smithi160.stdout:Mar 07 07:39:01 smithi160 systemd[1]: Starting Ceph mds.g for 826f6582-dc54-11ee-95c6-87774f69a715... 2024-03-07T07:39:02.144 INFO:journalctl@ceph.mds.g.smithi160.stdout:Mar 07 07:39:01 smithi160 podman[73648]: 2024-03-07 07:39:01.79099892 +0000 UTC m=+0.018637257 image pull e362cd7cebfee51077fb90b7f01895811140c9a4272edd83ad9910b05e3a46f6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2 2024-03-07T07:39:02.144 INFO:journalctl@ceph.mds.g.smithi160.stdout:Mar 07 07:39:01 smithi160 podman[73648]: 2024-03-07 07:39:01.899740765 +0000 UTC m=+0.127379096 container create ad2e3e0c37fbe51a3f2852fa654451b9096687bf80d080ba311fb9213a614d22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-g, ceph=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-8b896a9, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-07T07:39:02.144 INFO:journalctl@ceph.mds.g.smithi160.stdout:Mar 07 07:39:02 smithi160 podman[73648]: 2024-03-07 07:39:02.144467277 +0000 UTC m=+0.372105612 container init ad2e3e0c37fbe51a3f2852fa654451b9096687bf80d080ba311fb9213a614d22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-g, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-8b896a9, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True) 2024-03-07T07:39:02.428 INFO:journalctl@ceph.mds.g.smithi160.stdout:Mar 07 07:39:02 smithi160 podman[73648]: 2024-03-07 07:39:02.147169021 +0000 UTC m=+0.374807358 container start ad2e3e0c37fbe51a3f2852fa654451b9096687bf80d080ba311fb9213a614d22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-g, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.build-date=20240222, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-07T07:39:02.428 INFO:journalctl@ceph.mds.g.smithi160.stdout:Mar 07 07:39:02 smithi160 ceph-mds[73667]: -- 172.21.15.160:0/4082276339 <== mon.0 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5592c1c65a40 con 0x5592c1c39000 2024-03-07T07:39:02.428 INFO:journalctl@ceph.mds.g.smithi160.stdout:Mar 07 07:39:02 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-g[73663]: starting mds.g at 2024-03-07T07:39:02.428 INFO:journalctl@ceph.mds.g.smithi160.stdout:Mar 07 07:39:02 smithi160 bash[73648]: ad2e3e0c37fbe51a3f2852fa654451b9096687bf80d080ba311fb9213a614d22 2024-03-07T07:39:02.428 INFO:journalctl@ceph.mds.g.smithi160.stdout:Mar 07 07:39:02 smithi160 systemd[1]: Started Ceph mds.g for 826f6582-dc54-11ee-95c6-87774f69a715. 2024-03-07T07:39:03.077 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T07:39:03.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.189+0000 7f9577606640 1 -- 172.21.15.160:0/581126850 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9570100580 msgr2=0x7f95701009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:03.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.189+0000 7f9577606640 1 --2- 172.21.15.160:0/581126850 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9570100580 0x7f95701009a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f956c009900 tx=0x7f956c02bc30 comp rx=0 tx=0).stop 2024-03-07T07:39:03.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.189+0000 7f9577606640 1 -- 172.21.15.160:0/581126850 shutdown_connections 2024-03-07T07:39:03.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.189+0000 7f9577606640 1 --2- 172.21.15.160:0/581126850 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9570102130 0x7f95701089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:03.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.189+0000 7f9577606640 1 --2- 172.21.15.160:0/581126850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9570101770 0x7f9570101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:03.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.189+0000 7f9577606640 1 --2- 172.21.15.160:0/581126850 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9570100580 0x7f95701009a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:03.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.189+0000 7f9577606640 1 -- 172.21.15.160:0/581126850 >> 172.21.15.160:0/581126850 conn(0x7f95700fbaf0 msgr2=0x7f95700fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:03.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.190+0000 7f9577606640 1 -- 172.21.15.160:0/581126850 shutdown_connections 2024-03-07T07:39:03.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.190+0000 7f9577606640 1 -- 172.21.15.160:0/581126850 wait complete. 2024-03-07T07:39:03.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.190+0000 7f9577606640 1 Processor -- start 2024-03-07T07:39:03.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.190+0000 7f9577606640 1 -- start start 2024-03-07T07:39:03.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.190+0000 7f9577606640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9570100580 0x7f95701979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:03.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.190+0000 7f9577606640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9570101770 0x7f9570197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:03.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.190+0000 7f9577606640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9570102130 0x7f957019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:03.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.190+0000 7f9577606640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957010c210 con 0x7f9570100580 2024-03-07T07:39:03.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.190+0000 7f9577606640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957010c090 con 0x7f9570102130 2024-03-07T07:39:03.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.190+0000 7f9577606640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957010c390 con 0x7f9570101770 2024-03-07T07:39:03.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.190+0000 7f95753a2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9570100580 0x7f95701979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:03.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.190+0000 7f95753a2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9570100580 0x7f95701979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:36128/0 (socket says 172.21.15.160:36128) 2024-03-07T07:39:03.194 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.190+0000 7f95753a2640 1 -- 172.21.15.160:0/3143097399 learned_addr learned my addr 172.21.15.160:0/3143097399 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:39:03.194 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.190+0000 7f9575ba3640 1 --2- 172.21.15.160:0/3143097399 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9570102130 0x7f957019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:03.194 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.191+0000 7f9574ba1640 1 --2- 172.21.15.160:0/3143097399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9570101770 0x7f9570197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:03.194 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.191+0000 7f95753a2640 1 -- 172.21.15.160:0/3143097399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9570101770 msgr2=0x7f9570197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:03.194 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.191+0000 7f95753a2640 1 --2- 172.21.15.160:0/3143097399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9570101770 0x7f9570197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:03.194 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.191+0000 7f95753a2640 1 -- 172.21.15.160:0/3143097399 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9570102130 msgr2=0x7f957019ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:03.194 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.191+0000 7f95753a2640 1 --2- 172.21.15.160:0/3143097399 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9570102130 0x7f957019ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:03.194 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.191+0000 7f95753a2640 1 -- 172.21.15.160:0/3143097399 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f957019f5c0 con 0x7f9570100580 2024-03-07T07:39:03.194 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.191+0000 7f95753a2640 1 --2- 172.21.15.160:0/3143097399 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9570100580 0x7f95701979f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f956c036040 tx=0x7f956c004ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:03.194 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.191+0000 7f95667fc640 1 -- 172.21.15.160:0/3143097399 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956c043020 con 0x7f9570100580 2024-03-07T07:39:03.194 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.191+0000 7f9577606640 1 -- 172.21.15.160:0/3143097399 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f957019f850 con 0x7f9570100580 2024-03-07T07:39:03.194 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.191+0000 7f95667fc640 1 -- 172.21.15.160:0/3143097399 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f956c02c980 con 0x7f9570100580 2024-03-07T07:39:03.194 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.191+0000 7f95667fc640 1 -- 172.21.15.160:0/3143097399 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956c03e6b0 con 0x7f9570100580 2024-03-07T07:39:03.195 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.191+0000 7f9577606640 1 -- 172.21.15.160:0/3143097399 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f957019fdf0 con 0x7f9570100580 2024-03-07T07:39:03.195 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.192+0000 7f9577606640 1 -- 172.21.15.160:0/3143097399 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f957019fff0 con 0x7f9570100580 2024-03-07T07:39:03.195 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.194+0000 7f95667fc640 1 -- 172.21.15.160:0/3143097399 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f956c046020 con 0x7f9570100580 2024-03-07T07:39:03.196 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.195+0000 7f95667fc640 1 --2- 172.21.15.160:0/3143097399 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f955c0a1d80 0x7f955c0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:03.196 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.195+0000 7f9574ba1640 1 --2- 172.21.15.160:0/3143097399 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f955c0a1d80 0x7f955c0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:03.196 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.195+0000 7f95667fc640 1 -- 172.21.15.160:0/3143097399 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f956c0f04a0 con 0x7f9570100580 2024-03-07T07:39:03.196 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.195+0000 7f9574ba1640 1 --2- 172.21.15.160:0/3143097399 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f955c0a1d80 0x7f955c0a4240 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9570198eb0 tx=0x7f9560005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:03.199 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.199+0000 7f95667fc640 1 -- 172.21.15.160:0/3143097399 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f956c04d020 con 0x7f9570100580 2024-03-07T07:39:03.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.420+0000 7f9577606640 1 -- 172.21.15.160:0/3143097399 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95701a01c0 con 0x7f9570100580 2024-03-07T07:39:03.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.420+0000 7f95667fc640 1 -- 172.21.15.160:0/3143097399 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8468 (secure 0 0 0) 0x7f956c0bde90 con 0x7f9570100580 2024-03-07T07:39:03.421 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:39:03.421 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/2615961083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2615961083},{"type":"v1","addr":"172.21.15.160:6835","nonce":2615961083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1759006033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1759006033},{"type":"v1","addr":"172.21.15.160:6837","nonce":1759006033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6839/3763944721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":3763944721},{"type":"v1","addr":"172.21.15.160:6839","nonce":3763944721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1457364761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1457364761},{"type":"v1","addr":"172.21.15.177:6833","nonce":1457364761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/170428292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":170428292},{"type":"v1","addr":"172.21.15.177:6835","nonce":170428292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/645365846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":645365846},{"type":"v1","addr":"172.21.15.178:6833","nonce":645365846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/4088036516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":4088036516},{"type":"v1","addr":"172.21.15.177:6837","nonce":4088036516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/647840547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":647840547},{"type":"v1","addr":"172.21.15.178:6835","nonce":647840547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1247829154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1247829154},{"type":"v1","addr":"172.21.15.178:6837","nonce":1247829154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/2947354024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":2947354024},{"type":"v1","addr":"172.21.15.178:6839","nonce":2947354024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-07T07:39:03.421 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 11 2024-03-07T07:39:03.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.422+0000 7f9577606640 1 -- 172.21.15.160:0/3143097399 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f955c0a1d80 msgr2=0x7f955c0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:03.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.422+0000 7f9577606640 1 --2- 172.21.15.160:0/3143097399 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f955c0a1d80 0x7f955c0a4240 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9570198eb0 tx=0x7f9560005f90 comp rx=0 tx=0).stop 2024-03-07T07:39:03.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.422+0000 7f9577606640 1 -- 172.21.15.160:0/3143097399 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9570100580 msgr2=0x7f95701979f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:03.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.422+0000 7f9577606640 1 --2- 172.21.15.160:0/3143097399 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9570100580 0x7f95701979f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f956c036040 tx=0x7f956c004ee0 comp rx=0 tx=0).stop 2024-03-07T07:39:03.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.422+0000 7f9577606640 1 -- 172.21.15.160:0/3143097399 shutdown_connections 2024-03-07T07:39:03.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.422+0000 7f9577606640 1 --2- 172.21.15.160:0/3143097399 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f955c0a1d80 0x7f955c0a4240 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:03.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.422+0000 7f9577606640 1 --2- 172.21.15.160:0/3143097399 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9570102130 0x7f957019ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:03.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.422+0000 7f9577606640 1 --2- 172.21.15.160:0/3143097399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9570101770 0x7f9570197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:03.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.422+0000 7f9577606640 1 --2- 172.21.15.160:0/3143097399 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9570100580 0x7f95701979f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:03.424 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.422+0000 7f9577606640 1 -- 172.21.15.160:0/3143097399 >> 172.21.15.160:0/3143097399 conn(0x7f95700fbaf0 msgr2=0x7f95700fc130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:03.424 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.423+0000 7f9577606640 1 -- 172.21.15.160:0/3143097399 shutdown_connections 2024-03-07T07:39:03.424 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:03.423+0000 7f9577606640 1 -- 172.21.15.160:0/3143097399 wait complete. 2024-03-07T07:39:03.434 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:03 smithi160 ceph-mon[31963]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:39:03.434 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:03 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:03.434 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:03 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:03.435 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:03 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:03.435 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:03 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-07T07:39:03.435 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:03 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-07T07:39:03.435 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:03 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:39:03.435 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:03 smithi160 ceph-mon[31963]: Deploying daemon mds.a on smithi160 2024-03-07T07:39:03.435 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:03 smithi160 ceph-mon[31963]: mds.? [v2:172.21.15.160:6838/3763944721,v1:172.21.15.160:6839/3763944721] up:boot 2024-03-07T07:39:03.435 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:03 smithi160 ceph-mon[31963]: fsmap 10 up:standby 2024-03-07T07:39:03.435 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:03 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-07T07:39:03.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:03 smithi178 ceph-mon[36650]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:39:03.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:03 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:03.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:03 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:03.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:03 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:03.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:03 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-07T07:39:03.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:03 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-07T07:39:03.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:03 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:39:03.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:03 smithi178 ceph-mon[36650]: Deploying daemon mds.a on smithi160 2024-03-07T07:39:03.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:03 smithi178 ceph-mon[36650]: mds.? [v2:172.21.15.160:6838/3763944721,v1:172.21.15.160:6839/3763944721] up:boot 2024-03-07T07:39:03.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:03 smithi178 ceph-mon[36650]: fsmap 10 up:standby 2024-03-07T07:39:03.557 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:03 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-07T07:39:03.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:03 smithi177 ceph-mon[36578]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:39:03.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:03 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:03.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:03 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:03.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:03 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:03.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:03 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-07T07:39:03.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:03 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-07T07:39:03.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:03 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:39:03.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:03 smithi177 ceph-mon[36578]: Deploying daemon mds.a on smithi160 2024-03-07T07:39:03.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:03 smithi177 ceph-mon[36578]: mds.? [v2:172.21.15.160:6838/3763944721,v1:172.21.15.160:6839/3763944721] up:boot 2024-03-07T07:39:03.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:03 smithi177 ceph-mon[36578]: fsmap 10 up:standby 2024-03-07T07:39:03.608 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:03 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-07T07:39:04.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:04 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/3143097399' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:39:04.428 INFO:journalctl@ceph.mds.a.smithi160.stdout:Mar 07 07:39:04 smithi160 systemd[1]: Starting Ceph mds.a for 826f6582-dc54-11ee-95c6-87774f69a715... 2024-03-07T07:39:04.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:04 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/3143097399' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:39:04.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:04 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/3143097399' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:39:04.842 INFO:journalctl@ceph.mds.a.smithi160.stdout:Mar 07 07:39:04 smithi160 podman[74008]: 2024-03-07 07:39:04.474905051 +0000 UTC m=+0.018258276 image pull e362cd7cebfee51077fb90b7f01895811140c9a4272edd83ad9910b05e3a46f6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2 2024-03-07T07:39:04.842 INFO:journalctl@ceph.mds.a.smithi160.stdout:Mar 07 07:39:04 smithi160 podman[74008]: 2024-03-07 07:39:04.598200413 +0000 UTC m=+0.141553672 container create 9cb5a59bf35606c23c679224999a00de63828cc8824dd66f6e42d5b97bad698e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-a, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-8b896a9, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-07T07:39:04.842 INFO:journalctl@ceph.mds.a.smithi160.stdout:Mar 07 07:39:04 smithi160 podman[74008]: 2024-03-07 07:39:04.842839514 +0000 UTC m=+0.386192735 container init 9cb5a59bf35606c23c679224999a00de63828cc8824dd66f6e42d5b97bad698e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-a, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-07T07:39:05.178 INFO:journalctl@ceph.mds.a.smithi160.stdout:Mar 07 07:39:04 smithi160 podman[74008]: 2024-03-07 07:39:04.845536612 +0000 UTC m=+0.388889839 container start 9cb5a59bf35606c23c679224999a00de63828cc8824dd66f6e42d5b97bad698e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-8b896a9, ceph=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b) 2024-03-07T07:39:05.178 INFO:journalctl@ceph.mds.a.smithi160.stdout:Mar 07 07:39:04 smithi160 ceph-mds[74027]: -- 172.21.15.160:0/851372515 <== mon.0 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562ea8f55a40 con 0x562ea8f29000 2024-03-07T07:39:05.178 INFO:journalctl@ceph.mds.a.smithi160.stdout:Mar 07 07:39:04 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-a[74023]: starting mds.a at 2024-03-07T07:39:05.178 INFO:journalctl@ceph.mds.a.smithi160.stdout:Mar 07 07:39:04 smithi160 bash[74008]: 9cb5a59bf35606c23c679224999a00de63828cc8824dd66f6e42d5b97bad698e 2024-03-07T07:39:05.178 INFO:journalctl@ceph.mds.a.smithi160.stdout:Mar 07 07:39:04 smithi160 systemd[1]: Started Ceph mds.a for 826f6582-dc54-11ee-95c6-87774f69a715. 2024-03-07T07:39:05.437 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T07:39:05.546 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.544+0000 7fd1a3034640 1 -- 172.21.15.160:0/2318055609 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd19c103220 msgr2=0x7fd19c109ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:05.546 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.544+0000 7fd1a3034640 1 --2- 172.21.15.160:0/2318055609 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd19c103220 0x7fd19c109ab0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd194009ca0 tx=0x7fd19402c0f0 comp rx=0 tx=0).stop 2024-03-07T07:39:05.546 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.544+0000 7fd1a3034640 1 -- 172.21.15.160:0/2318055609 shutdown_connections 2024-03-07T07:39:05.547 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.544+0000 7fd1a3034640 1 --2- 172.21.15.160:0/2318055609 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd19c103220 0x7fd19c109ab0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:05.547 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.544+0000 7fd1a3034640 1 --2- 172.21.15.160:0/2318055609 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd19c102860 0x7fd19c102ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:05.547 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.544+0000 7fd1a3034640 1 --2- 172.21.15.160:0/2318055609 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd19c101670 0x7fd19c101a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:05.547 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.544+0000 7fd1a3034640 1 -- 172.21.15.160:0/2318055609 >> 172.21.15.160:0/2318055609 conn(0x7fd19c0fcc40 msgr2=0x7fd19c0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:05.547 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.544+0000 7fd1a3034640 1 -- 172.21.15.160:0/2318055609 shutdown_connections 2024-03-07T07:39:05.547 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.544+0000 7fd1a3034640 1 -- 172.21.15.160:0/2318055609 wait complete. 2024-03-07T07:39:05.547 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.544+0000 7fd1a3034640 1 Processor -- start 2024-03-07T07:39:05.547 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.544+0000 7fd1a3034640 1 -- start start 2024-03-07T07:39:05.547 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.544+0000 7fd1a3034640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd19c101670 0x7fd19c1978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:05.547 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.544+0000 7fd1a3034640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd19c102860 0x7fd19c197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:05.547 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.544+0000 7fd1a3034640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd19c103220 0x7fd19c19eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:05.547 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.544+0000 7fd1a3034640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c10d2d0 con 0x7fd19c103220 2024-03-07T07:39:05.547 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.544+0000 7fd1a3034640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c10d150 con 0x7fd19c101670 2024-03-07T07:39:05.547 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.544+0000 7fd1a3034640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c10d450 con 0x7fd19c102860 2024-03-07T07:39:05.548 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.545+0000 7fd1a15d1640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd19c103220 0x7fd19c19eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:05.548 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.545+0000 7fd1a15d1640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd19c103220 0x7fd19c19eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:36150/0 (socket says 172.21.15.160:36150) 2024-03-07T07:39:05.548 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.545+0000 7fd1a15d1640 1 -- 172.21.15.160:0/2083851844 learned_addr learned my addr 172.21.15.160:0/2083851844 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:39:05.548 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.545+0000 7fd1a0dd0640 1 --2- 172.21.15.160:0/2083851844 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd19c101670 0x7fd19c1978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:05.548 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.545+0000 7fd19bfff640 1 --2- 172.21.15.160:0/2083851844 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd19c102860 0x7fd19c197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:05.548 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.545+0000 7fd1a15d1640 1 -- 172.21.15.160:0/2083851844 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd19c102860 msgr2=0x7fd19c197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:05.548 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.545+0000 7fd1a15d1640 1 --2- 172.21.15.160:0/2083851844 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd19c102860 0x7fd19c197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:05.548 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.545+0000 7fd1a15d1640 1 -- 172.21.15.160:0/2083851844 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd19c101670 msgr2=0x7fd19c1978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:05.548 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.545+0000 7fd1a15d1640 1 --2- 172.21.15.160:0/2083851844 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd19c101670 0x7fd19c1978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:05.548 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.545+0000 7fd1a15d1640 1 -- 172.21.15.160:0/2083851844 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd19c19f510 con 0x7fd19c103220 2024-03-07T07:39:05.548 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.545+0000 7fd19bfff640 1 --2- 172.21.15.160:0/2083851844 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd19c102860 0x7fd19c197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T07:39:05.548 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.545+0000 7fd1a15d1640 1 --2- 172.21.15.160:0/2083851844 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd19c103220 0x7fd19c19eea0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd194037040 tx=0x7fd194032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:05.549 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.545+0000 7fd199ffb640 1 -- 172.21.15.160:0/2083851844 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd194041020 con 0x7fd19c103220 2024-03-07T07:39:05.549 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.545+0000 7fd199ffb640 1 -- 172.21.15.160:0/2083851844 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd194034d90 con 0x7fd19c103220 2024-03-07T07:39:05.549 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.545+0000 7fd1a3034640 1 -- 172.21.15.160:0/2083851844 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd19c19f7a0 con 0x7fd19c103220 2024-03-07T07:39:05.549 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.545+0000 7fd199ffb640 1 -- 172.21.15.160:0/2083851844 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd194039040 con 0x7fd19c103220 2024-03-07T07:39:05.549 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.545+0000 7fd1a3034640 1 -- 172.21.15.160:0/2083851844 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd19c19fc90 con 0x7fd19c103220 2024-03-07T07:39:05.550 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.546+0000 7fd1a3034640 1 -- 172.21.15.160:0/2083851844 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd19c108990 con 0x7fd19c103220 2024-03-07T07:39:05.550 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.547+0000 7fd199ffb640 1 -- 172.21.15.160:0/2083851844 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7fd194044020 con 0x7fd19c103220 2024-03-07T07:39:05.550 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.548+0000 7fd199ffb640 1 --2- 172.21.15.160:0/2083851844 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fd1840a1d80 0x7fd1840a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:05.550 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.548+0000 7fd1a0dd0640 1 --2- 172.21.15.160:0/2083851844 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fd1840a1d80 0x7fd1840a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:05.550 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.548+0000 7fd199ffb640 1 -- 172.21.15.160:0/2083851844 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd1940ef8c0 con 0x7fd19c103220 2024-03-07T07:39:05.550 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.548+0000 7fd1a0dd0640 1 --2- 172.21.15.160:0/2083851844 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fd1840a1d80 0x7fd1840a4240 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd19c1026c0 tx=0x7fd18c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:05.553 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.553+0000 7fd199ffb640 1 -- 172.21.15.160:0/2083851844 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd19404b020 con 0x7fd19c103220 2024-03-07T07:39:05.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:05 smithi178 ceph-mon[36650]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:39:05.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:05 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:05.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:05 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:05.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:05 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:05.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:05 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-07T07:39:05.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:05 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-07T07:39:05.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:05 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:39:05.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:05 smithi177 ceph-mon[36578]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:39:05.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:05 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:05.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:05 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:05.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:05 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:05.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:05 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-07T07:39:05.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:05 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-07T07:39:05.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:05 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:39:05.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:05 smithi160 ceph-mon[31963]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:39:05.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:05 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:05.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:05 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:05.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:05 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:05.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:05 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-07T07:39:05.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:05 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-07T07:39:05.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:05 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:39:05.764 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.764+0000 7fd1a3034640 1 -- 172.21.15.160:0/2083851844 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd19c02d030 con 0x7fd19c103220 2024-03-07T07:39:05.765 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.764+0000 7fd199ffb640 1 -- 172.21.15.160:0/2083851844 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9263 (secure 0 0 0) 0x7fd1940bd170 con 0x7fd19c103220 2024-03-07T07:39:05.765 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:39:05.765 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/2615961083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2615961083},{"type":"v1","addr":"172.21.15.160:6835","nonce":2615961083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1759006033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1759006033},{"type":"v1","addr":"172.21.15.160:6837","nonce":1759006033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6839/3763944721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":3763944721},{"type":"v1","addr":"172.21.15.160:6839","nonce":3763944721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6841/3611426637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6840","nonce":3611426637},{"type":"v1","addr":"172.21.15.160:6841","nonce":3611426637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1457364761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1457364761},{"type":"v1","addr":"172.21.15.177:6833","nonce":1457364761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/170428292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":170428292},{"type":"v1","addr":"172.21.15.177:6835","nonce":170428292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/645365846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":645365846},{"type":"v1","addr":"172.21.15.178:6833","nonce":645365846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/4088036516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":4088036516},{"type":"v1","addr":"172.21.15.177:6837","nonce":4088036516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/647840547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":647840547},{"type":"v1","addr":"172.21.15.178:6835","nonce":647840547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1247829154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1247829154},{"type":"v1","addr":"172.21.15.178:6837","nonce":1247829154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/2947354024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":2947354024},{"type":"v1","addr":"172.21.15.178:6839","nonce":2947354024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-07T07:39:05.765 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 12 2024-03-07T07:39:05.767 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.767+0000 7fd1a3034640 1 -- 172.21.15.160:0/2083851844 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fd1840a1d80 msgr2=0x7fd1840a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:05.767 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.767+0000 7fd1a3034640 1 --2- 172.21.15.160:0/2083851844 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fd1840a1d80 0x7fd1840a4240 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd19c1026c0 tx=0x7fd18c005f90 comp rx=0 tx=0).stop 2024-03-07T07:39:05.767 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.767+0000 7fd1a3034640 1 -- 172.21.15.160:0/2083851844 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd19c103220 msgr2=0x7fd19c19eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:05.767 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.767+0000 7fd1a3034640 1 --2- 172.21.15.160:0/2083851844 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd19c103220 0x7fd19c19eea0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd194037040 tx=0x7fd194032f20 comp rx=0 tx=0).stop 2024-03-07T07:39:05.767 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.767+0000 7fd1a3034640 1 -- 172.21.15.160:0/2083851844 shutdown_connections 2024-03-07T07:39:05.767 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.767+0000 7fd1a3034640 1 --2- 172.21.15.160:0/2083851844 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fd1840a1d80 0x7fd1840a4240 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:05.768 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.767+0000 7fd1a3034640 1 --2- 172.21.15.160:0/2083851844 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd19c103220 0x7fd19c19eea0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:05.768 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.767+0000 7fd1a3034640 1 --2- 172.21.15.160:0/2083851844 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd19c102860 0x7fd19c197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:05.768 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.767+0000 7fd1a3034640 1 --2- 172.21.15.160:0/2083851844 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd19c101670 0x7fd19c1978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:05.768 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.767+0000 7fd1a3034640 1 -- 172.21.15.160:0/2083851844 >> 172.21.15.160:0/2083851844 conn(0x7fd19c0fcc40 msgr2=0x7fd19c105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:05.768 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.767+0000 7fd1a3034640 1 -- 172.21.15.160:0/2083851844 shutdown_connections 2024-03-07T07:39:05.768 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:05.767+0000 7fd1a3034640 1 -- 172.21.15.160:0/2083851844 wait complete. 2024-03-07T07:39:06.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:06 smithi177 ceph-mon[36578]: Deploying daemon mds.b on smithi177 2024-03-07T07:39:06.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:06 smithi177 ceph-mon[36578]: mds.? [v2:172.21.15.160:6840/3611426637,v1:172.21.15.160:6841/3611426637] up:boot 2024-03-07T07:39:06.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:06 smithi177 ceph-mon[36578]: fsmap 11 up:standby 2024-03-07T07:39:06.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:06 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-07T07:39:06.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:06 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/2083851844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:39:06.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:06 smithi160 ceph-mon[31963]: Deploying daemon mds.b on smithi177 2024-03-07T07:39:06.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:06 smithi160 ceph-mon[31963]: mds.? [v2:172.21.15.160:6840/3611426637,v1:172.21.15.160:6841/3611426637] up:boot 2024-03-07T07:39:06.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:06 smithi160 ceph-mon[31963]: fsmap 11 up:standby 2024-03-07T07:39:06.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:06 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-07T07:39:06.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:06 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/2083851844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:39:06.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:06 smithi178 ceph-mon[36650]: Deploying daemon mds.b on smithi177 2024-03-07T07:39:06.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:06 smithi178 ceph-mon[36650]: mds.? [v2:172.21.15.160:6840/3611426637,v1:172.21.15.160:6841/3611426637] up:boot 2024-03-07T07:39:06.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:06 smithi178 ceph-mon[36650]: fsmap 11 up:standby 2024-03-07T07:39:06.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:06 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-07T07:39:06.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:06 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/2083851844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:39:07.312 INFO:journalctl@ceph.mds.b.smithi177.stdout:Mar 07 07:39:07 smithi177 systemd[1]: Starting Ceph mds.b for 826f6582-dc54-11ee-95c6-87774f69a715... 2024-03-07T07:39:07.579 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:07 smithi177 ceph-mon[36578]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:39:07.579 INFO:journalctl@ceph.mds.b.smithi177.stdout:Mar 07 07:39:07 smithi177 podman[67005]: 2024-03-07 07:39:07.362129133 +0000 UTC m=+0.017930788 image pull e362cd7cebfee51077fb90b7f01895811140c9a4272edd83ad9910b05e3a46f6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2 2024-03-07T07:39:07.580 INFO:journalctl@ceph.mds.b.smithi177.stdout:Mar 07 07:39:07 smithi177 podman[67005]: 2024-03-07 07:39:07.467384774 +0000 UTC m=+0.123186426 container create 5121d6ee5cd475884facacac9b89ca5cac2b84ae3958b456cdb35eea85a2599b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-8b896a9) 2024-03-07T07:39:07.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:07 smithi160 ceph-mon[31963]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:39:07.779 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T07:39:07.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:07 smithi178 ceph-mon[36650]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:39:07.857 INFO:journalctl@ceph.mds.b.smithi177.stdout:Mar 07 07:39:07 smithi177 podman[67005]: 2024-03-07 07:39:07.705377212 +0000 UTC m=+0.361178865 container init 5121d6ee5cd475884facacac9b89ca5cac2b84ae3958b456cdb35eea85a2599b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-b, ceph=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-8b896a9, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-03-07T07:39:07.857 INFO:journalctl@ceph.mds.b.smithi177.stdout:Mar 07 07:39:07 smithi177 podman[67005]: 2024-03-07 07:39:07.707984494 +0000 UTC m=+0.363786137 container start 5121d6ee5cd475884facacac9b89ca5cac2b84ae3958b456cdb35eea85a2599b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-b, RELEASE=wip-lusov-qdb-8b896a9, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-07T07:39:07.857 INFO:journalctl@ceph.mds.b.smithi177.stdout:Mar 07 07:39:07 smithi177 ceph-mds[67024]: -- 172.21.15.177:0/2265823422 <== mon.2 v2:172.21.15.177:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55dc42e69a40 con 0x55dc42e3c800 2024-03-07T07:39:07.857 INFO:journalctl@ceph.mds.b.smithi177.stdout:Mar 07 07:39:07 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-b[67020]: starting mds.b at 2024-03-07T07:39:07.857 INFO:journalctl@ceph.mds.b.smithi177.stdout:Mar 07 07:39:07 smithi177 bash[67005]: 5121d6ee5cd475884facacac9b89ca5cac2b84ae3958b456cdb35eea85a2599b 2024-03-07T07:39:07.857 INFO:journalctl@ceph.mds.b.smithi177.stdout:Mar 07 07:39:07 smithi177 systemd[1]: Started Ceph mds.b for 826f6582-dc54-11ee-95c6-87774f69a715. 2024-03-07T07:39:07.897 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:07.896+0000 7f4760f1d640 1 -- 172.21.15.160:0/2182790172 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f475c102150 msgr2=0x7f475c1089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:07.897 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:07.896+0000 7f4760f1d640 1 --2- 172.21.15.160:0/2182790172 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f475c102150 0x7f475c1089e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4754009ca0 tx=0x7f475402c0f0 comp rx=0 tx=0).stop 2024-03-07T07:39:07.898 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:07.896+0000 7f4760f1d640 1 -- 172.21.15.160:0/2182790172 shutdown_connections 2024-03-07T07:39:07.898 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:07.896+0000 7f4760f1d640 1 --2- 172.21.15.160:0/2182790172 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f475c102150 0x7f475c1089e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:07.898 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:07.896+0000 7f4760f1d640 1 --2- 172.21.15.160:0/2182790172 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f475c101790 0x7f475c101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:07.898 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:07.896+0000 7f4760f1d640 1 --2- 172.21.15.160:0/2182790172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f475c1005a0 0x7f475c1009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:07.898 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:07.897+0000 7f4760f1d640 1 -- 172.21.15.160:0/2182790172 >> 172.21.15.160:0/2182790172 conn(0x7f475c0fbb10 msgr2=0x7f475c0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:07.898 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:07.897+0000 7f4760f1d640 1 -- 172.21.15.160:0/2182790172 shutdown_connections 2024-03-07T07:39:07.898 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:07.897+0000 7f4760f1d640 1 -- 172.21.15.160:0/2182790172 wait complete. 2024-03-07T07:39:07.898 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:07.897+0000 7f4760f1d640 1 Processor -- start 2024-03-07T07:39:07.900 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:07.898+0000 7f4760f1d640 1 -- start start 2024-03-07T07:39:07.900 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:07.898+0000 7f4760f1d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f475c1005a0 0x7f475c1044e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:07.900 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:07.898+0000 7f4760f1d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f475c101790 0x7f475c102b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:07.900 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:07.898+0000 7f4760f1d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f475c102150 0x7f475c1030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:07.901 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:07.898+0000 7f4760f1d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475c10c230 con 0x7f475c102150 2024-03-07T07:39:07.901 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:07.898+0000 7f4760f1d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475c10c0b0 con 0x7f475c1005a0 2024-03-07T07:39:07.901 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:07.898+0000 7f4760f1d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475c10c3b0 con 0x7f475c101790 2024-03-07T07:39:07.901 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:07.898+0000 7f475ad9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f475c102150 0x7f475c1030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:07.901 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:07.898+0000 7f475ad9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f475c102150 0x7f475c1030b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:36154/0 (socket says 172.21.15.160:36154) 2024-03-07T07:39:07.901 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:07.898+0000 7f475ad9d640 1 -- 172.21.15.160:0/1427174831 learned_addr learned my addr 172.21.15.160:0/1427174831 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:39:07.901 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:07.898+0000 7f475a59c640 1 --2- 172.21.15.160:0/1427174831 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f475c1005a0 0x7f475c1044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:07.901 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:07.898+0000 7f4751d9b640 1 --2- 172.21.15.160:0/1427174831 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f475c101790 0x7f475c102b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:07.901 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:07.898+0000 7f475ad9d640 1 -- 172.21.15.160:0/1427174831 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f475c101790 msgr2=0x7f475c102b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:07.901 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:07.898+0000 7f475ad9d640 1 --2- 172.21.15.160:0/1427174831 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f475c101790 0x7f475c102b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:07.901 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:07.898+0000 7f475ad9d640 1 -- 172.21.15.160:0/1427174831 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f475c1005a0 msgr2=0x7f475c1044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:07.901 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:07.898+0000 7f475ad9d640 1 --2- 172.21.15.160:0/1427174831 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f475c1005a0 0x7f475c1044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:07.901 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:07.898+0000 7f475ad9d640 1 -- 172.21.15.160:0/1427174831 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f475c103970 con 0x7f475c102150 2024-03-07T07:39:07.902 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:07.898+0000 7f475a59c640 1 --2- 172.21.15.160:0/1427174831 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f475c1005a0 0x7f475c1044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:39:07.902 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:07.898+0000 7f475ad9d640 1 --2- 172.21.15.160:0/1427174831 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f475c102150 0x7f475c1030b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4754037040 tx=0x7f4754032ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:07.902 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:07.899+0000 7f4753fff640 1 -- 172.21.15.160:0/1427174831 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4754041020 con 0x7f475c102150 2024-03-07T07:39:07.902 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:07.899+0000 7f4760f1d640 1 -- 172.21.15.160:0/1427174831 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f475c1a1ad0 con 0x7f475c102150 2024-03-07T07:39:07.902 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:07.899+0000 7f4753fff640 1 -- 172.21.15.160:0/1427174831 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4754032340 con 0x7f475c102150 2024-03-07T07:39:07.902 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:07.899+0000 7f4753fff640 1 -- 172.21.15.160:0/1427174831 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4754039040 con 0x7f475c102150 2024-03-07T07:39:07.902 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:07.899+0000 7f4760f1d640 1 -- 172.21.15.160:0/1427174831 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f475c1a1fe0 con 0x7f475c102150 2024-03-07T07:39:07.902 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:07.900+0000 7f4760f1d640 1 -- 172.21.15.160:0/1427174831 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f475c1a2270 con 0x7f475c102150 2024-03-07T07:39:07.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:07.901+0000 7f4753fff640 1 -- 172.21.15.160:0/1427174831 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f4754044020 con 0x7f475c102150 2024-03-07T07:39:07.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:07.901+0000 7f4753fff640 1 --2- 172.21.15.160:0/1427174831 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f47480a1d80 0x7f47480a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:07.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:07.902+0000 7f475a59c640 1 --2- 172.21.15.160:0/1427174831 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f47480a1d80 0x7f47480a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:07.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:07.902+0000 7f4753fff640 1 -- 172.21.15.160:0/1427174831 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f47540ef720 con 0x7f475c102150 2024-03-07T07:39:07.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:07.902+0000 7f475a59c640 1 --2- 172.21.15.160:0/1427174831 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f47480a1d80 0x7f47480a4240 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f475c1015f0 tx=0x7f474c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:07.906 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:07.906+0000 7f4753fff640 1 -- 172.21.15.160:0/1427174831 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f475404b020 con 0x7f475c102150 2024-03-07T07:39:08.167 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:08.167+0000 7f4760f1d640 1 -- 172.21.15.160:0/1427174831 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f475c1a2440 con 0x7f475c102150 2024-03-07T07:39:08.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:08.168+0000 7f4753fff640 1 -- 172.21.15.160:0/1427174831 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9263 (secure 0 0 0) 0x7f47540bcf50 con 0x7f475c102150 2024-03-07T07:39:08.168 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:39:08.168 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/2615961083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2615961083},{"type":"v1","addr":"172.21.15.160:6835","nonce":2615961083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1759006033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1759006033},{"type":"v1","addr":"172.21.15.160:6837","nonce":1759006033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6839/3763944721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":3763944721},{"type":"v1","addr":"172.21.15.160:6839","nonce":3763944721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6841/3611426637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6840","nonce":3611426637},{"type":"v1","addr":"172.21.15.160:6841","nonce":3611426637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1457364761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1457364761},{"type":"v1","addr":"172.21.15.177:6833","nonce":1457364761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/170428292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":170428292},{"type":"v1","addr":"172.21.15.177:6835","nonce":170428292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/645365846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":645365846},{"type":"v1","addr":"172.21.15.178:6833","nonce":645365846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/4088036516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":4088036516},{"type":"v1","addr":"172.21.15.177:6837","nonce":4088036516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/647840547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":647840547},{"type":"v1","addr":"172.21.15.178:6835","nonce":647840547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1247829154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1247829154},{"type":"v1","addr":"172.21.15.178:6837","nonce":1247829154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/2947354024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":2947354024},{"type":"v1","addr":"172.21.15.178:6839","nonce":2947354024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-07T07:39:08.169 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 12 2024-03-07T07:39:08.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:08.170+0000 7f4760f1d640 1 -- 172.21.15.160:0/1427174831 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f47480a1d80 msgr2=0x7f47480a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:08.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:08.170+0000 7f4760f1d640 1 --2- 172.21.15.160:0/1427174831 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f47480a1d80 0x7f47480a4240 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f475c1015f0 tx=0x7f474c005f90 comp rx=0 tx=0).stop 2024-03-07T07:39:08.171 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:08.170+0000 7f4760f1d640 1 -- 172.21.15.160:0/1427174831 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f475c102150 msgr2=0x7f475c1030b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:08.171 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:08.170+0000 7f4760f1d640 1 --2- 172.21.15.160:0/1427174831 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f475c102150 0x7f475c1030b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4754037040 tx=0x7f4754032ea0 comp rx=0 tx=0).stop 2024-03-07T07:39:08.171 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:08.170+0000 7f4760f1d640 1 -- 172.21.15.160:0/1427174831 shutdown_connections 2024-03-07T07:39:08.171 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:08.170+0000 7f4760f1d640 1 --2- 172.21.15.160:0/1427174831 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f47480a1d80 0x7f47480a4240 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:08.171 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:08.170+0000 7f4760f1d640 1 --2- 172.21.15.160:0/1427174831 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f475c102150 0x7f475c1030b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:08.171 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:08.170+0000 7f4760f1d640 1 --2- 172.21.15.160:0/1427174831 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f475c101790 0x7f475c102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:08.171 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:08.170+0000 7f4760f1d640 1 --2- 172.21.15.160:0/1427174831 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f475c1005a0 0x7f475c1044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:08.171 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:08.170+0000 7f4760f1d640 1 -- 172.21.15.160:0/1427174831 >> 172.21.15.160:0/1427174831 conn(0x7f475c0fbb10 msgr2=0x7f475c0fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:08.171 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:08.170+0000 7f4760f1d640 1 -- 172.21.15.160:0/1427174831 shutdown_connections 2024-03-07T07:39:08.171 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:08.170+0000 7f4760f1d640 1 -- 172.21.15.160:0/1427174831 wait complete. 2024-03-07T07:39:09.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:08 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:09.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:08 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:09.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:08 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:09.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:08 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:09.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:08 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:09.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:08 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:39:09.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:08 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/1427174831' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:39:09.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:08 smithi177 ceph-mon[36578]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:39:09.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:08 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:09.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:08 smithi177 ceph-mon[36578]: mds.? [v2:172.21.15.177:6838/3329773838,v1:172.21.15.177:6839/3329773838] up:boot 2024-03-07T07:39:09.108 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:08 smithi177 ceph-mon[36578]: fsmap 12 up:standby 2024-03-07T07:39:09.108 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:08 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-07T07:39:09.128 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:08 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:09.128 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:08 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:09.129 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:08 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:09.129 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:08 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:09.129 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:08 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:09.129 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:08 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:39:09.129 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:08 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/1427174831' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:39:09.129 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:08 smithi178 ceph-mon[36650]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:39:09.129 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:08 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:09.129 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:08 smithi178 ceph-mon[36650]: mds.? [v2:172.21.15.177:6838/3329773838,v1:172.21.15.177:6839/3329773838] up:boot 2024-03-07T07:39:09.129 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:08 smithi178 ceph-mon[36650]: fsmap 12 up:standby 2024-03-07T07:39:09.129 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:08 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-07T07:39:09.130 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:08 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:09.130 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:08 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:09.130 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:08 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:09.130 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:08 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:09.130 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:08 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:09.130 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:08 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:39:09.130 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:08 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/1427174831' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:39:09.130 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:08 smithi160 ceph-mon[31963]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:39:09.130 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:08 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:09.130 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:08 smithi160 ceph-mon[31963]: mds.? [v2:172.21.15.177:6838/3329773838,v1:172.21.15.177:6839/3329773838] up:boot 2024-03-07T07:39:09.130 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:08 smithi160 ceph-mon[31963]: fsmap 12 up:standby 2024-03-07T07:39:09.131 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:08 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-07T07:39:10.182 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T07:39:10.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.288+0000 7f77f9161640 1 -- 172.21.15.160:0/4200380203 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f77f4104a90 msgr2=0x7f77f4106e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:10.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.288+0000 7f77f9161640 1 --2- 172.21.15.160:0/4200380203 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f77f4104a90 0x7f77f4106e80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f77e8009900 tx=0x7f77e802bc30 comp rx=0 tx=0).stop 2024-03-07T07:39:10.291 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.288+0000 7f77f9161640 1 -- 172.21.15.160:0/4200380203 shutdown_connections 2024-03-07T07:39:10.291 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.288+0000 7f77f9161640 1 --2- 172.21.15.160:0/4200380203 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f77f41073c0 0x7f77f41097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:10.291 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.288+0000 7f77f9161640 1 --2- 172.21.15.160:0/4200380203 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f77f4104a90 0x7f77f4106e80 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:10.291 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.288+0000 7f77f9161640 1 --2- 172.21.15.160:0/4200380203 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77f4102160 0x7f77f4104550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:10.291 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.288+0000 7f77f9161640 1 -- 172.21.15.160:0/4200380203 >> 172.21.15.160:0/4200380203 conn(0x7f77f40fbaf0 msgr2=0x7f77f40fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:10.291 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.288+0000 7f77f9161640 1 -- 172.21.15.160:0/4200380203 shutdown_connections 2024-03-07T07:39:10.291 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.288+0000 7f77f9161640 1 -- 172.21.15.160:0/4200380203 wait complete. 2024-03-07T07:39:10.291 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.289+0000 7f77f9161640 1 Processor -- start 2024-03-07T07:39:10.291 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.289+0000 7f77f9161640 1 -- start start 2024-03-07T07:39:10.291 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.289+0000 7f77f9161640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f77f4102160 0x7f77f4101ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:10.291 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.289+0000 7f77f9161640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f77f4104a90 0x7f77f4100110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:10.291 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.289+0000 7f77f9161640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77f41073c0 0x7f77f4100650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:10.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.289+0000 7f77f9161640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f410d300 con 0x7f77f4104a90 2024-03-07T07:39:10.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.289+0000 7f77f9161640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f410d180 con 0x7f77f4102160 2024-03-07T07:39:10.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.289+0000 7f77f9161640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f410d480 con 0x7f77f41073c0 2024-03-07T07:39:10.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.289+0000 7f77f259c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f77f4104a90 0x7f77f4100110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:10.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.289+0000 7f77f259c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f77f4104a90 0x7f77f4100110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:42092/0 (socket says 172.21.15.160:42092) 2024-03-07T07:39:10.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.289+0000 7f77f259c640 1 -- 172.21.15.160:0/2926243063 learned_addr learned my addr 172.21.15.160:0/2926243063 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:39:10.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.289+0000 7f77f359e640 1 --2- 172.21.15.160:0/2926243063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77f41073c0 0x7f77f4100650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:10.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.289+0000 7f77f2d9d640 1 --2- 172.21.15.160:0/2926243063 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f77f4102160 0x7f77f4101ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:10.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.290+0000 7f77f259c640 1 -- 172.21.15.160:0/2926243063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77f41073c0 msgr2=0x7f77f4100650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:10.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.290+0000 7f77f259c640 1 --2- 172.21.15.160:0/2926243063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77f41073c0 0x7f77f4100650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:10.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.290+0000 7f77f259c640 1 -- 172.21.15.160:0/2926243063 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f77f4102160 msgr2=0x7f77f4101ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:10.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.290+0000 7f77f259c640 1 --2- 172.21.15.160:0/2926243063 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f77f4102160 0x7f77f4101ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:10.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.290+0000 7f77f259c640 1 -- 172.21.15.160:0/2926243063 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f4100ee0 con 0x7f77f4104a90 2024-03-07T07:39:10.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.290+0000 7f77f259c640 1 --2- 172.21.15.160:0/2926243063 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f77f4104a90 0x7f77f4100110 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f77e8000c00 tx=0x7f77e802c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:10.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.290+0000 7f77dbfff640 1 -- 172.21.15.160:0/2926243063 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e8041070 con 0x7f77f4104a90 2024-03-07T07:39:10.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.290+0000 7f77dbfff640 1 -- 172.21.15.160:0/2926243063 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77e8004450 con 0x7f77f4104a90 2024-03-07T07:39:10.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.290+0000 7f77f9161640 1 -- 172.21.15.160:0/2926243063 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77f41a1a90 con 0x7f77f4104a90 2024-03-07T07:39:10.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.290+0000 7f77dbfff640 1 -- 172.21.15.160:0/2926243063 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e8034440 con 0x7f77f4104a90 2024-03-07T07:39:10.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.290+0000 7f77f9161640 1 -- 172.21.15.160:0/2926243063 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77f41a1fe0 con 0x7f77f4104a90 2024-03-07T07:39:10.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.291+0000 7f77f9161640 1 -- 172.21.15.160:0/2926243063 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77f40fd5e0 con 0x7f77f4104a90 2024-03-07T07:39:10.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.292+0000 7f77dbfff640 1 -- 172.21.15.160:0/2926243063 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f77e8044070 con 0x7f77f4104a90 2024-03-07T07:39:10.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.293+0000 7f77dbfff640 1 --2- 172.21.15.160:0/2926243063 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f77dc0a1d80 0x7f77dc0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:10.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.293+0000 7f77f2d9d640 1 --2- 172.21.15.160:0/2926243063 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f77dc0a1d80 0x7f77dc0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:10.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.293+0000 7f77dbfff640 1 -- 172.21.15.160:0/2926243063 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f77e80ef420 con 0x7f77f4104a90 2024-03-07T07:39:10.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.293+0000 7f77f2d9d640 1 --2- 172.21.15.160:0/2926243063 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f77dc0a1d80 0x7f77dc0a4240 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f77e4009bd0 tx=0x7f77e4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:10.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.297+0000 7f77dbfff640 1 -- 172.21.15.160:0/2926243063 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f77e80bc690 con 0x7f77f4104a90 2024-03-07T07:39:10.528 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.528+0000 7f77f9161640 1 -- 172.21.15.160:0/2926243063 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f77f4062260 con 0x7f77f4104a90 2024-03-07T07:39:10.529 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.528+0000 7f77dbfff640 1 -- 172.21.15.160:0/2926243063 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10058 (secure 0 0 0) 0x7f77e80c0730 con 0x7f77f4104a90 2024-03-07T07:39:10.529 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:39:10.529 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/2615961083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2615961083},{"type":"v1","addr":"172.21.15.160:6835","nonce":2615961083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1759006033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1759006033},{"type":"v1","addr":"172.21.15.160:6837","nonce":1759006033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6839/3763944721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":3763944721},{"type":"v1","addr":"172.21.15.160:6839","nonce":3763944721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6841/3611426637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6840","nonce":3611426637},{"type":"v1","addr":"172.21.15.160:6841","nonce":3611426637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1457364761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1457364761},{"type":"v1","addr":"172.21.15.177:6833","nonce":1457364761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/170428292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":170428292},{"type":"v1","addr":"172.21.15.177:6835","nonce":170428292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/645365846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":645365846},{"type":"v1","addr":"172.21.15.178:6833","nonce":645365846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/4088036516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":4088036516},{"type":"v1","addr":"172.21.15.177:6837","nonce":4088036516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/647840547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":647840547},{"type":"v1","addr":"172.21.15.178:6835","nonce":647840547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1247829154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1247829154},{"type":"v1","addr":"172.21.15.178:6837","nonce":1247829154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24497,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6839/3329773838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3329773838},{"type":"v1","addr":"172.21.15.177:6839","nonce":3329773838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/2947354024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":2947354024},{"type":"v1","addr":"172.21.15.178:6839","nonce":2947354024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-07T07:39:10.529 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 13 2024-03-07T07:39:10.531 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.531+0000 7f77f9161640 1 -- 172.21.15.160:0/2926243063 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f77dc0a1d80 msgr2=0x7f77dc0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:10.531 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.531+0000 7f77f9161640 1 --2- 172.21.15.160:0/2926243063 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f77dc0a1d80 0x7f77dc0a4240 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f77e4009bd0 tx=0x7f77e4005f90 comp rx=0 tx=0).stop 2024-03-07T07:39:10.531 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.531+0000 7f77f9161640 1 -- 172.21.15.160:0/2926243063 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f77f4104a90 msgr2=0x7f77f4100110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:10.531 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.531+0000 7f77f9161640 1 --2- 172.21.15.160:0/2926243063 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f77f4104a90 0x7f77f4100110 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f77e8000c00 tx=0x7f77e802c640 comp rx=0 tx=0).stop 2024-03-07T07:39:10.531 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.531+0000 7f77f9161640 1 -- 172.21.15.160:0/2926243063 shutdown_connections 2024-03-07T07:39:10.531 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.531+0000 7f77f9161640 1 --2- 172.21.15.160:0/2926243063 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f77dc0a1d80 0x7f77dc0a4240 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:10.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.531+0000 7f77f9161640 1 --2- 172.21.15.160:0/2926243063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77f41073c0 0x7f77f4100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:10.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.531+0000 7f77f9161640 1 --2- 172.21.15.160:0/2926243063 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f77f4104a90 0x7f77f4100110 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:10.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.531+0000 7f77f9161640 1 --2- 172.21.15.160:0/2926243063 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f77f4102160 0x7f77f4101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:10.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.531+0000 7f77f9161640 1 -- 172.21.15.160:0/2926243063 >> 172.21.15.160:0/2926243063 conn(0x7f77f40fbaf0 msgr2=0x7f77f4102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:10.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.531+0000 7f77f9161640 1 -- 172.21.15.160:0/2926243063 shutdown_connections 2024-03-07T07:39:10.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.531+0000 7f77f9161640 1 -- 172.21.15.160:0/2926243063 wait complete. 2024-03-07T07:39:10.543 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T07:39:10.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.644+0000 7f5a43514640 1 -- 172.21.15.160:0/2279269102 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5a3c1005a0 msgr2=0x7f5a3c1009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:10.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.644+0000 7f5a43514640 1 --2- 172.21.15.160:0/2279269102 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5a3c1005a0 0x7f5a3c1009c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5a38009900 tx=0x7f5a3802bc30 comp rx=0 tx=0).stop 2024-03-07T07:39:10.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.644+0000 7f5a43514640 1 -- 172.21.15.160:0/2279269102 shutdown_connections 2024-03-07T07:39:10.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.644+0000 7f5a43514640 1 --2- 172.21.15.160:0/2279269102 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a3c102150 0x7f5a3c1089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:10.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.644+0000 7f5a43514640 1 --2- 172.21.15.160:0/2279269102 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5a3c101790 0x7f5a3c101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:10.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.644+0000 7f5a43514640 1 --2- 172.21.15.160:0/2279269102 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5a3c1005a0 0x7f5a3c1009c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:10.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.644+0000 7f5a43514640 1 -- 172.21.15.160:0/2279269102 >> 172.21.15.160:0/2279269102 conn(0x7f5a3c0fbb10 msgr2=0x7f5a3c0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:10.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.644+0000 7f5a43514640 1 -- 172.21.15.160:0/2279269102 shutdown_connections 2024-03-07T07:39:10.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.644+0000 7f5a43514640 1 -- 172.21.15.160:0/2279269102 wait complete. 2024-03-07T07:39:10.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.644+0000 7f5a43514640 1 Processor -- start 2024-03-07T07:39:10.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.644+0000 7f5a43514640 1 -- start start 2024-03-07T07:39:10.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.644+0000 7f5a43514640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5a3c1005a0 0x7f5a3c1979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:10.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.644+0000 7f5a43514640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5a3c101790 0x7f5a3c197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:10.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.644+0000 7f5a43514640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a3c102150 0x7f5a3c19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:10.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.644+0000 7f5a43514640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c10d340 con 0x7f5a3c1005a0 2024-03-07T07:39:10.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.644+0000 7f5a43514640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c10d1c0 con 0x7f5a3c101790 2024-03-07T07:39:10.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.644+0000 7f5a43514640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c10d4c0 con 0x7f5a3c102150 2024-03-07T07:39:10.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.645+0000 7f5a412b0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5a3c1005a0 0x7f5a3c1979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:10.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.645+0000 7f5a412b0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5a3c1005a0 0x7f5a3c1979c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:42114/0 (socket says 172.21.15.160:42114) 2024-03-07T07:39:10.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.645+0000 7f5a412b0640 1 -- 172.21.15.160:0/1352572891 learned_addr learned my addr 172.21.15.160:0/1352572891 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:39:10.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.645+0000 7f5a41ab1640 1 --2- 172.21.15.160:0/1352572891 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a3c102150 0x7f5a3c19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:10.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.645+0000 7f5a40aaf640 1 --2- 172.21.15.160:0/1352572891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5a3c101790 0x7f5a3c197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:10.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.645+0000 7f5a412b0640 1 -- 172.21.15.160:0/1352572891 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a3c102150 msgr2=0x7f5a3c19ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:10.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.645+0000 7f5a412b0640 1 --2- 172.21.15.160:0/1352572891 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a3c102150 0x7f5a3c19ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:10.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.645+0000 7f5a412b0640 1 -- 172.21.15.160:0/1352572891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5a3c101790 msgr2=0x7f5a3c197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:10.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.645+0000 7f5a412b0640 1 --2- 172.21.15.160:0/1352572891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5a3c101790 0x7f5a3c197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:10.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.645+0000 7f5a412b0640 1 -- 172.21.15.160:0/1352572891 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a3c19f5e0 con 0x7f5a3c1005a0 2024-03-07T07:39:10.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.645+0000 7f5a40aaf640 1 --2- 172.21.15.160:0/1352572891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5a3c101790 0x7f5a3c197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:39:10.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.645+0000 7f5a412b0640 1 --2- 172.21.15.160:0/1352572891 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5a3c1005a0 0x7f5a3c1979c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5a38000c00 tx=0x7f5a38005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:10.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.645+0000 7f5a327fc640 1 -- 172.21.15.160:0/1352572891 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a38041070 con 0x7f5a3c1005a0 2024-03-07T07:39:10.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.645+0000 7f5a43514640 1 -- 172.21.15.160:0/1352572891 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a3c19f870 con 0x7f5a3c1005a0 2024-03-07T07:39:10.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.645+0000 7f5a327fc640 1 -- 172.21.15.160:0/1352572891 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a380044d0 con 0x7f5a3c1005a0 2024-03-07T07:39:10.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.645+0000 7f5a327fc640 1 -- 172.21.15.160:0/1352572891 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a38034440 con 0x7f5a3c1005a0 2024-03-07T07:39:10.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.645+0000 7f5a43514640 1 -- 172.21.15.160:0/1352572891 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a3c19fd60 con 0x7f5a3c1005a0 2024-03-07T07:39:10.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.646+0000 7f5a43514640 1 -- 172.21.15.160:0/1352572891 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a3c1078c0 con 0x7f5a3c1005a0 2024-03-07T07:39:10.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.647+0000 7f5a327fc640 1 -- 172.21.15.160:0/1352572891 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f5a38044070 con 0x7f5a3c1005a0 2024-03-07T07:39:10.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.648+0000 7f5a327fc640 1 --2- 172.21.15.160:0/1352572891 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f5a280a1d80 0x7f5a280a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:10.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.648+0000 7f5a40aaf640 1 --2- 172.21.15.160:0/1352572891 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f5a280a1d80 0x7f5a280a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:10.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.648+0000 7f5a327fc640 1 -- 172.21.15.160:0/1352572891 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5a380ef4d0 con 0x7f5a3c1005a0 2024-03-07T07:39:10.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.648+0000 7f5a40aaf640 1 --2- 172.21.15.160:0/1352572891 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f5a280a1d80 0x7f5a280a4240 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c198ed0 tx=0x7f5a2c005ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:10.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.652+0000 7f5a327fc640 1 -- 172.21.15.160:0/1352572891 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5a380bcda0 con 0x7f5a3c1005a0 2024-03-07T07:39:10.839 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.838+0000 7f5a43514640 1 -- 172.21.15.160:0/1352572891 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5a3c062260 con 0x7f5a3c1005a0 2024-03-07T07:39:10.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.840+0000 7f5a327fc640 1 -- 172.21.15.160:0/1352572891 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18479 (secure 0 0 0) 0x7f5a380bc6a0 con 0x7f5a3c1005a0 2024-03-07T07:39:10.840 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:39:10.840 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":81,"fsid":"826f6582-dc54-11ee-95c6-87774f69a715","created":"2024-03-07T07:31:14.229288+0000","modified":"2024-03-07T07:38:22.331696+0000","last_up_change":"2024-03-07T07:38:20.776225+0000","last_in_change":"2024-03-07T07:38:00.508917+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T07:34:34.529424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bee66c67-fb6e-4717-9685-ca62587d5f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6803","nonce":1368482926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6805","nonce":1368482926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6809","nonce":1368482926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6807","nonce":1368482926}]},"public_addr":"172.21.15.160:6803/1368482926","cluster_addr":"172.21.15.160:6805/1368482926","heartbeat_back_addr":"172.21.15.160:6809/1368482926","heartbeat_front_addr":"172.21.15.160:6807/1368482926","state":["exists","up"]},{"osd":1,"uuid":"49236251-f750-424e-992b-bddf55762556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6801","nonce":2922391884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6803","nonce":2922391884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6807","nonce":2922391884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6805","nonce":2922391884}]},"public_addr":"172.21.15.177:6801/2922391884","cluster_addr":"172.21.15.177:6803/2922391884","heartbeat_back_addr":"172.21.15.177:6807/2922391884","heartbeat_front_addr":"172.21.15.177:6805/2922391884","state":["exists","up"]},{"osd":2,"uuid":"7a432b28-f7ee-43fe-9196-7aaeccfecf77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6801","nonce":1217901551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6803","nonce":1217901551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6807","nonce":1217901551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6805","nonce":1217901551}]},"public_addr":"172.21.15.178:6801/1217901551","cluster_addr":"172.21.15.178:6803/1217901551","heartbeat_back_addr":"172.21.15.178:6807/1217901551","heartbeat_front_addr":"172.21.15.178:6805/1217901551","state":["exists","up"]},{"osd":3,"uuid":"4ad22a20-f1be-4002-a48d-ecd3953e63f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6811","nonce":42543989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6813","nonce":42543989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6817","nonce":42543989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6815","nonce":42543989}]},"public_addr":"172.21.15.160:6811/42543989","cluster_addr":"172.21.15.160:6813/42543989","heartbeat_back_addr":"172.21.15.160:6817/42543989","heartbeat_front_addr":"172.21.15.160:6815/42543989","state":["exists","up"]},{"osd":4,"uuid":"2b65b104-ac82-4ce3-b6c5-11a399103045","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6809","nonce":88178553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6811","nonce":88178553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6815","nonce":88178553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6813","nonce":88178553}]},"public_addr":"172.21.15.177:6809/88178553","cluster_addr":"172.21.15.177:6811/88178553","heartbeat_back_addr":"172.21.15.177:6815/88178553","heartbeat_front_addr":"172.21.15.177:6813/88178553","state":["exists","up"]},{"osd":5,"uuid":"30a1e714-08ff-44e8-9264-9ef164604b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6809","nonce":964732891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6811","nonce":964732891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6815","nonce":964732891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6813","nonce":964732891}]},"public_addr":"172.21.15.178:6809/964732891","cluster_addr":"172.21.15.178:6811/964732891","heartbeat_back_addr":"172.21.15.178:6815/964732891","heartbeat_front_addr":"172.21.15.178:6813/964732891","state":["exists","up"]},{"osd":6,"uuid":"b1e1646e-f8a9-42b8-b62d-9d7b5d6a1ff4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6819","nonce":3251327001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6821","nonce":3251327001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6825","nonce":3251327001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6823","nonce":3251327001}]},"public_addr":"172.21.15.160:6819/3251327001","cluster_addr":"172.21.15.160:6821/3251327001","heartbeat_back_addr":"172.21.15.160:6825/3251327001","heartbeat_front_addr":"172.21.15.160:6823/3251327001","state":["exists","up"]},{"osd":7,"uuid":"e66b400b-9747-4f6d-b05b-db22468e2750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6817","nonce":1175500850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6819","nonce":1175500850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6823","nonce":1175500850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6821","nonce":1175500850}]},"public_addr":"172.21.15.177:6817/1175500850","cluster_addr":"172.21.15.177:6819/1175500850","heartbeat_back_addr":"172.21.15.177:6823/1175500850","heartbeat_front_addr":"172.21.15.177:6821/1175500850","state":["exists","up"]},{"osd":8,"uuid":"3ee0f2c4-fca4-49fc-a98b-07aa0fcba0e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6817","nonce":688015531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6819","nonce":688015531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6823","nonce":688015531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6821","nonce":688015531}]},"public_addr":"172.21.15.178:6817/688015531","cluster_addr":"172.21.15.178:6819/688015531","heartbeat_back_addr":"172.21.15.178:6823/688015531","heartbeat_front_addr":"172.21.15.178:6821/688015531","state":["exists","up"]},{"osd":9,"uuid":"856039d7-f7ee-4dbd-b019-0073e705ce0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6827","nonce":3265797851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6829","nonce":3265797851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6833","nonce":3265797851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6831","nonce":3265797851}]},"public_addr":"172.21.15.160:6827/3265797851","cluster_addr":"172.21.15.160:6829/3265797851","heartbeat_back_addr":"172.21.15.160:6833/3265797851","heartbeat_front_addr":"172.21.15.160:6831/3265797851","state":["exists","up"]},{"osd":10,"uuid":"bdc9899d-d720-4429-ad53-4c388a4dbe75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6825","nonce":2867473412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6827","nonce":2867473412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6831","nonce":2867473412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6829","nonce":2867473412}]},"public_addr":"172.21.15.177:6825/2867473412","cluster_addr":"172.21.15.177:6827/2867473412","heartbeat_back_addr":"172.21.15.177:6831/2867473412","heartbeat_front_addr":"172.21.15.177:6829/2867473412","state":["exists","up"]},{"osd":11,"uuid":"cc9adb54-ef49-47c6-90e3-fc03bc0f5cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6825","nonce":4068724731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6827","nonce":4068724731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6831","nonce":4068724731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6829","nonce":4068724731}]},"public_addr":"172.21.15.178:6825/4068724731","cluster_addr":"172.21.15.178:6827/4068724731","heartbeat_back_addr":"172.21.15.178:6831/4068724731","heartbeat_front_addr":"172.21.15.178:6829/4068724731","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:33:44.141585+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:07.685585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:31.044386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:56.308925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:35:21.582097+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:35:45.983540+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:36:11.530242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:36:36.844681+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:02.303437+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:27.629867+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:53.289495+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:38:18.536169+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/3027450612":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/874709722":"2024-03-08T07:32:08.190837+0000","172.21.15.160:6800/1130675225":"2024-03-08T07:31:41.166858+0000","172.21.15.160:0/3857193980":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/3861968297":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6800/3859174315":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/3013534743":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6801/3859174315":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/1310937667":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6801/1130675225":"2024-03-08T07:31:41.166858+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T07:39:10.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.842+0000 7f5a43514640 1 -- 172.21.15.160:0/1352572891 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f5a280a1d80 msgr2=0x7f5a280a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:10.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.842+0000 7f5a43514640 1 --2- 172.21.15.160:0/1352572891 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f5a280a1d80 0x7f5a280a4240 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c198ed0 tx=0x7f5a2c005ec0 comp rx=0 tx=0).stop 2024-03-07T07:39:10.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.842+0000 7f5a43514640 1 -- 172.21.15.160:0/1352572891 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5a3c1005a0 msgr2=0x7f5a3c1979c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:10.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.842+0000 7f5a43514640 1 --2- 172.21.15.160:0/1352572891 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5a3c1005a0 0x7f5a3c1979c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5a38000c00 tx=0x7f5a38005d50 comp rx=0 tx=0).stop 2024-03-07T07:39:10.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.842+0000 7f5a43514640 1 -- 172.21.15.160:0/1352572891 shutdown_connections 2024-03-07T07:39:10.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.842+0000 7f5a43514640 1 --2- 172.21.15.160:0/1352572891 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f5a280a1d80 0x7f5a280a4240 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:10.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.842+0000 7f5a43514640 1 --2- 172.21.15.160:0/1352572891 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a3c102150 0x7f5a3c19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:10.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.842+0000 7f5a43514640 1 --2- 172.21.15.160:0/1352572891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5a3c101790 0x7f5a3c197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:10.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.842+0000 7f5a43514640 1 --2- 172.21.15.160:0/1352572891 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5a3c1005a0 0x7f5a3c1979c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:10.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.842+0000 7f5a43514640 1 -- 172.21.15.160:0/1352572891 >> 172.21.15.160:0/1352572891 conn(0x7f5a3c0fbb10 msgr2=0x7f5a3c104a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:10.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.842+0000 7f5a43514640 1 -- 172.21.15.160:0/1352572891 shutdown_connections 2024-03-07T07:39:10.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.842+0000 7f5a43514640 1 -- 172.21.15.160:0/1352572891 wait complete. 2024-03-07T07:39:10.852 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:10 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:10.852 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:10 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:10.852 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:10 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:10.852 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:10 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:10.852 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:10 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:10.853 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:10 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:10.853 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:10 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:39:10.853 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:10 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:39:10.853 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:10 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:10.853 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:10 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:39:10.853 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:10 smithi160 ceph-mon[31963]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:39:10.853 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:10 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/2926243063' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:39:10.854 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-07T07:34:34.529424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-07T07:39:10.855 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-07T07:39:10.944 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:10 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:10.945 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:10 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:10.945 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:10 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:10.945 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:10 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:10.945 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:10 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:10.945 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:10 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:10.945 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:10 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:39:10.945 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:10 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:39:10.945 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:10 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:10.945 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:10 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:39:10.945 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:10 smithi177 ceph-mon[36578]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:39:10.945 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:10 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/2926243063' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:39:10.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.960+0000 7f40252c7640 1 -- 172.21.15.160:0/368446533 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4020101b10 msgr2=0x7f4020101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:10.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.960+0000 7f40252c7640 1 --2- 172.21.15.160:0/368446533 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4020101b10 0x7f4020101f30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4010009900 tx=0x7f401002bc30 comp rx=0 tx=0).stop 2024-03-07T07:39:10.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.960+0000 7f40252c7640 1 -- 172.21.15.160:0/368446533 shutdown_connections 2024-03-07T07:39:10.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.960+0000 7f40252c7640 1 --2- 172.21.15.160:0/368446533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4020101200 0x7f4020108aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:10.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.960+0000 7f40252c7640 1 --2- 172.21.15.160:0/368446533 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4020100840 0x7f4020100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:10.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.960+0000 7f40252c7640 1 --2- 172.21.15.160:0/368446533 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4020101b10 0x7f4020101f30 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:10.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.960+0000 7f40252c7640 1 -- 172.21.15.160:0/368446533 >> 172.21.15.160:0/368446533 conn(0x7f40200fb290 msgr2=0x7f40200fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:10.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.961+0000 7f40252c7640 1 -- 172.21.15.160:0/368446533 shutdown_connections 2024-03-07T07:39:10.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.961+0000 7f40252c7640 1 -- 172.21.15.160:0/368446533 wait complete. 2024-03-07T07:39:10.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.961+0000 7f40252c7640 1 Processor -- start 2024-03-07T07:39:10.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.961+0000 7f40252c7640 1 -- start start 2024-03-07T07:39:10.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.961+0000 7f40252c7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4020100840 0x7f40201002a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:10.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.961+0000 7f40252c7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4020101200 0x7f402019c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:10.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.961+0000 7f40252c7640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4020101b10 0x7f402019ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:10.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.961+0000 7f40252c7640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402010c250 con 0x7f4020101b10 2024-03-07T07:39:10.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.961+0000 7f40252c7640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402010c0d0 con 0x7f4020100840 2024-03-07T07:39:10.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.961+0000 7f40252c7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402010c3d0 con 0x7f4020101200 2024-03-07T07:39:10.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.962+0000 7f401f7fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4020101b10 0x7f402019ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:10.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.962+0000 7f401f7fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4020101b10 0x7f402019ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:42124/0 (socket says 172.21.15.160:42124) 2024-03-07T07:39:10.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.962+0000 7f401f7fe640 1 -- 172.21.15.160:0/1504598847 learned_addr learned my addr 172.21.15.160:0/1504598847 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:39:10.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.962+0000 7f401e7fc640 1 --2- 172.21.15.160:0/1504598847 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4020101200 0x7f402019c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:10.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.962+0000 7f401effd640 1 --2- 172.21.15.160:0/1504598847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4020100840 0x7f40201002a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:10.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.962+0000 7f401f7fe640 1 -- 172.21.15.160:0/1504598847 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4020101200 msgr2=0x7f402019c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:10.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.962+0000 7f401f7fe640 1 --2- 172.21.15.160:0/1504598847 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4020101200 0x7f402019c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:10.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.962+0000 7f401f7fe640 1 -- 172.21.15.160:0/1504598847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4020100840 msgr2=0x7f40201002a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:10.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.962+0000 7f401f7fe640 1 --2- 172.21.15.160:0/1504598847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4020100840 0x7f40201002a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:10.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.962+0000 7f401f7fe640 1 -- 172.21.15.160:0/1504598847 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f402019f610 con 0x7f4020101b10 2024-03-07T07:39:10.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.962+0000 7f401e7fc640 1 --2- 172.21.15.160:0/1504598847 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4020101200 0x7f402019c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:39:10.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.962+0000 7f401f7fe640 1 --2- 172.21.15.160:0/1504598847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4020101b10 0x7f402019ed00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f401800eb10 tx=0x7f401800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:10.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.962+0000 7f4007fff640 1 -- 172.21.15.160:0/1504598847 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4018004b20 con 0x7f4020101b10 2024-03-07T07:39:10.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.962+0000 7f4007fff640 1 -- 172.21.15.160:0/1504598847 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4018004cc0 con 0x7f4020101b10 2024-03-07T07:39:10.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.962+0000 7f40252c7640 1 -- 172.21.15.160:0/1504598847 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f402019f8a0 con 0x7f4020101b10 2024-03-07T07:39:10.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.962+0000 7f4007fff640 1 -- 172.21.15.160:0/1504598847 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4018010760 con 0x7f4020101b10 2024-03-07T07:39:10.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.962+0000 7f40252c7640 1 -- 172.21.15.160:0/1504598847 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f402019fdf0 con 0x7f4020101b10 2024-03-07T07:39:10.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.963+0000 7f40252c7640 1 -- 172.21.15.160:0/1504598847 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4020107980 con 0x7f4020101b10 2024-03-07T07:39:10.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.964+0000 7f4007fff640 1 -- 172.21.15.160:0/1504598847 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f4018005020 con 0x7f4020101b10 2024-03-07T07:39:10.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.965+0000 7f4007fff640 1 --2- 172.21.15.160:0/1504598847 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f40080a1d80 0x7f40080a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:10.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.965+0000 7f401effd640 1 --2- 172.21.15.160:0/1504598847 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f40080a1d80 0x7f40080a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:10.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.965+0000 7f4007fff640 1 -- 172.21.15.160:0/1504598847 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f40180ce010 con 0x7f4020101b10 2024-03-07T07:39:10.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.965+0000 7f401effd640 1 --2- 172.21.15.160:0/1504598847 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f40080a1d80 0x7f40080a4240 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f401002c170 tx=0x7f401002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:10.969 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:10.969+0000 7f4007fff640 1 -- 172.21.15.160:0/1504598847 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f401809b8e0 con 0x7f4020101b10 2024-03-07T07:39:11.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:10 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:11.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:10 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:11.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:10 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:11.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:10 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:11.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:10 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:11.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:10 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:11.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:10 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:39:11.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:10 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:39:11.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:10 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:11.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:10 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:39:11.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:10 smithi178 ceph-mon[36650]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:39:11.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:10 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/2926243063' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:39:11.143 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.143+0000 7f40252c7640 1 -- 172.21.15.160:0/1504598847 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f402002d230 con 0x7f4020101b10 2024-03-07T07:39:11.144 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.143+0000 7f4007fff640 1 -- 172.21.15.160:0/1504598847 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f401809b1e0 con 0x7f4020101b10 2024-03-07T07:39:11.144 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 1 2024-03-07T07:39:11.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.145+0000 7f40252c7640 1 -- 172.21.15.160:0/1504598847 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f40080a1d80 msgr2=0x7f40080a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:11.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.145+0000 7f40252c7640 1 --2- 172.21.15.160:0/1504598847 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f40080a1d80 0x7f40080a4240 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f401002c170 tx=0x7f401002df90 comp rx=0 tx=0).stop 2024-03-07T07:39:11.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.145+0000 7f40252c7640 1 -- 172.21.15.160:0/1504598847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4020101b10 msgr2=0x7f402019ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:11.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.145+0000 7f40252c7640 1 --2- 172.21.15.160:0/1504598847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4020101b10 0x7f402019ed00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f401800eb10 tx=0x7f401800c2d0 comp rx=0 tx=0).stop 2024-03-07T07:39:11.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.146+0000 7f40252c7640 1 -- 172.21.15.160:0/1504598847 shutdown_connections 2024-03-07T07:39:11.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.146+0000 7f40252c7640 1 --2- 172.21.15.160:0/1504598847 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f40080a1d80 0x7f40080a4240 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:11.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.146+0000 7f40252c7640 1 --2- 172.21.15.160:0/1504598847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4020101b10 0x7f402019ed00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:11.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.146+0000 7f40252c7640 1 --2- 172.21.15.160:0/1504598847 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4020101200 0x7f402019c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:11.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.146+0000 7f40252c7640 1 --2- 172.21.15.160:0/1504598847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4020100840 0x7f40201002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:11.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.146+0000 7f40252c7640 1 -- 172.21.15.160:0/1504598847 >> 172.21.15.160:0/1504598847 conn(0x7f40200fb290 msgr2=0x7f4020104b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:11.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.146+0000 7f40252c7640 1 -- 172.21.15.160:0/1504598847 shutdown_connections 2024-03-07T07:39:11.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.146+0000 7f40252c7640 1 -- 172.21.15.160:0/1504598847 wait complete. 2024-03-07T07:39:11.157 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-07T07:39:11.282 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.276+0000 7f21a44dd640 1 -- 172.21.15.160:0/2849165853 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f219c1073c0 msgr2=0x7f219c1097b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:11.282 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.276+0000 7f21a44dd640 1 --2- 172.21.15.160:0/2849165853 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f219c1073c0 0x7f219c1097b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2194009ca0 tx=0x7f219402c0f0 comp rx=0 tx=0).stop 2024-03-07T07:39:11.282 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.276+0000 7f21a44dd640 1 -- 172.21.15.160:0/2849165853 shutdown_connections 2024-03-07T07:39:11.282 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.276+0000 7f21a44dd640 1 --2- 172.21.15.160:0/2849165853 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f219c1073c0 0x7f219c1097b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:11.282 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.276+0000 7f21a44dd640 1 --2- 172.21.15.160:0/2849165853 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f219c104a90 0x7f219c106e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:11.282 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.276+0000 7f21a44dd640 1 --2- 172.21.15.160:0/2849165853 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f219c102160 0x7f219c104550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:11.282 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.276+0000 7f21a44dd640 1 -- 172.21.15.160:0/2849165853 >> 172.21.15.160:0/2849165853 conn(0x7f219c0fbaf0 msgr2=0x7f219c0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:11.282 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.276+0000 7f21a44dd640 1 -- 172.21.15.160:0/2849165853 shutdown_connections 2024-03-07T07:39:11.282 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.276+0000 7f21a44dd640 1 -- 172.21.15.160:0/2849165853 wait complete. 2024-03-07T07:39:11.282 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.276+0000 7f21a44dd640 1 Processor -- start 2024-03-07T07:39:11.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.276+0000 7f21a44dd640 1 -- start start 2024-03-07T07:39:11.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.276+0000 7f21a44dd640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f219c102160 0x7f219c197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:11.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.276+0000 7f21a44dd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f219c104a90 0x7f219c197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:11.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.276+0000 7f21a44dd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f219c1073c0 0x7f219c19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:11.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.276+0000 7f21a44dd640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219c10d310 con 0x7f219c104a90 2024-03-07T07:39:11.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.276+0000 7f21a44dd640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219c10d190 con 0x7f219c102160 2024-03-07T07:39:11.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.276+0000 7f21a44dd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219c10d490 con 0x7f219c1073c0 2024-03-07T07:39:11.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.277+0000 7f21a1a78640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f219c104a90 0x7f219c197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:11.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.277+0000 7f21a1a78640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f219c104a90 0x7f219c197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:42144/0 (socket says 172.21.15.160:42144) 2024-03-07T07:39:11.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.277+0000 7f21a2279640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f219c102160 0x7f219c197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:11.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.277+0000 7f21a1a78640 1 -- 172.21.15.160:0/1642131720 learned_addr learned my addr 172.21.15.160:0/1642131720 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:39:11.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.277+0000 7f21a2a7a640 1 --2- 172.21.15.160:0/1642131720 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f219c1073c0 0x7f219c19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:11.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.277+0000 7f21a2279640 1 -- 172.21.15.160:0/1642131720 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f219c1073c0 msgr2=0x7f219c19ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:11.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.277+0000 7f21a2279640 1 --2- 172.21.15.160:0/1642131720 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f219c1073c0 0x7f219c19ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:11.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.277+0000 7f21a2279640 1 -- 172.21.15.160:0/1642131720 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f219c104a90 msgr2=0x7f219c197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:11.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.277+0000 7f21a2279640 1 --2- 172.21.15.160:0/1642131720 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f219c104a90 0x7f219c197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:11.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.277+0000 7f21a2279640 1 -- 172.21.15.160:0/1642131720 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f219c19f5a0 con 0x7f219c102160 2024-03-07T07:39:11.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.277+0000 7f21a1a78640 1 --2- 172.21.15.160:0/1642131720 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f219c104a90 0x7f219c197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:39:11.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.277+0000 7f21a2a7a640 1 --2- 172.21.15.160:0/1642131720 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f219c1073c0 0x7f219c19ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:39:11.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.277+0000 7f21a2279640 1 --2- 172.21.15.160:0/1642131720 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f219c102160 0x7f219c197980 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f219800e9f0 tx=0x7f219800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:11.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.277+0000 7f21937fe640 1 -- 172.21.15.160:0/1642131720 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219800ccc0 con 0x7f219c102160 2024-03-07T07:39:11.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.277+0000 7f21a44dd640 1 -- 172.21.15.160:0/1642131720 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f219c19f890 con 0x7f219c102160 2024-03-07T07:39:11.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.277+0000 7f21937fe640 1 -- 172.21.15.160:0/1642131720 <== mon.2 v2:172.21.15.177:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2198010910 con 0x7f219c102160 2024-03-07T07:39:11.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.277+0000 7f21937fe640 1 -- 172.21.15.160:0/1642131720 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2198018960 con 0x7f219c102160 2024-03-07T07:39:11.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.277+0000 7f21a44dd640 1 -- 172.21.15.160:0/1642131720 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f219c19fe30 con 0x7f219c102160 2024-03-07T07:39:11.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.279+0000 7f21a44dd640 1 -- 172.21.15.160:0/1642131720 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f219c0fd5e0 con 0x7f219c102160 2024-03-07T07:39:11.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.279+0000 7f21937fe640 1 -- 172.21.15.160:0/1642131720 <== mon.2 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f2198010ac0 con 0x7f219c102160 2024-03-07T07:39:11.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.280+0000 7f21937fe640 1 --2- 172.21.15.160:0/1642131720 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f21880a1d80 0x7f21880a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:11.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.280+0000 7f21a1a78640 1 --2- 172.21.15.160:0/1642131720 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f21880a1d80 0x7f21880a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:11.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.280+0000 7f21937fe640 1 -- 172.21.15.160:0/1642131720 <== mon.2 v2:172.21.15.177:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f21980cdd30 con 0x7f219c102160 2024-03-07T07:39:11.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.281+0000 7f21a1a78640 1 --2- 172.21.15.160:0/1642131720 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f21880a1d80 0x7f21880a4240 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f219c198e90 tx=0x7f218c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:11.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.285+0000 7f21937fe640 1 -- 172.21.15.160:0/1642131720 <== mon.2 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f219809af10 con 0x7f219c102160 2024-03-07T07:39:11.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.518+0000 7f21a44dd640 1 -- 172.21.15.160:0/1642131720 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f219c062260 con 0x7f219c102160 2024-03-07T07:39:11.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.518+0000 7f21937fe640 1 -- 172.21.15.160:0/1642131720 <== mon.2 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) v1 ==== 84+0+65 (secure 0 0 0) 0x7f219809efb0 con 0x7f219c102160 2024-03-07T07:39:11.519 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:39:11.519 INFO:teuthology.orchestra.run.smithi160.stdout:[ 2024-03-07T07:39:11.519 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:39:11.519 INFO:teuthology.orchestra.run.smithi160.stdout: "poolnum": 1, 2024-03-07T07:39:11.519 INFO:teuthology.orchestra.run.smithi160.stdout: "poolname": ".mgr" 2024-03-07T07:39:11.519 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:39:11.519 INFO:teuthology.orchestra.run.smithi160.stdout:] 2024-03-07T07:39:11.522 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.521+0000 7f21a44dd640 1 -- 172.21.15.160:0/1642131720 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f21880a1d80 msgr2=0x7f21880a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:11.522 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.521+0000 7f21a44dd640 1 --2- 172.21.15.160:0/1642131720 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f21880a1d80 0x7f21880a4240 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f219c198e90 tx=0x7f218c005f90 comp rx=0 tx=0).stop 2024-03-07T07:39:11.522 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.521+0000 7f21a44dd640 1 -- 172.21.15.160:0/1642131720 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f219c102160 msgr2=0x7f219c197980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:11.522 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.521+0000 7f21a44dd640 1 --2- 172.21.15.160:0/1642131720 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f219c102160 0x7f219c197980 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f219800e9f0 tx=0x7f219800eeb0 comp rx=0 tx=0).stop 2024-03-07T07:39:11.522 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.521+0000 7f21a44dd640 1 -- 172.21.15.160:0/1642131720 shutdown_connections 2024-03-07T07:39:11.522 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.521+0000 7f21a44dd640 1 --2- 172.21.15.160:0/1642131720 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f21880a1d80 0x7f21880a4240 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:11.522 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.521+0000 7f21a44dd640 1 --2- 172.21.15.160:0/1642131720 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f219c1073c0 0x7f219c19ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:11.522 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.521+0000 7f21a44dd640 1 --2- 172.21.15.160:0/1642131720 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f219c104a90 0x7f219c197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:11.522 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.521+0000 7f21a44dd640 1 --2- 172.21.15.160:0/1642131720 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f219c102160 0x7f219c197980 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:11.522 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.521+0000 7f21a44dd640 1 -- 172.21.15.160:0/1642131720 >> 172.21.15.160:0/1642131720 conn(0x7f219c0fbaf0 msgr2=0x7f219c102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:11.522 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.521+0000 7f21a44dd640 1 -- 172.21.15.160:0/1642131720 shutdown_connections 2024-03-07T07:39:11.522 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.521+0000 7f21a44dd640 1 -- 172.21.15.160:0/1642131720 wait complete. 2024-03-07T07:39:11.534 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-07T07:39:11.534 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-07T07:39:11.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.648+0000 7f8048914640 1 -- 172.21.15.160:0/592787970 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8044102860 msgr2=0x7f8044102ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:11.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.648+0000 7f8048914640 1 --2- 172.21.15.160:0/592787970 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8044102860 0x7f8044102ce0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f802c009900 tx=0x7f802c02bc30 comp rx=0 tx=0).stop 2024-03-07T07:39:11.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.648+0000 7f8048914640 1 -- 172.21.15.160:0/592787970 shutdown_connections 2024-03-07T07:39:11.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.648+0000 7f8048914640 1 --2- 172.21.15.160:0/592787970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8044103220 0x7f8044109ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:11.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.648+0000 7f8048914640 1 --2- 172.21.15.160:0/592787970 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8044102860 0x7f8044102ce0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:11.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.648+0000 7f8048914640 1 --2- 172.21.15.160:0/592787970 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8044101670 0x7f8044101a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:11.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.648+0000 7f8048914640 1 -- 172.21.15.160:0/592787970 >> 172.21.15.160:0/592787970 conn(0x7f80440fcc00 msgr2=0x7f80440ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:11.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.648+0000 7f8048914640 1 -- 172.21.15.160:0/592787970 shutdown_connections 2024-03-07T07:39:11.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.648+0000 7f8048914640 1 -- 172.21.15.160:0/592787970 wait complete. 2024-03-07T07:39:11.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.648+0000 7f8048914640 1 Processor -- start 2024-03-07T07:39:11.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.649+0000 7f8048914640 1 -- start start 2024-03-07T07:39:11.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.649+0000 7f8048914640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8044101670 0x7f80441979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:11.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.649+0000 7f8048914640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8044102860 0x7f8044197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:11.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.649+0000 7f8048914640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8044103220 0x7f804419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:11.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.649+0000 7f8048914640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804410d300 con 0x7f8044102860 2024-03-07T07:39:11.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.649+0000 7f8048914640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804410d180 con 0x7f8044103220 2024-03-07T07:39:11.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.649+0000 7f8048914640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804410d480 con 0x7f8044101670 2024-03-07T07:39:11.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.649+0000 7f803bfff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8044102860 0x7f8044197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:11.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.649+0000 7f803bfff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8044102860 0x7f8044197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:42170/0 (socket says 172.21.15.160:42170) 2024-03-07T07:39:11.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.649+0000 7f803bfff640 1 -- 172.21.15.160:0/2043254291 learned_addr learned my addr 172.21.15.160:0/2043254291 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:39:11.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.649+0000 7f804259c640 1 --2- 172.21.15.160:0/2043254291 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8044101670 0x7f80441979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:11.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.649+0000 7f8042d9d640 1 --2- 172.21.15.160:0/2043254291 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8044103220 0x7f804419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:11.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.649+0000 7f803bfff640 1 -- 172.21.15.160:0/2043254291 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8044101670 msgr2=0x7f80441979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:11.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.649+0000 7f803bfff640 1 --2- 172.21.15.160:0/2043254291 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8044101670 0x7f80441979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:11.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.649+0000 7f803bfff640 1 -- 172.21.15.160:0/2043254291 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8044103220 msgr2=0x7f804419ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:11.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.649+0000 7f803bfff640 1 --2- 172.21.15.160:0/2043254291 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8044103220 0x7f804419ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:11.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.649+0000 7f803bfff640 1 -- 172.21.15.160:0/2043254291 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f804419f600 con 0x7f8044102860 2024-03-07T07:39:11.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.649+0000 7f8042d9d640 1 --2- 172.21.15.160:0/2043254291 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8044103220 0x7f804419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T07:39:11.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.649+0000 7f803bfff640 1 --2- 172.21.15.160:0/2043254291 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8044102860 0x7f8044197f20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f802c009230 tx=0x7f802c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:11.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.649+0000 7f803b7fe640 1 -- 172.21.15.160:0/2043254291 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802c041070 con 0x7f8044102860 2024-03-07T07:39:11.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.649+0000 7f8048914640 1 -- 172.21.15.160:0/2043254291 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f804419f890 con 0x7f8044102860 2024-03-07T07:39:11.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.649+0000 7f803b7fe640 1 -- 172.21.15.160:0/2043254291 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f802c0054e0 con 0x7f8044102860 2024-03-07T07:39:11.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.649+0000 7f803b7fe640 1 -- 172.21.15.160:0/2043254291 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802c0346a0 con 0x7f8044102860 2024-03-07T07:39:11.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.650+0000 7f8048914640 1 -- 172.21.15.160:0/2043254291 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f804419fdd0 con 0x7f8044102860 2024-03-07T07:39:11.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.650+0000 7f8048914640 1 -- 172.21.15.160:0/2043254291 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80441089b0 con 0x7f8044102860 2024-03-07T07:39:11.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.651+0000 7f803b7fe640 1 -- 172.21.15.160:0/2043254291 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f802c044070 con 0x7f8044102860 2024-03-07T07:39:11.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.652+0000 7f803b7fe640 1 --2- 172.21.15.160:0/2043254291 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f80300a1d80 0x7f80300a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:11.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.652+0000 7f804259c640 1 --2- 172.21.15.160:0/2043254291 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f80300a1d80 0x7f80300a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:11.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.652+0000 7f803b7fe640 1 -- 172.21.15.160:0/2043254291 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f802c0ef610 con 0x7f8044102860 2024-03-07T07:39:11.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.652+0000 7f804259c640 1 --2- 172.21.15.160:0/2043254291 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f80300a1d80 0x7f80300a4240 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f80441026c0 tx=0x7f803400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:11.656 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.656+0000 7f803b7fe640 1 -- 172.21.15.160:0/2043254291 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f802c0bcec0 con 0x7f8044102860 2024-03-07T07:39:11.839 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:11.839+0000 7f8048914640 1 -- 172.21.15.160:0/2043254291 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f804402d230 con 0x7f8044102860 2024-03-07T07:39:12.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:11 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/1352572891' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:39:12.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:11 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/1504598847' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-07T07:39:12.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:11 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/1642131720' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-07T07:39:12.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:11 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:12.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:11 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:12.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:11 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:12.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:11 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:12.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:11 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:12.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:11 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:12.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:11 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:39:12.108 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:11 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:39:12.108 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:11 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:12.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:11 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/1352572891' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:39:12.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:11 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/1504598847' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-07T07:39:12.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:11 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/1642131720' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-07T07:39:12.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:11 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:12.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:11 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:12.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:11 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:12.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:11 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:12.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:11 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:12.179 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:11 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:12.179 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:11 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:39:12.179 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:11 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:39:12.179 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:11 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:12.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:11 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/1352572891' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:39:12.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:11 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/1504598847' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-07T07:39:12.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:11 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/1642131720' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-07T07:39:12.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:11 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:12.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:11 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:12.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:11 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:12.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:11 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:12.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:11 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:12.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:11 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:12.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:11 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:39:12.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:11 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:39:12.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:11 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:39:12.812 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.812+0000 7f803b7fe640 1 -- 172.21.15.160:0/2043254291 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) v1 ==== 138+0+0 (secure 0 0 0) 0x7f802c0bc7c0 con 0x7f8044102860 2024-03-07T07:39:12.812 INFO:teuthology.orchestra.run.smithi160.stderr:pool 'cephfs_metadata' created 2024-03-07T07:39:12.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.814+0000 7f8048914640 1 -- 172.21.15.160:0/2043254291 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f80300a1d80 msgr2=0x7f80300a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:12.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.814+0000 7f8048914640 1 --2- 172.21.15.160:0/2043254291 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f80300a1d80 0x7f80300a4240 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f80441026c0 tx=0x7f803400a2d0 comp rx=0 tx=0).stop 2024-03-07T07:39:12.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.814+0000 7f8048914640 1 -- 172.21.15.160:0/2043254291 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8044102860 msgr2=0x7f8044197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:12.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.814+0000 7f8048914640 1 --2- 172.21.15.160:0/2043254291 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8044102860 0x7f8044197f20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f802c009230 tx=0x7f802c004040 comp rx=0 tx=0).stop 2024-03-07T07:39:12.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.815+0000 7f8048914640 1 -- 172.21.15.160:0/2043254291 shutdown_connections 2024-03-07T07:39:12.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.815+0000 7f8048914640 1 --2- 172.21.15.160:0/2043254291 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f80300a1d80 0x7f80300a4240 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:12.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.815+0000 7f8048914640 1 --2- 172.21.15.160:0/2043254291 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8044103220 0x7f804419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:12.816 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.815+0000 7f8048914640 1 --2- 172.21.15.160:0/2043254291 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8044102860 0x7f8044197f20 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:12.816 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.815+0000 7f8048914640 1 --2- 172.21.15.160:0/2043254291 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8044101670 0x7f80441979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:12.816 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.815+0000 7f8048914640 1 -- 172.21.15.160:0/2043254291 >> 172.21.15.160:0/2043254291 conn(0x7f80440fcc00 msgr2=0x7f8044105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:12.816 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.815+0000 7f8048914640 1 -- 172.21.15.160:0/2043254291 shutdown_connections 2024-03-07T07:39:12.816 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.815+0000 7f8048914640 1 -- 172.21.15.160:0/2043254291 wait complete. 2024-03-07T07:39:12.833 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-07T07:39:12.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.945+0000 7ff19c7c9640 1 -- 172.21.15.160:0/3981813433 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1941032f0 msgr2=0x7ff194109b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:12.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.945+0000 7ff19c7c9640 1 --2- 172.21.15.160:0/3981813433 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1941032f0 0x7ff194109b80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff18c009cd0 tx=0x7ff18c02c0f0 comp rx=0 tx=0).stop 2024-03-07T07:39:12.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.945+0000 7ff19c7c9640 1 -- 172.21.15.160:0/3981813433 shutdown_connections 2024-03-07T07:39:12.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.945+0000 7ff19c7c9640 1 --2- 172.21.15.160:0/3981813433 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1941032f0 0x7ff194109b80 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:12.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.945+0000 7ff19c7c9640 1 --2- 172.21.15.160:0/3981813433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff194102930 0x7ff194102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:12.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.945+0000 7ff19c7c9640 1 --2- 172.21.15.160:0/3981813433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1940ff960 0x7ff1940ffd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:12.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.945+0000 7ff19c7c9640 1 -- 172.21.15.160:0/3981813433 >> 172.21.15.160:0/3981813433 conn(0x7ff1940fb290 msgr2=0x7ff1940fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:12.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.945+0000 7ff19c7c9640 1 -- 172.21.15.160:0/3981813433 shutdown_connections 2024-03-07T07:39:12.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.945+0000 7ff19c7c9640 1 -- 172.21.15.160:0/3981813433 wait complete. 2024-03-07T07:39:12.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.946+0000 7ff19c7c9640 1 Processor -- start 2024-03-07T07:39:12.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.946+0000 7ff19c7c9640 1 -- start start 2024-03-07T07:39:12.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.946+0000 7ff19c7c9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1940ff960 0x7ff194197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:12.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.946+0000 7ff19c7c9640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff194102930 0x7ff194197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:12.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.946+0000 7ff19c7c9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1941032f0 0x7ff19419eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:12.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.946+0000 7ff19c7c9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19410d340 con 0x7ff1940ff960 2024-03-07T07:39:12.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.946+0000 7ff19c7c9640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19410d1c0 con 0x7ff194102930 2024-03-07T07:39:12.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.946+0000 7ff19c7c9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19410d4c0 con 0x7ff1941032f0 2024-03-07T07:39:12.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.946+0000 7ff19a565640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1940ff960 0x7ff194197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:12.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.946+0000 7ff19a565640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1940ff960 0x7ff194197920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:42184/0 (socket says 172.21.15.160:42184) 2024-03-07T07:39:12.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.946+0000 7ff19a565640 1 -- 172.21.15.160:0/3496954906 learned_addr learned my addr 172.21.15.160:0/3496954906 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:39:12.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.946+0000 7ff199d64640 1 --2- 172.21.15.160:0/3496954906 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff194102930 0x7ff194197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:12.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.946+0000 7ff19a565640 1 -- 172.21.15.160:0/3496954906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1941032f0 msgr2=0x7ff19419eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:12.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.946+0000 7ff19a565640 1 --2- 172.21.15.160:0/3496954906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1941032f0 0x7ff19419eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:12.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.946+0000 7ff19a565640 1 -- 172.21.15.160:0/3496954906 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff194102930 msgr2=0x7ff194197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:12.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.946+0000 7ff19a565640 1 --2- 172.21.15.160:0/3496954906 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff194102930 0x7ff194197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:12.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.946+0000 7ff19a565640 1 -- 172.21.15.160:0/3496954906 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff19419f540 con 0x7ff1940ff960 2024-03-07T07:39:12.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.946+0000 7ff199d64640 1 --2- 172.21.15.160:0/3496954906 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff194102930 0x7ff194197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:39:12.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.947+0000 7ff19a565640 1 --2- 172.21.15.160:0/3496954906 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1940ff960 0x7ff194197920 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff19000c970 tx=0x7ff19000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:12.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.947+0000 7ff18b7fe640 1 -- 172.21.15.160:0/3496954906 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff190004220 con 0x7ff1940ff960 2024-03-07T07:39:12.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.947+0000 7ff19c7c9640 1 -- 172.21.15.160:0/3496954906 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff19419f830 con 0x7ff1940ff960 2024-03-07T07:39:12.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.947+0000 7ff18b7fe640 1 -- 172.21.15.160:0/3496954906 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff1900043c0 con 0x7ff1940ff960 2024-03-07T07:39:12.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.947+0000 7ff18b7fe640 1 -- 172.21.15.160:0/3496954906 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19000f450 con 0x7ff1940ff960 2024-03-07T07:39:12.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.947+0000 7ff19c7c9640 1 -- 172.21.15.160:0/3496954906 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff19419fdd0 con 0x7ff1940ff960 2024-03-07T07:39:12.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.948+0000 7ff19c7c9640 1 -- 172.21.15.160:0/3496954906 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff194108a00 con 0x7ff1940ff960 2024-03-07T07:39:12.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.949+0000 7ff18b7fe640 1 -- 172.21.15.160:0/3496954906 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7ff1900079b0 con 0x7ff1940ff960 2024-03-07T07:39:12.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.950+0000 7ff18b7fe640 1 --2- 172.21.15.160:0/3496954906 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7ff1800a1d80 0x7ff1800a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:12.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.950+0000 7ff199d64640 1 --2- 172.21.15.160:0/3496954906 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7ff1800a1d80 0x7ff1800a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:12.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.950+0000 7ff18b7fe640 1 -- 172.21.15.160:0/3496954906 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7497+0+0 (secure 0 0 0) 0x7ff1900cdd20 con 0x7ff1940ff960 2024-03-07T07:39:12.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.950+0000 7ff199d64640 1 --2- 172.21.15.160:0/3496954906 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7ff1800a1d80 0x7ff1800a4240 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff1840058c0 tx=0x7ff184005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:12.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:12.954+0000 7ff18b7fe640 1 -- 172.21.15.160:0/3496954906 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff190014030 con 0x7ff1940ff960 2024-03-07T07:39:13.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:12 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/2043254291' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-07T07:39:13.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:12 smithi177 ceph-mon[36578]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:39:13.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.130+0000 7ff19c7c9640 1 -- 172.21.15.160:0/3496954906 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff19402d3f0 con 0x7ff1940ff960 2024-03-07T07:39:13.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:12 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/2043254291' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-07T07:39:13.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:12 smithi160 ceph-mon[31963]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:39:13.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:12 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/2043254291' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-07T07:39:13.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:12 smithi178 ceph-mon[36650]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:39:13.814 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.814+0000 7ff18b7fe640 1 -- 172.21.15.160:0/3496954906 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v83) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff19009b3e0 con 0x7ff1940ff960 2024-03-07T07:39:13.815 INFO:teuthology.orchestra.run.smithi160.stderr:pool 'cephfs_data' created 2024-03-07T07:39:13.817 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.817+0000 7ff19c7c9640 1 -- 172.21.15.160:0/3496954906 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7ff1800a1d80 msgr2=0x7ff1800a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:13.817 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.817+0000 7ff19c7c9640 1 --2- 172.21.15.160:0/3496954906 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7ff1800a1d80 0x7ff1800a4240 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff1840058c0 tx=0x7ff184005850 comp rx=0 tx=0).stop 2024-03-07T07:39:13.817 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.817+0000 7ff19c7c9640 1 -- 172.21.15.160:0/3496954906 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1940ff960 msgr2=0x7ff194197920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:13.817 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.817+0000 7ff19c7c9640 1 --2- 172.21.15.160:0/3496954906 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1940ff960 0x7ff194197920 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff19000c970 tx=0x7ff19000ce30 comp rx=0 tx=0).stop 2024-03-07T07:39:13.818 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.817+0000 7ff19c7c9640 1 -- 172.21.15.160:0/3496954906 shutdown_connections 2024-03-07T07:39:13.818 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.817+0000 7ff19c7c9640 1 --2- 172.21.15.160:0/3496954906 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7ff1800a1d80 0x7ff1800a4240 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:13.818 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.817+0000 7ff19c7c9640 1 --2- 172.21.15.160:0/3496954906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1941032f0 0x7ff19419eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:13.818 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.817+0000 7ff19c7c9640 1 --2- 172.21.15.160:0/3496954906 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff194102930 0x7ff194197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:13.818 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.817+0000 7ff19c7c9640 1 --2- 172.21.15.160:0/3496954906 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1940ff960 0x7ff194197920 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:13.818 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.817+0000 7ff19c7c9640 1 -- 172.21.15.160:0/3496954906 >> 172.21.15.160:0/3496954906 conn(0x7ff1940fb290 msgr2=0x7ff194105bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:13.818 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.817+0000 7ff19c7c9640 1 -- 172.21.15.160:0/3496954906 shutdown_connections 2024-03-07T07:39:13.818 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.817+0000 7ff19c7c9640 1 -- 172.21.15.160:0/3496954906 wait complete. 2024-03-07T07:39:13.838 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-07T07:39:13.946 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.943+0000 7f2a26283640 1 -- 172.21.15.160:0/2724386123 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a20107350 msgr2=0x7f2a201097a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:13.946 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.943+0000 7f2a26283640 1 --2- 172.21.15.160:0/2724386123 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a20107350 0x7f2a201097a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2a18009ca0 tx=0x7f2a1802c0f0 comp rx=0 tx=0).stop 2024-03-07T07:39:13.946 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.943+0000 7f2a26283640 1 -- 172.21.15.160:0/2724386123 shutdown_connections 2024-03-07T07:39:13.946 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.943+0000 7f2a26283640 1 --2- 172.21.15.160:0/2724386123 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a20107350 0x7f2a201097a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:13.946 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.943+0000 7f2a26283640 1 --2- 172.21.15.160:0/2724386123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a20104a20 0x7f2a20106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:13.946 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.943+0000 7f2a26283640 1 --2- 172.21.15.160:0/2724386123 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a201020f0 0x7f2a201044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:13.946 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.943+0000 7f2a26283640 1 -- 172.21.15.160:0/2724386123 >> 172.21.15.160:0/2724386123 conn(0x7f2a200fbb00 msgr2=0x7f2a200fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:13.946 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.943+0000 7f2a26283640 1 -- 172.21.15.160:0/2724386123 shutdown_connections 2024-03-07T07:39:13.946 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.943+0000 7f2a26283640 1 -- 172.21.15.160:0/2724386123 wait complete. 2024-03-07T07:39:13.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.944+0000 7f2a26283640 1 Processor -- start 2024-03-07T07:39:13.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.944+0000 7f2a26283640 1 -- start start 2024-03-07T07:39:13.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.944+0000 7f2a26283640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a201020f0 0x7f2a20197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:13.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.944+0000 7f2a26283640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a20104a20 0x7f2a20197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:13.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.944+0000 7f2a26283640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a20107350 0x7f2a2019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:13.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.944+0000 7f2a26283640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2010d300 con 0x7f2a20107350 2024-03-07T07:39:13.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.944+0000 7f2a26283640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2010d180 con 0x7f2a20104a20 2024-03-07T07:39:13.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.944+0000 7f2a26283640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2010d480 con 0x7f2a201020f0 2024-03-07T07:39:13.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.944+0000 7f2a24820640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a20107350 0x7f2a2019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:13.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.944+0000 7f2a24820640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a20107350 0x7f2a2019ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:42204/0 (socket says 172.21.15.160:42204) 2024-03-07T07:39:13.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.944+0000 7f2a24820640 1 -- 172.21.15.160:0/2021713868 learned_addr learned my addr 172.21.15.160:0/2021713868 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:39:13.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.944+0000 7f2a1f7fe640 1 --2- 172.21.15.160:0/2021713868 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a20104a20 0x7f2a20197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:13.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.944+0000 7f2a1ffff640 1 --2- 172.21.15.160:0/2021713868 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a201020f0 0x7f2a20197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:13.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.944+0000 7f2a24820640 1 -- 172.21.15.160:0/2021713868 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a201020f0 msgr2=0x7f2a20197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:13.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.944+0000 7f2a24820640 1 --2- 172.21.15.160:0/2021713868 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a201020f0 0x7f2a20197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:13.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.944+0000 7f2a24820640 1 -- 172.21.15.160:0/2021713868 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a20104a20 msgr2=0x7f2a20197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:13.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.944+0000 7f2a24820640 1 --2- 172.21.15.160:0/2021713868 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a20104a20 0x7f2a20197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:13.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.944+0000 7f2a24820640 1 -- 172.21.15.160:0/2021713868 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a2019f590 con 0x7f2a20107350 2024-03-07T07:39:13.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.945+0000 7f2a24820640 1 --2- 172.21.15.160:0/2021713868 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a20107350 0x7f2a2019ef20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2a1802c700 tx=0x7f2a18032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:13.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.945+0000 7f2a1d7fa640 1 -- 172.21.15.160:0/2021713868 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a18039040 con 0x7f2a20107350 2024-03-07T07:39:13.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.945+0000 7f2a1d7fa640 1 -- 172.21.15.160:0/2021713868 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a18044070 con 0x7f2a20107350 2024-03-07T07:39:13.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.945+0000 7f2a26283640 1 -- 172.21.15.160:0/2021713868 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a2019f820 con 0x7f2a20107350 2024-03-07T07:39:13.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.945+0000 7f2a1d7fa640 1 -- 172.21.15.160:0/2021713868 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a180042a0 con 0x7f2a20107350 2024-03-07T07:39:13.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.945+0000 7f2a26283640 1 -- 172.21.15.160:0/2021713868 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a2019fd10 con 0x7f2a20107350 2024-03-07T07:39:13.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.946+0000 7f2a26283640 1 -- 172.21.15.160:0/2021713868 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a200fd5f0 con 0x7f2a20107350 2024-03-07T07:39:13.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.947+0000 7f2a1d7fa640 1 -- 172.21.15.160:0/2021713868 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f2a18003970 con 0x7f2a20107350 2024-03-07T07:39:13.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.947+0000 7f2a1d7fa640 1 --2- 172.21.15.160:0/2021713868 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2a080a1d80 0x7f2a080a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:13.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.947+0000 7f2a1ffff640 1 --2- 172.21.15.160:0/2021713868 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2a080a1d80 0x7f2a080a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:13.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.947+0000 7f2a1d7fa640 1 -- 172.21.15.160:0/2021713868 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f2a180f06c0 con 0x7f2a20107350 2024-03-07T07:39:13.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.948+0000 7f2a1ffff640 1 --2- 172.21.15.160:0/2021713868 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2a080a1d80 0x7f2a080a4240 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2a10005d00 tx=0x7f2a1000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:13.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:13.951+0000 7f2a1d7fa640 1 -- 172.21.15.160:0/2021713868 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f2a180bdd30 con 0x7f2a20107350 2024-03-07T07:39:14.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:13 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/2043254291' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-07T07:39:14.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:13 smithi177 ceph-mon[36578]: osdmap e82: 12 total, 12 up, 12 in 2024-03-07T07:39:14.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:13 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/3496954906' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-07T07:39:14.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.162+0000 7f2a26283640 1 -- 172.21.15.160:0/2021713868 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f2a2002d030 con 0x7f2a20107350 2024-03-07T07:39:14.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:13 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/2043254291' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-07T07:39:14.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:13 smithi160 ceph-mon[31963]: osdmap e82: 12 total, 12 up, 12 in 2024-03-07T07:39:14.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:13 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/3496954906' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-07T07:39:14.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:13 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/2043254291' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-07T07:39:14.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:13 smithi178 ceph-mon[36650]: osdmap e82: 12 total, 12 up, 12 in 2024-03-07T07:39:14.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:13 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/3496954906' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-07T07:39:14.829 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.829+0000 7f2a1d7fa640 1 -- 172.21.15.160:0/2021713868 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f2a180bd630 con 0x7f2a20107350 2024-03-07T07:39:14.829 INFO:teuthology.orchestra.run.smithi160.stderr:new fs with metadata pool 2 and data pool 3 2024-03-07T07:39:14.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.831+0000 7f2a26283640 1 -- 172.21.15.160:0/2021713868 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2a080a1d80 msgr2=0x7f2a080a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:14.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.831+0000 7f2a26283640 1 --2- 172.21.15.160:0/2021713868 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2a080a1d80 0x7f2a080a4240 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2a10005d00 tx=0x7f2a1000b410 comp rx=0 tx=0).stop 2024-03-07T07:39:14.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.831+0000 7f2a26283640 1 -- 172.21.15.160:0/2021713868 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a20107350 msgr2=0x7f2a2019ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:14.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.831+0000 7f2a26283640 1 --2- 172.21.15.160:0/2021713868 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a20107350 0x7f2a2019ef20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2a1802c700 tx=0x7f2a18032f20 comp rx=0 tx=0).stop 2024-03-07T07:39:14.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.831+0000 7f2a26283640 1 -- 172.21.15.160:0/2021713868 shutdown_connections 2024-03-07T07:39:14.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.831+0000 7f2a26283640 1 --2- 172.21.15.160:0/2021713868 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2a080a1d80 0x7f2a080a4240 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:14.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.831+0000 7f2a26283640 1 --2- 172.21.15.160:0/2021713868 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a20107350 0x7f2a2019ef20 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:14.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.831+0000 7f2a26283640 1 --2- 172.21.15.160:0/2021713868 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a20104a20 0x7f2a20197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:14.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.831+0000 7f2a26283640 1 --2- 172.21.15.160:0/2021713868 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a201020f0 0x7f2a20197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:14.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.831+0000 7f2a26283640 1 -- 172.21.15.160:0/2021713868 >> 172.21.15.160:0/2021713868 conn(0x7f2a200fbb00 msgr2=0x7f2a20102a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:14.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.831+0000 7f2a26283640 1 -- 172.21.15.160:0/2021713868 shutdown_connections 2024-03-07T07:39:14.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.831+0000 7f2a26283640 1 -- 172.21.15.160:0/2021713868 wait complete. 2024-03-07T07:39:14.845 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T07:39:14.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.946+0000 7fb1ca31d640 1 -- 172.21.15.160:0/2258705251 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1c4101680 msgr2=0x7fb1c4101aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:14.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.946+0000 7fb1ca31d640 1 --2- 172.21.15.160:0/2258705251 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1c4101680 0x7fb1c4101aa0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8009900 tx=0x7fb1b802bc30 comp rx=0 tx=0).stop 2024-03-07T07:39:14.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.946+0000 7fb1ca31d640 1 -- 172.21.15.160:0/2258705251 shutdown_connections 2024-03-07T07:39:14.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.946+0000 7fb1ca31d640 1 --2- 172.21.15.160:0/2258705251 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb1c4103230 0x7fb1c4109ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:14.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.946+0000 7fb1ca31d640 1 --2- 172.21.15.160:0/2258705251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1c4102870 0x7fb1c4102cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:14.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.946+0000 7fb1ca31d640 1 --2- 172.21.15.160:0/2258705251 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1c4101680 0x7fb1c4101aa0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:14.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.947+0000 7fb1ca31d640 1 -- 172.21.15.160:0/2258705251 >> 172.21.15.160:0/2258705251 conn(0x7fb1c40fcc10 msgr2=0x7fb1c40ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:14.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.947+0000 7fb1ca31d640 1 -- 172.21.15.160:0/2258705251 shutdown_connections 2024-03-07T07:39:14.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.947+0000 7fb1ca31d640 1 -- 172.21.15.160:0/2258705251 wait complete. 2024-03-07T07:39:14.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.947+0000 7fb1ca31d640 1 Processor -- start 2024-03-07T07:39:14.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.947+0000 7fb1ca31d640 1 -- start start 2024-03-07T07:39:14.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.947+0000 7fb1ca31d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb1c4101680 0x7fb1c4197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:14.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.947+0000 7fb1ca31d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1c4102870 0x7fb1c4197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:14.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.947+0000 7fb1ca31d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1c4103230 0x7fb1c419efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:14.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.947+0000 7fb1ca31d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c410d350 con 0x7fb1c4103230 2024-03-07T07:39:14.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.947+0000 7fb1ca31d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c410d1d0 con 0x7fb1c4101680 2024-03-07T07:39:14.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.947+0000 7fb1ca31d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c410d4d0 con 0x7fb1c4102870 2024-03-07T07:39:14.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.948+0000 7fb1c88ba640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1c4103230 0x7fb1c419efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:14.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.948+0000 7fb1c88ba640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1c4103230 0x7fb1c419efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:42220/0 (socket says 172.21.15.160:42220) 2024-03-07T07:39:14.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.948+0000 7fb1c88ba640 1 -- 172.21.15.160:0/793313048 learned_addr learned my addr 172.21.15.160:0/793313048 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:39:14.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.948+0000 7fb1c3fff640 1 --2- 172.21.15.160:0/793313048 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb1c4101680 0x7fb1c4197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:14.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.948+0000 7fb1c37fe640 1 --2- 172.21.15.160:0/793313048 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1c4102870 0x7fb1c4197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:14.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.948+0000 7fb1c88ba640 1 -- 172.21.15.160:0/793313048 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1c4102870 msgr2=0x7fb1c4197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:14.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.948+0000 7fb1c88ba640 1 --2- 172.21.15.160:0/793313048 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1c4102870 0x7fb1c4197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:14.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.948+0000 7fb1c88ba640 1 -- 172.21.15.160:0/793313048 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb1c4101680 msgr2=0x7fb1c4197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:14.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.948+0000 7fb1c88ba640 1 --2- 172.21.15.160:0/793313048 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb1c4101680 0x7fb1c4197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:14.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.948+0000 7fb1c88ba640 1 -- 172.21.15.160:0/793313048 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c419f650 con 0x7fb1c4103230 2024-03-07T07:39:14.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.948+0000 7fb1c37fe640 1 --2- 172.21.15.160:0/793313048 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1c4102870 0x7fb1c4197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:39:14.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.948+0000 7fb1c88ba640 1 --2- 172.21.15.160:0/793313048 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1c4103230 0x7fb1c419efe0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc00cad0 tx=0x7fb1bc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:14.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.948+0000 7fb1c17fa640 1 -- 172.21.15.160:0/793313048 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc00ae20 con 0x7fb1c4103230 2024-03-07T07:39:14.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.948+0000 7fb1c17fa640 1 -- 172.21.15.160:0/793313048 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb1bc005020 con 0x7fb1c4103230 2024-03-07T07:39:14.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.948+0000 7fb1ca31d640 1 -- 172.21.15.160:0/793313048 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1c419f940 con 0x7fb1c4103230 2024-03-07T07:39:14.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.948+0000 7fb1c17fa640 1 -- 172.21.15.160:0/793313048 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc00f6f0 con 0x7fb1c4103230 2024-03-07T07:39:14.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.948+0000 7fb1ca31d640 1 -- 172.21.15.160:0/793313048 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1c419fe90 con 0x7fb1c4103230 2024-03-07T07:39:14.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.949+0000 7fb1ca31d640 1 -- 172.21.15.160:0/793313048 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1c4108a20 con 0x7fb1c4103230 2024-03-07T07:39:14.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.950+0000 7fb1c17fa640 1 -- 172.21.15.160:0/793313048 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7fb1bc0051d0 con 0x7fb1c4103230 2024-03-07T07:39:14.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.951+0000 7fb1c17fa640 1 --2- 172.21.15.160:0/793313048 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fb1ac0a1e00 0x7fb1ac0a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:14.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.951+0000 7fb1c3fff640 1 --2- 172.21.15.160:0/793313048 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fb1ac0a1e00 0x7fb1ac0a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:14.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.951+0000 7fb1c17fa640 1 -- 172.21.15.160:0/793313048 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb1bc0cdc30 con 0x7fb1c4103230 2024-03-07T07:39:14.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.951+0000 7fb1c3fff640 1 --2- 172.21.15.160:0/793313048 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fb1ac0a1e00 0x7fb1ac0a42c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8000c00 tx=0x7fb1b8004f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:14.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:14.955+0000 7fb1c17fa640 1 -- 172.21.15.160:0/793313048 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb1bc014030 con 0x7fb1c4103230 2024-03-07T07:39:15.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:14 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/3496954906' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-07T07:39:15.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:14 smithi177 ceph-mon[36578]: osdmap e83: 12 total, 12 up, 12 in 2024-03-07T07:39:15.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:14 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/2021713868' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-07T07:39:15.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:14 smithi177 ceph-mon[36578]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-03-07T07:39:15.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:14 smithi177 ceph-mon[36578]: pgmap v286: 97 pgs: 31 creating+peering, 55 unknown, 11 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:39:15.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:14 smithi177 ceph-mon[36578]: osdmap e84: 12 total, 12 up, 12 in 2024-03-07T07:39:15.131 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.130+0000 7fb1ca31d640 1 -- 172.21.15.160:0/793313048 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1c4062260 con 0x7fb1c4103230 2024-03-07T07:39:15.132 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.132+0000 7fb1c17fa640 1 -- 172.21.15.160:0/793313048 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22255 (secure 0 0 0) 0x7fb1bc09b1b0 con 0x7fb1c4103230 2024-03-07T07:39:15.132 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:39:15.133 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":84,"fsid":"826f6582-dc54-11ee-95c6-87774f69a715","created":"2024-03-07T07:31:14.229288+0000","modified":"2024-03-07T07:39:14.163855+0000","last_up_change":"2024-03-07T07:38:20.776225+0000","last_in_change":"2024-03-07T07:38:00.508917+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T07:34:34.529424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T07:39:12.075432+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T07:39:13.367820+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bee66c67-fb6e-4717-9685-ca62587d5f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6803","nonce":1368482926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6805","nonce":1368482926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6809","nonce":1368482926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6807","nonce":1368482926}]},"public_addr":"172.21.15.160:6803/1368482926","cluster_addr":"172.21.15.160:6805/1368482926","heartbeat_back_addr":"172.21.15.160:6809/1368482926","heartbeat_front_addr":"172.21.15.160:6807/1368482926","state":["exists","up"]},{"osd":1,"uuid":"49236251-f750-424e-992b-bddf55762556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6801","nonce":2922391884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6803","nonce":2922391884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6807","nonce":2922391884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6805","nonce":2922391884}]},"public_addr":"172.21.15.177:6801/2922391884","cluster_addr":"172.21.15.177:6803/2922391884","heartbeat_back_addr":"172.21.15.177:6807/2922391884","heartbeat_front_addr":"172.21.15.177:6805/2922391884","state":["exists","up"]},{"osd":2,"uuid":"7a432b28-f7ee-43fe-9196-7aaeccfecf77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6801","nonce":1217901551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6803","nonce":1217901551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6807","nonce":1217901551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6805","nonce":1217901551}]},"public_addr":"172.21.15.178:6801/1217901551","cluster_addr":"172.21.15.178:6803/1217901551","heartbeat_back_addr":"172.21.15.178:6807/1217901551","heartbeat_front_addr":"172.21.15.178:6805/1217901551","state":["exists","up"]},{"osd":3,"uuid":"4ad22a20-f1be-4002-a48d-ecd3953e63f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6811","nonce":42543989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6813","nonce":42543989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6817","nonce":42543989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6815","nonce":42543989}]},"public_addr":"172.21.15.160:6811/42543989","cluster_addr":"172.21.15.160:6813/42543989","heartbeat_back_addr":"172.21.15.160:6817/42543989","heartbeat_front_addr":"172.21.15.160:6815/42543989","state":["exists","up"]},{"osd":4,"uuid":"2b65b104-ac82-4ce3-b6c5-11a399103045","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6809","nonce":88178553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6811","nonce":88178553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6815","nonce":88178553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6813","nonce":88178553}]},"public_addr":"172.21.15.177:6809/88178553","cluster_addr":"172.21.15.177:6811/88178553","heartbeat_back_addr":"172.21.15.177:6815/88178553","heartbeat_front_addr":"172.21.15.177:6813/88178553","state":["exists","up"]},{"osd":5,"uuid":"30a1e714-08ff-44e8-9264-9ef164604b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6809","nonce":964732891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6811","nonce":964732891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6815","nonce":964732891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6813","nonce":964732891}]},"public_addr":"172.21.15.178:6809/964732891","cluster_addr":"172.21.15.178:6811/964732891","heartbeat_back_addr":"172.21.15.178:6815/964732891","heartbeat_front_addr":"172.21.15.178:6813/964732891","state":["exists","up"]},{"osd":6,"uuid":"b1e1646e-f8a9-42b8-b62d-9d7b5d6a1ff4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6819","nonce":3251327001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6821","nonce":3251327001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6825","nonce":3251327001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6823","nonce":3251327001}]},"public_addr":"172.21.15.160:6819/3251327001","cluster_addr":"172.21.15.160:6821/3251327001","heartbeat_back_addr":"172.21.15.160:6825/3251327001","heartbeat_front_addr":"172.21.15.160:6823/3251327001","state":["exists","up"]},{"osd":7,"uuid":"e66b400b-9747-4f6d-b05b-db22468e2750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6817","nonce":1175500850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6819","nonce":1175500850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6823","nonce":1175500850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6821","nonce":1175500850}]},"public_addr":"172.21.15.177:6817/1175500850","cluster_addr":"172.21.15.177:6819/1175500850","heartbeat_back_addr":"172.21.15.177:6823/1175500850","heartbeat_front_addr":"172.21.15.177:6821/1175500850","state":["exists","up"]},{"osd":8,"uuid":"3ee0f2c4-fca4-49fc-a98b-07aa0fcba0e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6817","nonce":688015531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6819","nonce":688015531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6823","nonce":688015531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6821","nonce":688015531}]},"public_addr":"172.21.15.178:6817/688015531","cluster_addr":"172.21.15.178:6819/688015531","heartbeat_back_addr":"172.21.15.178:6823/688015531","heartbeat_front_addr":"172.21.15.178:6821/688015531","state":["exists","up"]},{"osd":9,"uuid":"856039d7-f7ee-4dbd-b019-0073e705ce0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6827","nonce":3265797851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6829","nonce":3265797851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6833","nonce":3265797851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6831","nonce":3265797851}]},"public_addr":"172.21.15.160:6827/3265797851","cluster_addr":"172.21.15.160:6829/3265797851","heartbeat_back_addr":"172.21.15.160:6833/3265797851","heartbeat_front_addr":"172.21.15.160:6831/3265797851","state":["exists","up"]},{"osd":10,"uuid":"bdc9899d-d720-4429-ad53-4c388a4dbe75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6825","nonce":2867473412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6827","nonce":2867473412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6831","nonce":2867473412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6829","nonce":2867473412}]},"public_addr":"172.21.15.177:6825/2867473412","cluster_addr":"172.21.15.177:6827/2867473412","heartbeat_back_addr":"172.21.15.177:6831/2867473412","heartbeat_front_addr":"172.21.15.177:6829/2867473412","state":["exists","up"]},{"osd":11,"uuid":"cc9adb54-ef49-47c6-90e3-fc03bc0f5cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6825","nonce":4068724731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6827","nonce":4068724731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6831","nonce":4068724731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6829","nonce":4068724731}]},"public_addr":"172.21.15.178:6825/4068724731","cluster_addr":"172.21.15.178:6827/4068724731","heartbeat_back_addr":"172.21.15.178:6831/4068724731","heartbeat_front_addr":"172.21.15.178:6829/4068724731","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:33:44.141585+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:07.685585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:31.044386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:56.308925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:35:21.582097+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:35:45.983540+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:36:11.530242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:36:36.844681+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:02.303437+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:27.629867+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:53.289495+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:38:18.536169+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/3027450612":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/874709722":"2024-03-08T07:32:08.190837+0000","172.21.15.160:6800/1130675225":"2024-03-08T07:31:41.166858+0000","172.21.15.160:0/3857193980":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/3861968297":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6800/3859174315":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/3013534743":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6801/3859174315":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/1310937667":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6801/1130675225":"2024-03-08T07:31:41.166858+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T07:39:15.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.134+0000 7fb1ca31d640 1 -- 172.21.15.160:0/793313048 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fb1ac0a1e00 msgr2=0x7fb1ac0a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:15.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.134+0000 7fb1ca31d640 1 --2- 172.21.15.160:0/793313048 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fb1ac0a1e00 0x7fb1ac0a42c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8000c00 tx=0x7fb1b8004f10 comp rx=0 tx=0).stop 2024-03-07T07:39:15.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.134+0000 7fb1ca31d640 1 -- 172.21.15.160:0/793313048 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1c4103230 msgr2=0x7fb1c419efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:15.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.134+0000 7fb1ca31d640 1 --2- 172.21.15.160:0/793313048 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1c4103230 0x7fb1c419efe0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc00cad0 tx=0x7fb1bc00cf90 comp rx=0 tx=0).stop 2024-03-07T07:39:15.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.134+0000 7fb1ca31d640 1 -- 172.21.15.160:0/793313048 shutdown_connections 2024-03-07T07:39:15.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.134+0000 7fb1ca31d640 1 --2- 172.21.15.160:0/793313048 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fb1ac0a1e00 0x7fb1ac0a42c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:15.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.134+0000 7fb1ca31d640 1 --2- 172.21.15.160:0/793313048 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1c4103230 0x7fb1c419efe0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:15.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.134+0000 7fb1ca31d640 1 --2- 172.21.15.160:0/793313048 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1c4102870 0x7fb1c4197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:15.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.134+0000 7fb1ca31d640 1 --2- 172.21.15.160:0/793313048 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb1c4101680 0x7fb1c4197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:15.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.134+0000 7fb1ca31d640 1 -- 172.21.15.160:0/793313048 >> 172.21.15.160:0/793313048 conn(0x7fb1c40fcc10 msgr2=0x7fb1c4105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:15.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.134+0000 7fb1ca31d640 1 -- 172.21.15.160:0/793313048 shutdown_connections 2024-03-07T07:39:15.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.134+0000 7fb1ca31d640 1 -- 172.21.15.160:0/793313048 wait complete. 2024-03-07T07:39:15.144 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:14 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/3496954906' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-07T07:39:15.144 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:14 smithi160 ceph-mon[31963]: osdmap e83: 12 total, 12 up, 12 in 2024-03-07T07:39:15.144 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:14 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/2021713868' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-07T07:39:15.145 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:14 smithi160 ceph-mon[31963]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-03-07T07:39:15.145 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:14 smithi160 ceph-mon[31963]: pgmap v286: 97 pgs: 31 creating+peering, 55 unknown, 11 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:39:15.145 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:14 smithi160 ceph-mon[31963]: osdmap e84: 12 total, 12 up, 12 in 2024-03-07T07:39:15.146 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T07:39:15.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.248+0000 7f19bfa0d640 1 -- 172.21.15.160:0/3806847177 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19b8101790 msgr2=0x7f19b8101c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:15.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.248+0000 7f19bfa0d640 1 --2- 172.21.15.160:0/3806847177 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19b8101790 0x7f19b8101c10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f19b4009900 tx=0x7f19b402bc30 comp rx=0 tx=0).stop 2024-03-07T07:39:15.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.248+0000 7f19bfa0d640 1 -- 172.21.15.160:0/3806847177 shutdown_connections 2024-03-07T07:39:15.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.248+0000 7f19bfa0d640 1 --2- 172.21.15.160:0/3806847177 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19b8102150 0x7f19b81089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:15.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.248+0000 7f19bfa0d640 1 --2- 172.21.15.160:0/3806847177 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19b8101790 0x7f19b8101c10 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:15.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.248+0000 7f19bfa0d640 1 --2- 172.21.15.160:0/3806847177 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19b81005a0 0x7f19b81009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:15.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.248+0000 7f19bfa0d640 1 -- 172.21.15.160:0/3806847177 >> 172.21.15.160:0/3806847177 conn(0x7f19b80fbb10 msgr2=0x7f19b80fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:15.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.248+0000 7f19bfa0d640 1 -- 172.21.15.160:0/3806847177 shutdown_connections 2024-03-07T07:39:15.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.248+0000 7f19bfa0d640 1 -- 172.21.15.160:0/3806847177 wait complete. 2024-03-07T07:39:15.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.249+0000 7f19bfa0d640 1 Processor -- start 2024-03-07T07:39:15.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.249+0000 7f19bfa0d640 1 -- start start 2024-03-07T07:39:15.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.249+0000 7f19bfa0d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19b81005a0 0x7f19b806a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:15.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.249+0000 7f19bfa0d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19b8101790 0x7f19b819c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:15.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.249+0000 7f19bfa0d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19b8102150 0x7f19b819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:15.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.249+0000 7f19bfa0d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b810c270 con 0x7f19b8101790 2024-03-07T07:39:15.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.249+0000 7f19bfa0d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b810c0f0 con 0x7f19b81005a0 2024-03-07T07:39:15.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.249+0000 7f19bfa0d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b810c3f0 con 0x7f19b8102150 2024-03-07T07:39:15.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.249+0000 7f19bcfa8640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19b8101790 0x7f19b819c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:15.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.249+0000 7f19bcfa8640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19b8101790 0x7f19b819c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:42228/0 (socket says 172.21.15.160:42228) 2024-03-07T07:39:15.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.249+0000 7f19bcfa8640 1 -- 172.21.15.160:0/982520421 learned_addr learned my addr 172.21.15.160:0/982520421 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:39:15.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.249+0000 7f19bd7a9640 1 --2- 172.21.15.160:0/982520421 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19b81005a0 0x7f19b806a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:15.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.249+0000 7f19bdfaa640 1 --2- 172.21.15.160:0/982520421 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19b8102150 0x7f19b819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:15.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.249+0000 7f19bcfa8640 1 -- 172.21.15.160:0/982520421 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19b8102150 msgr2=0x7f19b819ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:15.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.249+0000 7f19bcfa8640 1 --2- 172.21.15.160:0/982520421 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19b8102150 0x7f19b819ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:15.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.249+0000 7f19bcfa8640 1 -- 172.21.15.160:0/982520421 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19b81005a0 msgr2=0x7f19b806a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:15.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.249+0000 7f19bcfa8640 1 --2- 172.21.15.160:0/982520421 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19b81005a0 0x7f19b806a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:15.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.249+0000 7f19bcfa8640 1 -- 172.21.15.160:0/982520421 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b819f690 con 0x7f19b8101790 2024-03-07T07:39:15.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.249+0000 7f19bdfaa640 1 --2- 172.21.15.160:0/982520421 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19b8102150 0x7f19b819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T07:39:15.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.249+0000 7f19bd7a9640 1 --2- 172.21.15.160:0/982520421 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19b81005a0 0x7f19b806a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:39:15.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.249+0000 7f19bcfa8640 1 --2- 172.21.15.160:0/982520421 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19b8101790 0x7f19b819c760 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f19b4009230 tx=0x7f19b4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:15.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.249+0000 7f19ae7fc640 1 -- 172.21.15.160:0/982520421 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b4041070 con 0x7f19b8101790 2024-03-07T07:39:15.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.250+0000 7f19bfa0d640 1 -- 172.21.15.160:0/982520421 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19b819f8c0 con 0x7f19b8101790 2024-03-07T07:39:15.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.250+0000 7f19ae7fc640 1 -- 172.21.15.160:0/982520421 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f19b40054e0 con 0x7f19b8101790 2024-03-07T07:39:15.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.250+0000 7f19ae7fc640 1 -- 172.21.15.160:0/982520421 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b40346a0 con 0x7f19b8101790 2024-03-07T07:39:15.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.250+0000 7f19bfa0d640 1 -- 172.21.15.160:0/982520421 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19b819fdb0 con 0x7f19b8101790 2024-03-07T07:39:15.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.250+0000 7f19bfa0d640 1 -- 172.21.15.160:0/982520421 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19b81078c0 con 0x7f19b8101790 2024-03-07T07:39:15.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.251+0000 7f19ae7fc640 1 -- 172.21.15.160:0/982520421 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f19b4044070 con 0x7f19b8101790 2024-03-07T07:39:15.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.252+0000 7f19ae7fc640 1 --2- 172.21.15.160:0/982520421 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f19a40a1d80 0x7f19a40a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:15.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.252+0000 7f19bd7a9640 1 --2- 172.21.15.160:0/982520421 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f19a40a1d80 0x7f19a40a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:15.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.252+0000 7f19ae7fc640 1 -- 172.21.15.160:0/982520421 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f19b40efaa0 con 0x7f19b8101790 2024-03-07T07:39:15.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.252+0000 7f19bd7a9640 1 --2- 172.21.15.160:0/982520421 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f19a40a1d80 0x7f19a40a4240 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f19b81015f0 tx=0x7f19a800a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:15.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.256+0000 7f19ae7fc640 1 -- 172.21.15.160:0/982520421 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f19b40bd020 con 0x7f19b8101790 2024-03-07T07:39:15.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:14 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/3496954906' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-07T07:39:15.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:14 smithi178 ceph-mon[36650]: osdmap e83: 12 total, 12 up, 12 in 2024-03-07T07:39:15.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:14 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/2021713868' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-07T07:39:15.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:14 smithi178 ceph-mon[36650]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-03-07T07:39:15.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:14 smithi178 ceph-mon[36650]: pgmap v286: 97 pgs: 31 creating+peering, 55 unknown, 11 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:39:15.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:14 smithi178 ceph-mon[36650]: osdmap e84: 12 total, 12 up, 12 in 2024-03-07T07:39:15.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.434+0000 7f19bfa0d640 1 -- 172.21.15.160:0/982520421 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f19b802d250 con 0x7f19b8101790 2024-03-07T07:39:15.435 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.435+0000 7f19ae7fc640 1 -- 172.21.15.160:0/982520421 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22255 (secure 0 0 0) 0x7f19b40bc920 con 0x7f19b8101790 2024-03-07T07:39:15.436 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:39:15.436 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":84,"fsid":"826f6582-dc54-11ee-95c6-87774f69a715","created":"2024-03-07T07:31:14.229288+0000","modified":"2024-03-07T07:39:14.163855+0000","last_up_change":"2024-03-07T07:38:20.776225+0000","last_in_change":"2024-03-07T07:38:00.508917+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T07:34:34.529424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T07:39:12.075432+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T07:39:13.367820+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bee66c67-fb6e-4717-9685-ca62587d5f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6803","nonce":1368482926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6805","nonce":1368482926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6809","nonce":1368482926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6807","nonce":1368482926}]},"public_addr":"172.21.15.160:6803/1368482926","cluster_addr":"172.21.15.160:6805/1368482926","heartbeat_back_addr":"172.21.15.160:6809/1368482926","heartbeat_front_addr":"172.21.15.160:6807/1368482926","state":["exists","up"]},{"osd":1,"uuid":"49236251-f750-424e-992b-bddf55762556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6801","nonce":2922391884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6803","nonce":2922391884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6807","nonce":2922391884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6805","nonce":2922391884}]},"public_addr":"172.21.15.177:6801/2922391884","cluster_addr":"172.21.15.177:6803/2922391884","heartbeat_back_addr":"172.21.15.177:6807/2922391884","heartbeat_front_addr":"172.21.15.177:6805/2922391884","state":["exists","up"]},{"osd":2,"uuid":"7a432b28-f7ee-43fe-9196-7aaeccfecf77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6801","nonce":1217901551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6803","nonce":1217901551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6807","nonce":1217901551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6805","nonce":1217901551}]},"public_addr":"172.21.15.178:6801/1217901551","cluster_addr":"172.21.15.178:6803/1217901551","heartbeat_back_addr":"172.21.15.178:6807/1217901551","heartbeat_front_addr":"172.21.15.178:6805/1217901551","state":["exists","up"]},{"osd":3,"uuid":"4ad22a20-f1be-4002-a48d-ecd3953e63f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6811","nonce":42543989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6813","nonce":42543989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6817","nonce":42543989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6815","nonce":42543989}]},"public_addr":"172.21.15.160:6811/42543989","cluster_addr":"172.21.15.160:6813/42543989","heartbeat_back_addr":"172.21.15.160:6817/42543989","heartbeat_front_addr":"172.21.15.160:6815/42543989","state":["exists","up"]},{"osd":4,"uuid":"2b65b104-ac82-4ce3-b6c5-11a399103045","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6809","nonce":88178553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6811","nonce":88178553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6815","nonce":88178553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6813","nonce":88178553}]},"public_addr":"172.21.15.177:6809/88178553","cluster_addr":"172.21.15.177:6811/88178553","heartbeat_back_addr":"172.21.15.177:6815/88178553","heartbeat_front_addr":"172.21.15.177:6813/88178553","state":["exists","up"]},{"osd":5,"uuid":"30a1e714-08ff-44e8-9264-9ef164604b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6809","nonce":964732891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6811","nonce":964732891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6815","nonce":964732891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6813","nonce":964732891}]},"public_addr":"172.21.15.178:6809/964732891","cluster_addr":"172.21.15.178:6811/964732891","heartbeat_back_addr":"172.21.15.178:6815/964732891","heartbeat_front_addr":"172.21.15.178:6813/964732891","state":["exists","up"]},{"osd":6,"uuid":"b1e1646e-f8a9-42b8-b62d-9d7b5d6a1ff4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6819","nonce":3251327001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6821","nonce":3251327001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6825","nonce":3251327001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6823","nonce":3251327001}]},"public_addr":"172.21.15.160:6819/3251327001","cluster_addr":"172.21.15.160:6821/3251327001","heartbeat_back_addr":"172.21.15.160:6825/3251327001","heartbeat_front_addr":"172.21.15.160:6823/3251327001","state":["exists","up"]},{"osd":7,"uuid":"e66b400b-9747-4f6d-b05b-db22468e2750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6817","nonce":1175500850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6819","nonce":1175500850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6823","nonce":1175500850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6821","nonce":1175500850}]},"public_addr":"172.21.15.177:6817/1175500850","cluster_addr":"172.21.15.177:6819/1175500850","heartbeat_back_addr":"172.21.15.177:6823/1175500850","heartbeat_front_addr":"172.21.15.177:6821/1175500850","state":["exists","up"]},{"osd":8,"uuid":"3ee0f2c4-fca4-49fc-a98b-07aa0fcba0e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6817","nonce":688015531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6819","nonce":688015531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6823","nonce":688015531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6821","nonce":688015531}]},"public_addr":"172.21.15.178:6817/688015531","cluster_addr":"172.21.15.178:6819/688015531","heartbeat_back_addr":"172.21.15.178:6823/688015531","heartbeat_front_addr":"172.21.15.178:6821/688015531","state":["exists","up"]},{"osd":9,"uuid":"856039d7-f7ee-4dbd-b019-0073e705ce0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6827","nonce":3265797851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6829","nonce":3265797851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6833","nonce":3265797851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6831","nonce":3265797851}]},"public_addr":"172.21.15.160:6827/3265797851","cluster_addr":"172.21.15.160:6829/3265797851","heartbeat_back_addr":"172.21.15.160:6833/3265797851","heartbeat_front_addr":"172.21.15.160:6831/3265797851","state":["exists","up"]},{"osd":10,"uuid":"bdc9899d-d720-4429-ad53-4c388a4dbe75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6825","nonce":2867473412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6827","nonce":2867473412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6831","nonce":2867473412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6829","nonce":2867473412}]},"public_addr":"172.21.15.177:6825/2867473412","cluster_addr":"172.21.15.177:6827/2867473412","heartbeat_back_addr":"172.21.15.177:6831/2867473412","heartbeat_front_addr":"172.21.15.177:6829/2867473412","state":["exists","up"]},{"osd":11,"uuid":"cc9adb54-ef49-47c6-90e3-fc03bc0f5cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6825","nonce":4068724731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6827","nonce":4068724731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6831","nonce":4068724731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6829","nonce":4068724731}]},"public_addr":"172.21.15.178:6825/4068724731","cluster_addr":"172.21.15.178:6827/4068724731","heartbeat_back_addr":"172.21.15.178:6831/4068724731","heartbeat_front_addr":"172.21.15.178:6829/4068724731","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:33:44.141585+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:07.685585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:31.044386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:56.308925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:35:21.582097+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:35:45.983540+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:36:11.530242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:36:36.844681+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:02.303437+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:27.629867+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:53.289495+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:38:18.536169+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/3027450612":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/874709722":"2024-03-08T07:32:08.190837+0000","172.21.15.160:6800/1130675225":"2024-03-08T07:31:41.166858+0000","172.21.15.160:0/3857193980":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/3861968297":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6800/3859174315":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/3013534743":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6801/3859174315":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/1310937667":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6801/1130675225":"2024-03-08T07:31:41.166858+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T07:39:15.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.437+0000 7f19bfa0d640 1 -- 172.21.15.160:0/982520421 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f19a40a1d80 msgr2=0x7f19a40a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:15.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.437+0000 7f19bfa0d640 1 --2- 172.21.15.160:0/982520421 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f19a40a1d80 0x7f19a40a4240 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f19b81015f0 tx=0x7f19a800a560 comp rx=0 tx=0).stop 2024-03-07T07:39:15.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.437+0000 7f19bfa0d640 1 -- 172.21.15.160:0/982520421 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19b8101790 msgr2=0x7f19b819c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:15.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.437+0000 7f19bfa0d640 1 --2- 172.21.15.160:0/982520421 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19b8101790 0x7f19b819c760 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f19b4009230 tx=0x7f19b4004040 comp rx=0 tx=0).stop 2024-03-07T07:39:15.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.437+0000 7f19bfa0d640 1 -- 172.21.15.160:0/982520421 shutdown_connections 2024-03-07T07:39:15.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.437+0000 7f19bfa0d640 1 --2- 172.21.15.160:0/982520421 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f19a40a1d80 0x7f19a40a4240 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:15.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.437+0000 7f19bfa0d640 1 --2- 172.21.15.160:0/982520421 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19b8102150 0x7f19b819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:15.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.437+0000 7f19bfa0d640 1 --2- 172.21.15.160:0/982520421 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19b8101790 0x7f19b819c760 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:15.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.437+0000 7f19bfa0d640 1 --2- 172.21.15.160:0/982520421 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19b81005a0 0x7f19b806a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:15.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.437+0000 7f19bfa0d640 1 -- 172.21.15.160:0/982520421 >> 172.21.15.160:0/982520421 conn(0x7f19b80fbb10 msgr2=0x7f19b8104a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:15.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.437+0000 7f19bfa0d640 1 -- 172.21.15.160:0/982520421 shutdown_connections 2024-03-07T07:39:15.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.437+0000 7f19bfa0d640 1 -- 172.21.15.160:0/982520421 wait complete. 2024-03-07T07:39:15.449 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-07T07:39:15.556 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.554+0000 7f164946b640 1 -- 172.21.15.160:0/2776705220 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1644102120 msgr2=0x7f16441089b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:15.556 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.554+0000 7f164946b640 1 --2- 172.21.15.160:0/2776705220 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1644102120 0x7f16441089b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f163c009ca0 tx=0x7f163c02c0f0 comp rx=0 tx=0).stop 2024-03-07T07:39:15.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.554+0000 7f164946b640 1 -- 172.21.15.160:0/2776705220 shutdown_connections 2024-03-07T07:39:15.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.554+0000 7f164946b640 1 --2- 172.21.15.160:0/2776705220 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1644102120 0x7f16441089b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:15.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.554+0000 7f164946b640 1 --2- 172.21.15.160:0/2776705220 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1644101760 0x7f1644101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:15.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.554+0000 7f164946b640 1 --2- 172.21.15.160:0/2776705220 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1644100570 0x7f1644100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:15.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.554+0000 7f164946b640 1 -- 172.21.15.160:0/2776705220 >> 172.21.15.160:0/2776705220 conn(0x7f16440fbb00 msgr2=0x7f16440fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:15.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.554+0000 7f164946b640 1 -- 172.21.15.160:0/2776705220 shutdown_connections 2024-03-07T07:39:15.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.554+0000 7f164946b640 1 -- 172.21.15.160:0/2776705220 wait complete. 2024-03-07T07:39:15.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.555+0000 7f164946b640 1 Processor -- start 2024-03-07T07:39:15.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.555+0000 7f164946b640 1 -- start start 2024-03-07T07:39:15.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.555+0000 7f164946b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1644100570 0x7f164406a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:15.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.555+0000 7f164946b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1644101760 0x7f164419c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:15.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.555+0000 7f164946b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1644102120 0x7f164419ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:15.558 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.555+0000 7f164946b640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164410c200 con 0x7f1644101760 2024-03-07T07:39:15.558 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.555+0000 7f164946b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164410c080 con 0x7f1644102120 2024-03-07T07:39:15.558 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.555+0000 7f164946b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164410c380 con 0x7f1644100570 2024-03-07T07:39:15.558 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.555+0000 7f16427fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1644101760 0x7f164419c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:15.558 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.555+0000 7f16427fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1644101760 0x7f164419c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:42256/0 (socket says 172.21.15.160:42256) 2024-03-07T07:39:15.558 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.555+0000 7f16427fc640 1 -- 172.21.15.160:0/2382618864 learned_addr learned my addr 172.21.15.160:0/2382618864 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:39:15.558 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.555+0000 7f1642ffd640 1 --2- 172.21.15.160:0/2382618864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1644100570 0x7f164406a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:15.558 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.555+0000 7f16427fc640 1 -- 172.21.15.160:0/2382618864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1644100570 msgr2=0x7f164406a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:15.558 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.555+0000 7f16427fc640 1 --2- 172.21.15.160:0/2382618864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1644100570 0x7f164406a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:15.558 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.555+0000 7f16427fc640 1 -- 172.21.15.160:0/2382618864 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1644102120 msgr2=0x7f164419ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:15.558 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.555+0000 7f16427fc640 1 --2- 172.21.15.160:0/2382618864 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1644102120 0x7f164419ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:15.558 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.555+0000 7f16427fc640 1 -- 172.21.15.160:0/2382618864 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f164419f4c0 con 0x7f1644101760 2024-03-07T07:39:15.558 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.555+0000 7f16427fc640 1 --2- 172.21.15.160:0/2382618864 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1644101760 0x7f164419c5c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f163800c920 tx=0x7f163800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:15.559 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.556+0000 7f162bfff640 1 -- 172.21.15.160:0/2382618864 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1638007c70 con 0x7f1644101760 2024-03-07T07:39:15.559 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.556+0000 7f164946b640 1 -- 172.21.15.160:0/2382618864 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f164419f7b0 con 0x7f1644101760 2024-03-07T07:39:15.559 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.556+0000 7f162bfff640 1 -- 172.21.15.160:0/2382618864 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1638007e10 con 0x7f1644101760 2024-03-07T07:39:15.559 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.556+0000 7f162bfff640 1 -- 172.21.15.160:0/2382618864 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1638018890 con 0x7f1644101760 2024-03-07T07:39:15.559 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.556+0000 7f164946b640 1 -- 172.21.15.160:0/2382618864 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f164419fd00 con 0x7f1644101760 2024-03-07T07:39:15.560 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.557+0000 7f164946b640 1 -- 172.21.15.160:0/2382618864 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1644107830 con 0x7f1644101760 2024-03-07T07:39:15.560 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.558+0000 7f162bfff640 1 -- 172.21.15.160:0/2382618864 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f1638018a70 con 0x7f1644101760 2024-03-07T07:39:15.560 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.558+0000 7f162bfff640 1 --2- 172.21.15.160:0/2382618864 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f162c0a1d80 0x7f162c0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:15.560 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.558+0000 7f1642ffd640 1 --2- 172.21.15.160:0/2382618864 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f162c0a1d80 0x7f162c0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:15.560 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.558+0000 7f162bfff640 1 -- 172.21.15.160:0/2382618864 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f16380cde00 con 0x7f1644101760 2024-03-07T07:39:15.560 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.559+0000 7f1642ffd640 1 --2- 172.21.15.160:0/2382618864 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f162c0a1d80 0x7f162c0a4240 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1634005fd0 tx=0x7f1634005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:15.562 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.562+0000 7f162bfff640 1 -- 172.21.15.160:0/2382618864 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1638014030 con 0x7f1644101760 2024-03-07T07:39:15.779 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.779+0000 7f164946b640 1 -- 172.21.15.160:0/2382618864 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f164402d200 con 0x7f1644101760 2024-03-07T07:39:15.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.830+0000 7f162bfff640 1 -- 172.21.15.160:0/2382618864 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7f163800de20 con 0x7f1644101760 2024-03-07T07:39:15.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.832+0000 7f164946b640 1 -- 172.21.15.160:0/2382618864 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f162c0a1d80 msgr2=0x7f162c0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:15.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.832+0000 7f164946b640 1 --2- 172.21.15.160:0/2382618864 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f162c0a1d80 0x7f162c0a4240 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1634005fd0 tx=0x7f1634005850 comp rx=0 tx=0).stop 2024-03-07T07:39:15.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.832+0000 7f164946b640 1 -- 172.21.15.160:0/2382618864 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1644101760 msgr2=0x7f164419c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:15.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.832+0000 7f164946b640 1 --2- 172.21.15.160:0/2382618864 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1644101760 0x7f164419c5c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f163800c920 tx=0x7f163800cde0 comp rx=0 tx=0).stop 2024-03-07T07:39:15.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.832+0000 7f164946b640 1 -- 172.21.15.160:0/2382618864 shutdown_connections 2024-03-07T07:39:15.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.832+0000 7f164946b640 1 --2- 172.21.15.160:0/2382618864 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f162c0a1d80 0x7f162c0a4240 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:15.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.832+0000 7f164946b640 1 --2- 172.21.15.160:0/2382618864 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1644102120 0x7f164419ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:15.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.832+0000 7f164946b640 1 --2- 172.21.15.160:0/2382618864 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1644101760 0x7f164419c5c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:15.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.832+0000 7f164946b640 1 --2- 172.21.15.160:0/2382618864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1644100570 0x7f164406a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:15.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.832+0000 7f164946b640 1 -- 172.21.15.160:0/2382618864 >> 172.21.15.160:0/2382618864 conn(0x7f16440fbb00 msgr2=0x7f1644104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:15.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.832+0000 7f164946b640 1 -- 172.21.15.160:0/2382618864 shutdown_connections 2024-03-07T07:39:15.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.832+0000 7f164946b640 1 -- 172.21.15.160:0/2382618864 wait complete. 2024-03-07T07:39:15.846 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-03-07T07:39:15.846 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-03-07T07:39:15.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.952+0000 7fcabc3c9640 1 -- 172.21.15.160:0/3163379724 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcab4102110 msgr2=0x7fcab41089a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:15.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.952+0000 7fcabc3c9640 1 --2- 172.21.15.160:0/3163379724 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcab4102110 0x7fcab41089a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fcab0009ca0 tx=0x7fcab002c0f0 comp rx=0 tx=0).stop 2024-03-07T07:39:15.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.952+0000 7fcabc3c9640 1 -- 172.21.15.160:0/3163379724 shutdown_connections 2024-03-07T07:39:15.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.952+0000 7fcabc3c9640 1 --2- 172.21.15.160:0/3163379724 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcab4102110 0x7fcab41089a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:15.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.952+0000 7fcabc3c9640 1 --2- 172.21.15.160:0/3163379724 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcab4101750 0x7fcab4101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:15.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.952+0000 7fcabc3c9640 1 --2- 172.21.15.160:0/3163379724 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcab4100560 0x7fcab4100980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:15.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.952+0000 7fcabc3c9640 1 -- 172.21.15.160:0/3163379724 >> 172.21.15.160:0/3163379724 conn(0x7fcab40fbaf0 msgr2=0x7fcab40fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:15.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.952+0000 7fcabc3c9640 1 -- 172.21.15.160:0/3163379724 shutdown_connections 2024-03-07T07:39:15.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.952+0000 7fcabc3c9640 1 -- 172.21.15.160:0/3163379724 wait complete. 2024-03-07T07:39:15.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.952+0000 7fcabc3c9640 1 Processor -- start 2024-03-07T07:39:15.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.952+0000 7fcabc3c9640 1 -- start start 2024-03-07T07:39:15.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.952+0000 7fcabc3c9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcab4100560 0x7fcab41968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:15.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.952+0000 7fcabc3c9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcab4101750 0x7fcab4196e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:15.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.952+0000 7fcabc3c9640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcab4102110 0x7fcab419dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:15.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.952+0000 7fcabc3c9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab410c250 con 0x7fcab4100560 2024-03-07T07:39:15.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.952+0000 7fcabc3c9640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab410c0d0 con 0x7fcab4102110 2024-03-07T07:39:15.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.952+0000 7fcabc3c9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab410c3d0 con 0x7fcab4101750 2024-03-07T07:39:15.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.953+0000 7fcaba165640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcab4100560 0x7fcab41968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:15.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.953+0000 7fcaba165640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcab4100560 0x7fcab41968f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:42274/0 (socket says 172.21.15.160:42274) 2024-03-07T07:39:15.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.953+0000 7fcaba165640 1 -- 172.21.15.160:0/3394550451 learned_addr learned my addr 172.21.15.160:0/3394550451 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:39:15.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.953+0000 7fcab9964640 1 --2- 172.21.15.160:0/3394550451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcab4101750 0x7fcab4196e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:15.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.953+0000 7fcaba966640 1 --2- 172.21.15.160:0/3394550451 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcab4102110 0x7fcab419dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:15.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.953+0000 7fcaba165640 1 -- 172.21.15.160:0/3394550451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcab4101750 msgr2=0x7fcab4196e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:15.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.953+0000 7fcaba165640 1 --2- 172.21.15.160:0/3394550451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcab4101750 0x7fcab4196e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:15.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.953+0000 7fcaba165640 1 -- 172.21.15.160:0/3394550451 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcab4102110 msgr2=0x7fcab419dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:15.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.953+0000 7fcaba165640 1 --2- 172.21.15.160:0/3394550451 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcab4102110 0x7fcab419dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:15.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.953+0000 7fcaba165640 1 -- 172.21.15.160:0/3394550451 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab419e510 con 0x7fcab4100560 2024-03-07T07:39:15.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.953+0000 7fcaba966640 1 --2- 172.21.15.160:0/3394550451 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcab4102110 0x7fcab419dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:39:15.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.953+0000 7fcaba165640 1 --2- 172.21.15.160:0/3394550451 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcab4100560 0x7fcab41968f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcaac00e9c0 tx=0x7fcaac00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:15.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.953+0000 7fcaab7fe640 1 -- 172.21.15.160:0/3394550451 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaac00ccc0 con 0x7fcab4100560 2024-03-07T07:39:15.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.953+0000 7fcaab7fe640 1 -- 172.21.15.160:0/3394550451 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcaac0035f0 con 0x7fcab4100560 2024-03-07T07:39:15.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.953+0000 7fcabc3c9640 1 -- 172.21.15.160:0/3394550451 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcab419e800 con 0x7fcab4100560 2024-03-07T07:39:15.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.953+0000 7fcaab7fe640 1 -- 172.21.15.160:0/3394550451 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaac00ce60 con 0x7fcab4100560 2024-03-07T07:39:15.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.953+0000 7fcabc3c9640 1 -- 172.21.15.160:0/3394550451 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcab419ed00 con 0x7fcab4100560 2024-03-07T07:39:15.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.954+0000 7fcabc3c9640 1 -- 172.21.15.160:0/3394550451 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcab4107880 con 0x7fcab4100560 2024-03-07T07:39:15.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.955+0000 7fcaab7fe640 1 -- 172.21.15.160:0/3394550451 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7fcaac0037a0 con 0x7fcab4100560 2024-03-07T07:39:15.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.956+0000 7fcaab7fe640 1 --2- 172.21.15.160:0/3394550451 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fcaa00a1d80 0x7fcaa00a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:15.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.956+0000 7fcab9964640 1 --2- 172.21.15.160:0/3394550451 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fcaa00a1d80 0x7fcaa00a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:15.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.956+0000 7fcaab7fe640 1 -- 172.21.15.160:0/3394550451 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fcaac0ce060 con 0x7fcab4100560 2024-03-07T07:39:15.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.956+0000 7fcab9964640 1 --2- 172.21.15.160:0/3394550451 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fcaa00a1d80 0x7fcaa00a4240 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcab4197e00 tx=0x7fcaa4009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:15.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:15.960+0000 7fcaab7fe640 1 -- 172.21.15.160:0/3394550451 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fcaac09b5e0 con 0x7fcab4100560 2024-03-07T07:39:16.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:15 smithi177 ceph-mon[36578]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-07T07:39:16.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:15 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/2021713868' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-07T07:39:16.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:15 smithi177 ceph-mon[36578]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-03-07T07:39:16.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:15 smithi177 ceph-mon[36578]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-03-07T07:39:16.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:15 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/793313048' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:39:16.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:15 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/982520421' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:39:16.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:15 smithi177 ceph-mon[36578]: osdmap e85: 12 total, 12 up, 12 in 2024-03-07T07:39:16.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:15 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/2382618864' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-07T07:39:16.175 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.175+0000 7fcabc3c9640 1 -- 172.21.15.160:0/3394550451 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7fcab4062260 con 0x7fcab4100560 2024-03-07T07:39:16.177 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:15 smithi160 ceph-mon[31963]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-07T07:39:16.177 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:15 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/2021713868' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-07T07:39:16.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:15 smithi160 ceph-mon[31963]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-03-07T07:39:16.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:15 smithi160 ceph-mon[31963]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-03-07T07:39:16.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:15 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/793313048' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:39:16.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:15 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/982520421' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:39:16.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:15 smithi160 ceph-mon[31963]: osdmap e85: 12 total, 12 up, 12 in 2024-03-07T07:39:16.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:15 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/2382618864' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-07T07:39:16.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:15 smithi178 ceph-mon[36650]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-07T07:39:16.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:15 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/2021713868' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-07T07:39:16.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:15 smithi178 ceph-mon[36650]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-03-07T07:39:16.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:15 smithi178 ceph-mon[36650]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-03-07T07:39:16.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:15 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/793313048' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:39:16.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:15 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/982520421' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:39:16.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:15 smithi178 ceph-mon[36650]: osdmap e85: 12 total, 12 up, 12 in 2024-03-07T07:39:16.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:15 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/2382618864' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-07T07:39:16.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.840+0000 7fcaab7fe640 1 -- 172.21.15.160:0/3394550451 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v16) v1 ==== 105+0+0 (secure 0 0 0) 0x7fcaac09aee0 con 0x7fcab4100560 2024-03-07T07:39:16.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.842+0000 7fcabc3c9640 1 -- 172.21.15.160:0/3394550451 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fcaa00a1d80 msgr2=0x7fcaa00a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:16.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.842+0000 7fcabc3c9640 1 --2- 172.21.15.160:0/3394550451 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fcaa00a1d80 0x7fcaa00a4240 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcab4197e00 tx=0x7fcaa4009110 comp rx=0 tx=0).stop 2024-03-07T07:39:16.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.842+0000 7fcabc3c9640 1 -- 172.21.15.160:0/3394550451 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcab4100560 msgr2=0x7fcab41968f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:16.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.842+0000 7fcabc3c9640 1 --2- 172.21.15.160:0/3394550451 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcab4100560 0x7fcab41968f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcaac00e9c0 tx=0x7fcaac00edc0 comp rx=0 tx=0).stop 2024-03-07T07:39:16.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.842+0000 7fcabc3c9640 1 -- 172.21.15.160:0/3394550451 shutdown_connections 2024-03-07T07:39:16.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.842+0000 7fcabc3c9640 1 --2- 172.21.15.160:0/3394550451 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fcaa00a1d80 0x7fcaa00a4240 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:16.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.842+0000 7fcabc3c9640 1 --2- 172.21.15.160:0/3394550451 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcab4102110 0x7fcab419dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:16.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.842+0000 7fcabc3c9640 1 --2- 172.21.15.160:0/3394550451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcab4101750 0x7fcab4196e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:16.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.842+0000 7fcabc3c9640 1 --2- 172.21.15.160:0/3394550451 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcab4100560 0x7fcab41968f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:16.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.842+0000 7fcabc3c9640 1 -- 172.21.15.160:0/3394550451 >> 172.21.15.160:0/3394550451 conn(0x7fcab40fbaf0 msgr2=0x7fcab4104a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:16.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.843+0000 7fcabc3c9640 1 -- 172.21.15.160:0/3394550451 shutdown_connections 2024-03-07T07:39:16.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.843+0000 7fcabc3c9640 1 -- 172.21.15.160:0/3394550451 wait complete. 2024-03-07T07:39:16.858 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-03-07T07:39:16.972 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.969+0000 7f6401234640 1 -- 172.21.15.160:0/852456987 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63fc101670 msgr2=0x7f63fc101a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:16.972 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.969+0000 7f6401234640 1 --2- 172.21.15.160:0/852456987 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63fc101670 0x7f63fc101a90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f63ec009900 tx=0x7f63ec02bc30 comp rx=0 tx=0).stop 2024-03-07T07:39:16.972 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.969+0000 7f6401234640 1 -- 172.21.15.160:0/852456987 shutdown_connections 2024-03-07T07:39:16.972 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.969+0000 7f6401234640 1 --2- 172.21.15.160:0/852456987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f63fc103220 0x7f63fc109ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:16.972 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.969+0000 7f6401234640 1 --2- 172.21.15.160:0/852456987 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f63fc102860 0x7f63fc102ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:16.972 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.969+0000 7f6401234640 1 --2- 172.21.15.160:0/852456987 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63fc101670 0x7f63fc101a90 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:16.972 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.969+0000 7f6401234640 1 -- 172.21.15.160:0/852456987 >> 172.21.15.160:0/852456987 conn(0x7f63fc0fcc00 msgr2=0x7f63fc0ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:16.973 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.969+0000 7f6401234640 1 -- 172.21.15.160:0/852456987 shutdown_connections 2024-03-07T07:39:16.973 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.969+0000 7f6401234640 1 -- 172.21.15.160:0/852456987 wait complete. 2024-03-07T07:39:16.973 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.970+0000 7f6401234640 1 Processor -- start 2024-03-07T07:39:16.973 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.970+0000 7f6401234640 1 -- start start 2024-03-07T07:39:16.973 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.970+0000 7f6401234640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63fc101670 0x7f63fc197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:16.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.970+0000 7f6401234640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f63fc102860 0x7f63fc197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:16.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.970+0000 7f6401234640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f63fc103220 0x7f63fc19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:16.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.970+0000 7f6401234640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63fc10d2f0 con 0x7f63fc101670 2024-03-07T07:39:16.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.970+0000 7f6401234640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63fc10d170 con 0x7f63fc103220 2024-03-07T07:39:16.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.970+0000 7f6401234640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63fc10d470 con 0x7f63fc102860 2024-03-07T07:39:16.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.970+0000 7f63fad9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63fc101670 0x7f63fc197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:16.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.970+0000 7f63fad9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63fc101670 0x7f63fc197970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:42298/0 (socket says 172.21.15.160:42298) 2024-03-07T07:39:16.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.970+0000 7f63fad9d640 1 -- 172.21.15.160:0/1393757855 learned_addr learned my addr 172.21.15.160:0/1393757855 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:39:16.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.970+0000 7f63f3fff640 1 --2- 172.21.15.160:0/1393757855 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f63fc102860 0x7f63fc197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:16.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.970+0000 7f63fb59e640 1 --2- 172.21.15.160:0/1393757855 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f63fc103220 0x7f63fc19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:16.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.970+0000 7f63fad9d640 1 -- 172.21.15.160:0/1393757855 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f63fc102860 msgr2=0x7f63fc197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:16.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.970+0000 7f63fad9d640 1 --2- 172.21.15.160:0/1393757855 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f63fc102860 0x7f63fc197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:16.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.970+0000 7f63fad9d640 1 -- 172.21.15.160:0/1393757855 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f63fc103220 msgr2=0x7f63fc19ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:16.975 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.970+0000 7f63fad9d640 1 --2- 172.21.15.160:0/1393757855 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f63fc103220 0x7f63fc19ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:16.975 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.970+0000 7f63fad9d640 1 -- 172.21.15.160:0/1393757855 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63fc19f590 con 0x7f63fc101670 2024-03-07T07:39:16.975 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.970+0000 7f63fb59e640 1 --2- 172.21.15.160:0/1393757855 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f63fc103220 0x7f63fc19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:39:16.975 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.970+0000 7f63fad9d640 1 --2- 172.21.15.160:0/1393757855 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63fc101670 0x7f63fc197970 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f63ec000c00 tx=0x7f63ec005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:16.975 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.971+0000 7f63f8d99640 1 -- 172.21.15.160:0/1393757855 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ec041070 con 0x7f63fc101670 2024-03-07T07:39:16.975 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.971+0000 7f63f8d99640 1 -- 172.21.15.160:0/1393757855 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f63ec0044d0 con 0x7f63fc101670 2024-03-07T07:39:16.975 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.971+0000 7f6401234640 1 -- 172.21.15.160:0/1393757855 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63fc19f820 con 0x7f63fc101670 2024-03-07T07:39:16.975 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.971+0000 7f63f8d99640 1 -- 172.21.15.160:0/1393757855 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ec034440 con 0x7f63fc101670 2024-03-07T07:39:16.975 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.971+0000 7f6401234640 1 -- 172.21.15.160:0/1393757855 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63fc19fd10 con 0x7f63fc101670 2024-03-07T07:39:16.975 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.972+0000 7f6401234640 1 -- 172.21.15.160:0/1393757855 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63fc108ab0 con 0x7f63fc101670 2024-03-07T07:39:16.976 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.973+0000 7f63f8d99640 1 -- 172.21.15.160:0/1393757855 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f63ec044070 con 0x7f63fc101670 2024-03-07T07:39:16.976 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.973+0000 7f63f8d99640 1 --2- 172.21.15.160:0/1393757855 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f63e80a1d80 0x7f63e80a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:16.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.973+0000 7f63f3fff640 1 --2- 172.21.15.160:0/1393757855 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f63e80a1d80 0x7f63e80a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:16.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.973+0000 7f63f8d99640 1 -- 172.21.15.160:0/1393757855 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f63ec0ef820 con 0x7f63fc101670 2024-03-07T07:39:16.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.974+0000 7f63f3fff640 1 --2- 172.21.15.160:0/1393757855 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f63e80a1d80 0x7f63e80a4240 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f63fc198e80 tx=0x7f63e400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:16.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:16.977+0000 7f63f8d99640 1 -- 172.21.15.160:0/1393757855 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f63ec0bcda0 con 0x7f63fc101670 2024-03-07T07:39:17.106 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:16 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/2382618864' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-07T07:39:17.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:16 smithi177 ceph-mon[36578]: mds.? [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] up:active 2024-03-07T07:39:17.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:16 smithi177 ceph-mon[36578]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-07T07:39:17.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:16 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/3394550451' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-07T07:39:17.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:16 smithi177 ceph-mon[36578]: pgmap v289: 97 pgs: 33 creating+peering, 31 unknown, 33 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-03-07T07:39:17.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:16 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/2382618864' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-07T07:39:17.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:16 smithi160 ceph-mon[31963]: mds.? [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] up:active 2024-03-07T07:39:17.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:16 smithi160 ceph-mon[31963]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-07T07:39:17.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:16 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/3394550451' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-07T07:39:17.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:16 smithi160 ceph-mon[31963]: pgmap v289: 97 pgs: 33 creating+peering, 31 unknown, 33 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-03-07T07:39:17.191 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.190+0000 7f6401234640 1 -- 172.21.15.160:0/1393757855 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f63fc02d250 con 0x7f63fc101670 2024-03-07T07:39:17.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:16 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/2382618864' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-07T07:39:17.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:16 smithi178 ceph-mon[36650]: mds.? [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] up:active 2024-03-07T07:39:17.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:16 smithi178 ceph-mon[36650]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-07T07:39:17.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:16 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/3394550451' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-07T07:39:17.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:16 smithi178 ceph-mon[36650]: pgmap v289: 97 pgs: 33 creating+peering, 31 unknown, 33 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-03-07T07:39:17.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.847+0000 7f63f8d99640 1 -- 172.21.15.160:0/1393757855 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v18) v1 ==== 121+0+0 (secure 0 0 0) 0x7f63ec0bc6a0 con 0x7f63fc101670 2024-03-07T07:39:17.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.849+0000 7f6401234640 1 -- 172.21.15.160:0/1393757855 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f63e80a1d80 msgr2=0x7f63e80a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:17.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.849+0000 7f6401234640 1 --2- 172.21.15.160:0/1393757855 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f63e80a1d80 0x7f63e80a4240 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f63fc198e80 tx=0x7f63e400a2d0 comp rx=0 tx=0).stop 2024-03-07T07:39:17.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.849+0000 7f6401234640 1 -- 172.21.15.160:0/1393757855 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63fc101670 msgr2=0x7f63fc197970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:17.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.849+0000 7f6401234640 1 --2- 172.21.15.160:0/1393757855 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63fc101670 0x7f63fc197970 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f63ec000c00 tx=0x7f63ec005d50 comp rx=0 tx=0).stop 2024-03-07T07:39:17.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.850+0000 7f6401234640 1 -- 172.21.15.160:0/1393757855 shutdown_connections 2024-03-07T07:39:17.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.850+0000 7f6401234640 1 --2- 172.21.15.160:0/1393757855 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f63e80a1d80 0x7f63e80a4240 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:17.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.850+0000 7f6401234640 1 --2- 172.21.15.160:0/1393757855 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f63fc103220 0x7f63fc19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:17.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.850+0000 7f6401234640 1 --2- 172.21.15.160:0/1393757855 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f63fc102860 0x7f63fc197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:17.850 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.850+0000 7f6401234640 1 --2- 172.21.15.160:0/1393757855 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63fc101670 0x7f63fc197970 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:17.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.850+0000 7f6401234640 1 -- 172.21.15.160:0/1393757855 >> 172.21.15.160:0/1393757855 conn(0x7f63fc0fcc00 msgr2=0x7f63fc105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:17.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.850+0000 7f6401234640 1 -- 172.21.15.160:0/1393757855 shutdown_connections 2024-03-07T07:39:17.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.850+0000 7f6401234640 1 -- 172.21.15.160:0/1393757855 wait complete. 2024-03-07T07:39:17.867 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-03-07T07:39:17.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.976+0000 7fd2fb66b640 1 -- 172.21.15.160:0/3900390716 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2f4103230 msgr2=0x7fd2f4109ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:17.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.976+0000 7fd2fb66b640 1 --2- 172.21.15.160:0/3900390716 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2f4103230 0x7fd2f4109ac0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec009ca0 tx=0x7fd2ec02c0f0 comp rx=0 tx=0).stop 2024-03-07T07:39:17.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.976+0000 7fd2fb66b640 1 -- 172.21.15.160:0/3900390716 shutdown_connections 2024-03-07T07:39:17.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.976+0000 7fd2fb66b640 1 --2- 172.21.15.160:0/3900390716 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2f4103230 0x7fd2f4109ac0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:17.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.976+0000 7fd2fb66b640 1 --2- 172.21.15.160:0/3900390716 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2f4102870 0x7fd2f4102cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:17.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.976+0000 7fd2fb66b640 1 --2- 172.21.15.160:0/3900390716 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2f4101680 0x7fd2f4101aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:17.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.976+0000 7fd2fb66b640 1 -- 172.21.15.160:0/3900390716 >> 172.21.15.160:0/3900390716 conn(0x7fd2f40fcbd0 msgr2=0x7fd2f40ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:17.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.976+0000 7fd2fb66b640 1 -- 172.21.15.160:0/3900390716 shutdown_connections 2024-03-07T07:39:17.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.976+0000 7fd2fb66b640 1 -- 172.21.15.160:0/3900390716 wait complete. 2024-03-07T07:39:17.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.976+0000 7fd2fb66b640 1 Processor -- start 2024-03-07T07:39:17.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.976+0000 7fd2fb66b640 1 -- start start 2024-03-07T07:39:17.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.977+0000 7fd2fb66b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2f4101680 0x7fd2f4197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:17.980 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.977+0000 7fd2fb66b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2f4102870 0x7fd2f4197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:17.980 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.977+0000 7fd2fb66b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2f4103230 0x7fd2f419eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:17.980 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.977+0000 7fd2fb66b640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f410d2e0 con 0x7fd2f4103230 2024-03-07T07:39:17.980 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.977+0000 7fd2fb66b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f410d160 con 0x7fd2f4101680 2024-03-07T07:39:17.980 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.977+0000 7fd2fb66b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f410d460 con 0x7fd2f4102870 2024-03-07T07:39:17.980 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.977+0000 7fd2f9c08640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2f4103230 0x7fd2f419eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:17.980 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.977+0000 7fd2f9c08640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2f4103230 0x7fd2f419eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:42318/0 (socket says 172.21.15.160:42318) 2024-03-07T07:39:17.980 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.977+0000 7fd2f9c08640 1 -- 172.21.15.160:0/3494546573 learned_addr learned my addr 172.21.15.160:0/3494546573 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:39:17.980 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.977+0000 7fd2f9407640 1 --2- 172.21.15.160:0/3494546573 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2f4101680 0x7fd2f4197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:17.980 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.977+0000 7fd2f8c06640 1 --2- 172.21.15.160:0/3494546573 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2f4102870 0x7fd2f4197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:17.980 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.977+0000 7fd2f9c08640 1 -- 172.21.15.160:0/3494546573 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2f4102870 msgr2=0x7fd2f4197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:17.980 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.977+0000 7fd2f9c08640 1 --2- 172.21.15.160:0/3494546573 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2f4102870 0x7fd2f4197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:17.980 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.977+0000 7fd2f9c08640 1 -- 172.21.15.160:0/3494546573 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2f4101680 msgr2=0x7fd2f4197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:17.981 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.977+0000 7fd2f9c08640 1 --2- 172.21.15.160:0/3494546573 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2f4101680 0x7fd2f4197900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:17.981 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.977+0000 7fd2f9c08640 1 -- 172.21.15.160:0/3494546573 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f419f520 con 0x7fd2f4103230 2024-03-07T07:39:17.981 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.977+0000 7fd2f9c08640 1 --2- 172.21.15.160:0/3494546573 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2f4103230 0x7fd2f419eeb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec037040 tx=0x7fd2ec032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:17.981 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.977+0000 7fd2ea7fc640 1 -- 172.21.15.160:0/3494546573 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec041020 con 0x7fd2f4103230 2024-03-07T07:39:17.981 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.977+0000 7fd2fb66b640 1 -- 172.21.15.160:0/3494546573 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2f419f7b0 con 0x7fd2f4103230 2024-03-07T07:39:17.981 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.977+0000 7fd2ea7fc640 1 -- 172.21.15.160:0/3494546573 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd2ec034d90 con 0x7fd2f4103230 2024-03-07T07:39:17.981 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.977+0000 7fd2ea7fc640 1 -- 172.21.15.160:0/3494546573 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec039040 con 0x7fd2f4103230 2024-03-07T07:39:17.981 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.977+0000 7fd2fb66b640 1 -- 172.21.15.160:0/3494546573 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2f419fcf0 con 0x7fd2f4103230 2024-03-07T07:39:17.981 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.978+0000 7fd2fb66b640 1 -- 172.21.15.160:0/3494546573 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2f41089a0 con 0x7fd2f4103230 2024-03-07T07:39:17.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.979+0000 7fd2ea7fc640 1 -- 172.21.15.160:0/3494546573 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7fd2ec044020 con 0x7fd2f4103230 2024-03-07T07:39:17.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.980+0000 7fd2ea7fc640 1 --2- 172.21.15.160:0/3494546573 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fd2e00a1d80 0x7fd2e00a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:17.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.980+0000 7fd2f9407640 1 --2- 172.21.15.160:0/3494546573 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fd2e00a1d80 0x7fd2e00a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:17.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.980+0000 7fd2ea7fc640 1 -- 172.21.15.160:0/3494546573 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd2ec0eff50 con 0x7fd2f4103230 2024-03-07T07:39:17.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.981+0000 7fd2f9407640 1 --2- 172.21.15.160:0/3494546573 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fd2e00a1d80 0x7fd2e00a4240 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd2f41026d0 tx=0x7fd2f0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:17.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:17.984+0000 7fd2ea7fc640 1 -- 172.21.15.160:0/3494546573 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd2ec04b020 con 0x7fd2f4103230 2024-03-07T07:39:18.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:17 smithi177 ceph-mon[36578]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-07T07:39:18.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:17 smithi177 ceph-mon[36578]: Cluster is now healthy 2024-03-07T07:39:18.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:17 smithi177 ceph-mon[36578]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-07T07:39:18.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:17 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/3394550451' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-07T07:39:18.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:17 smithi177 ceph-mon[36578]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-07T07:39:18.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:17 smithi177 ceph-mon[36578]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-07T07:39:18.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:17 smithi177 ceph-mon[36578]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-03-07T07:39:18.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:17 smithi177 ceph-mon[36578]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-03-07T07:39:18.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:17 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/1393757855' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-07T07:39:18.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:17 smithi160 ceph-mon[31963]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-07T07:39:18.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:17 smithi160 ceph-mon[31963]: Cluster is now healthy 2024-03-07T07:39:18.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:17 smithi160 ceph-mon[31963]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-07T07:39:18.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:17 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/3394550451' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-07T07:39:18.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:17 smithi160 ceph-mon[31963]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-07T07:39:18.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:17 smithi160 ceph-mon[31963]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-07T07:39:18.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:17 smithi160 ceph-mon[31963]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-03-07T07:39:18.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:17 smithi160 ceph-mon[31963]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-03-07T07:39:18.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:17 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/1393757855' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-07T07:39:18.196 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.196+0000 7fd2fb66b640 1 -- 172.21.15.160:0/3494546573 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7fd2f402d030 con 0x7fd2f4103230 2024-03-07T07:39:18.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:17 smithi178 ceph-mon[36650]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-07T07:39:18.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:17 smithi178 ceph-mon[36650]: Cluster is now healthy 2024-03-07T07:39:18.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:17 smithi178 ceph-mon[36650]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-07T07:39:18.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:17 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/3394550451' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-07T07:39:18.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:17 smithi178 ceph-mon[36650]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-07T07:39:18.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:17 smithi178 ceph-mon[36650]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-07T07:39:18.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:17 smithi178 ceph-mon[36650]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-03-07T07:39:18.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:17 smithi178 ceph-mon[36650]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-03-07T07:39:18.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:17 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/1393757855' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-07T07:39:18.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.858+0000 7fd2ea7fc640 1 -- 172.21.15.160:0/3494546573 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v20) v1 ==== 115+0+0 (secure 0 0 0) 0x7fd2ec0bd4d0 con 0x7fd2f4103230 2024-03-07T07:39:18.861 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.860+0000 7fd2fb66b640 1 -- 172.21.15.160:0/3494546573 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fd2e00a1d80 msgr2=0x7fd2e00a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:18.861 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.860+0000 7fd2fb66b640 1 --2- 172.21.15.160:0/3494546573 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fd2e00a1d80 0x7fd2e00a4240 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd2f41026d0 tx=0x7fd2f0005f90 comp rx=0 tx=0).stop 2024-03-07T07:39:18.861 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.860+0000 7fd2fb66b640 1 -- 172.21.15.160:0/3494546573 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2f4103230 msgr2=0x7fd2f419eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:18.861 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.860+0000 7fd2fb66b640 1 --2- 172.21.15.160:0/3494546573 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2f4103230 0x7fd2f419eeb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec037040 tx=0x7fd2ec032f20 comp rx=0 tx=0).stop 2024-03-07T07:39:18.861 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.860+0000 7fd2fb66b640 1 -- 172.21.15.160:0/3494546573 shutdown_connections 2024-03-07T07:39:18.861 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.860+0000 7fd2fb66b640 1 --2- 172.21.15.160:0/3494546573 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fd2e00a1d80 0x7fd2e00a4240 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:18.861 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.860+0000 7fd2fb66b640 1 --2- 172.21.15.160:0/3494546573 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2f4103230 0x7fd2f419eeb0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:18.861 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.860+0000 7fd2fb66b640 1 --2- 172.21.15.160:0/3494546573 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2f4102870 0x7fd2f4197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:18.863 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.860+0000 7fd2fb66b640 1 --2- 172.21.15.160:0/3494546573 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2f4101680 0x7fd2f4197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:18.863 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.860+0000 7fd2fb66b640 1 -- 172.21.15.160:0/3494546573 >> 172.21.15.160:0/3494546573 conn(0x7fd2f40fcbd0 msgr2=0x7fd2f4105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:18.863 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.860+0000 7fd2fb66b640 1 -- 172.21.15.160:0/3494546573 shutdown_connections 2024-03-07T07:39:18.863 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.860+0000 7fd2fb66b640 1 -- 172.21.15.160:0/3494546573 wait complete. 2024-03-07T07:39:18.874 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-03-07T07:39:18.874 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-03-07T07:39:18.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.982+0000 7efd74710640 1 -- 172.21.15.160:0/1342667828 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd6c102c80 msgr2=0x7efd6c1030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:18.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.982+0000 7efd74710640 1 --2- 172.21.15.160:0/1342667828 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd6c102c80 0x7efd6c1030a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7efd5c009900 tx=0x7efd5c02bc30 comp rx=0 tx=0).stop 2024-03-07T07:39:18.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.982+0000 7efd74710640 1 -- 172.21.15.160:0/1342667828 shutdown_connections 2024-03-07T07:39:18.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.982+0000 7efd74710640 1 --2- 172.21.15.160:0/1342667828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd6c1000b0 0x7efd6c109c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:18.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.982+0000 7efd74710640 1 --2- 172.21.15.160:0/1342667828 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd6c0ff6f0 0x7efd6c0ffb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:18.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.982+0000 7efd74710640 1 --2- 172.21.15.160:0/1342667828 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd6c102c80 0x7efd6c1030a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:18.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.982+0000 7efd74710640 1 -- 172.21.15.160:0/1342667828 >> 172.21.15.160:0/1342667828 conn(0x7efd6c0fb250 msgr2=0x7efd6c0fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:18.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.982+0000 7efd74710640 1 -- 172.21.15.160:0/1342667828 shutdown_connections 2024-03-07T07:39:18.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.982+0000 7efd74710640 1 -- 172.21.15.160:0/1342667828 wait complete. 2024-03-07T07:39:18.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.982+0000 7efd74710640 1 Processor -- start 2024-03-07T07:39:18.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.982+0000 7efd74710640 1 -- start start 2024-03-07T07:39:18.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.983+0000 7efd74710640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd6c0ff6f0 0x7efd6c1979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:18.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.983+0000 7efd74710640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd6c1000b0 0x7efd6c197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:18.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.983+0000 7efd74710640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd6c102c80 0x7efd6c19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:18.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.983+0000 7efd74710640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c10d300 con 0x7efd6c1000b0 2024-03-07T07:39:18.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.983+0000 7efd74710640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c10d180 con 0x7efd6c0ff6f0 2024-03-07T07:39:18.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.983+0000 7efd74710640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c10d480 con 0x7efd6c102c80 2024-03-07T07:39:18.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.983+0000 7efd71cab640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd6c1000b0 0x7efd6c197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:18.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.983+0000 7efd71cab640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd6c1000b0 0x7efd6c197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:42338/0 (socket says 172.21.15.160:42338) 2024-03-07T07:39:18.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.983+0000 7efd71cab640 1 -- 172.21.15.160:0/3194614291 learned_addr learned my addr 172.21.15.160:0/3194614291 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:39:18.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.983+0000 7efd724ac640 1 --2- 172.21.15.160:0/3194614291 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd6c0ff6f0 0x7efd6c1979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:18.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.983+0000 7efd72cad640 1 --2- 172.21.15.160:0/3194614291 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd6c102c80 0x7efd6c19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:18.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.983+0000 7efd71cab640 1 -- 172.21.15.160:0/3194614291 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd6c102c80 msgr2=0x7efd6c19ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:18.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.983+0000 7efd71cab640 1 --2- 172.21.15.160:0/3194614291 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd6c102c80 0x7efd6c19ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:18.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.983+0000 7efd71cab640 1 -- 172.21.15.160:0/3194614291 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd6c0ff6f0 msgr2=0x7efd6c1979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:18.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.983+0000 7efd71cab640 1 --2- 172.21.15.160:0/3194614291 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd6c0ff6f0 0x7efd6c1979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:18.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.983+0000 7efd71cab640 1 -- 172.21.15.160:0/3194614291 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd6c19f5d0 con 0x7efd6c1000b0 2024-03-07T07:39:18.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.983+0000 7efd71cab640 1 --2- 172.21.15.160:0/3194614291 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd6c1000b0 0x7efd6c197ef0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efd6800e9f0 tx=0x7efd6800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:18.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.983+0000 7efd637fe640 1 -- 172.21.15.160:0/3194614291 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6800cd80 con 0x7efd6c1000b0 2024-03-07T07:39:18.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.983+0000 7efd74710640 1 -- 172.21.15.160:0/3194614291 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd6c19f8c0 con 0x7efd6c1000b0 2024-03-07T07:39:18.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.983+0000 7efd637fe640 1 -- 172.21.15.160:0/3194614291 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efd68005500 con 0x7efd6c1000b0 2024-03-07T07:39:18.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.983+0000 7efd637fe640 1 -- 172.21.15.160:0/3194614291 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd68010690 con 0x7efd6c1000b0 2024-03-07T07:39:18.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.983+0000 7efd74710640 1 -- 172.21.15.160:0/3194614291 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd6c19fe10 con 0x7efd6c1000b0 2024-03-07T07:39:18.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.984+0000 7efd74710640 1 -- 172.21.15.160:0/3194614291 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd6c108b60 con 0x7efd6c1000b0 2024-03-07T07:39:18.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.985+0000 7efd637fe640 1 -- 172.21.15.160:0/3194614291 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7efd680046c0 con 0x7efd6c1000b0 2024-03-07T07:39:18.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.986+0000 7efd637fe640 1 --2- 172.21.15.160:0/3194614291 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7efd580a1d80 0x7efd580a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:18.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.986+0000 7efd724ac640 1 --2- 172.21.15.160:0/3194614291 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7efd580a1d80 0x7efd580a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:18.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.986+0000 7efd637fe640 1 -- 172.21.15.160:0/3194614291 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7efd680ce050 con 0x7efd6c1000b0 2024-03-07T07:39:18.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.986+0000 7efd724ac640 1 --2- 172.21.15.160:0/3194614291 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7efd580a1d80 0x7efd580a4240 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efd5c000c00 tx=0x7efd5c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:18.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:18.990+0000 7efd637fe640 1 -- 172.21.15.160:0/3194614291 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7efd6809b5d0 con 0x7efd6c1000b0 2024-03-07T07:39:19.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:18 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/1393757855' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-07T07:39:19.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:18 smithi177 ceph-mon[36578]: mds.? [v2:172.21.15.177:6838/3329773838,v1:172.21.15.177:6839/3329773838] up:active 2024-03-07T07:39:19.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:18 smithi177 ceph-mon[36578]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-07T07:39:19.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:18 smithi177 ceph-mon[36578]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-07T07:39:19.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:18 smithi177 ceph-mon[36578]: Cluster is now healthy 2024-03-07T07:39:19.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:18 smithi177 ceph-mon[36578]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-03-07T07:39:19.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:18 smithi177 ceph-mon[36578]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-03-07T07:39:19.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:18 smithi177 ceph-mon[36578]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-03-07T07:39:19.107 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:18 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/3494546573' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-07T07:39:19.108 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:18 smithi177 ceph-mon[36578]: pgmap v290: 97 pgs: 33 creating+peering, 16 unknown, 48 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 187 B/s wr, 0 op/s 2024-03-07T07:39:19.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:18 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/1393757855' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-07T07:39:19.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:18 smithi160 ceph-mon[31963]: mds.? [v2:172.21.15.177:6838/3329773838,v1:172.21.15.177:6839/3329773838] up:active 2024-03-07T07:39:19.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:18 smithi160 ceph-mon[31963]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-07T07:39:19.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:18 smithi160 ceph-mon[31963]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-07T07:39:19.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:18 smithi160 ceph-mon[31963]: Cluster is now healthy 2024-03-07T07:39:19.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:18 smithi160 ceph-mon[31963]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-03-07T07:39:19.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:18 smithi160 ceph-mon[31963]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-03-07T07:39:19.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:18 smithi160 ceph-mon[31963]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-03-07T07:39:19.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:18 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/3494546573' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-07T07:39:19.179 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:18 smithi160 ceph-mon[31963]: pgmap v290: 97 pgs: 33 creating+peering, 16 unknown, 48 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 187 B/s wr, 0 op/s 2024-03-07T07:39:19.189 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.189+0000 7efd74710640 1 -- 172.21.15.160:0/3194614291 --> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7efd6c061aa0 con 0x7efd580a1d80 2024-03-07T07:39:19.206 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:18 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/1393757855' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-07T07:39:19.206 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:18 smithi178 ceph-mon[36650]: mds.? [v2:172.21.15.177:6838/3329773838,v1:172.21.15.177:6839/3329773838] up:active 2024-03-07T07:39:19.206 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:18 smithi178 ceph-mon[36650]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-07T07:39:19.206 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:18 smithi178 ceph-mon[36650]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-07T07:39:19.206 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:18 smithi178 ceph-mon[36650]: Cluster is now healthy 2024-03-07T07:39:19.206 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:18 smithi178 ceph-mon[36650]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-03-07T07:39:19.207 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:18 smithi178 ceph-mon[36650]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-03-07T07:39:19.207 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:18 smithi178 ceph-mon[36650]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-03-07T07:39:19.207 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:18 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/3494546573' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-07T07:39:19.207 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:18 smithi178 ceph-mon[36650]: pgmap v290: 97 pgs: 33 creating+peering, 16 unknown, 48 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 187 B/s wr, 0 op/s 2024-03-07T07:39:19.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.210+0000 7efd637fe640 1 -- 172.21.15.160:0/3194614291 <== mgr.14152 v2:172.21.15.160:6800/419630619 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7efd6c061aa0 con 0x7efd580a1d80 2024-03-07T07:39:19.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.212+0000 7efd74710640 1 -- 172.21.15.160:0/3194614291 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7efd580a1d80 msgr2=0x7efd580a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:19.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.212+0000 7efd74710640 1 --2- 172.21.15.160:0/3194614291 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7efd580a1d80 0x7efd580a4240 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efd5c000c00 tx=0x7efd5c005fd0 comp rx=0 tx=0).stop 2024-03-07T07:39:19.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.212+0000 7efd74710640 1 -- 172.21.15.160:0/3194614291 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd6c1000b0 msgr2=0x7efd6c197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:19.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.212+0000 7efd74710640 1 --2- 172.21.15.160:0/3194614291 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd6c1000b0 0x7efd6c197ef0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efd6800e9f0 tx=0x7efd6800edf0 comp rx=0 tx=0).stop 2024-03-07T07:39:19.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.212+0000 7efd74710640 1 -- 172.21.15.160:0/3194614291 shutdown_connections 2024-03-07T07:39:19.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.212+0000 7efd74710640 1 --2- 172.21.15.160:0/3194614291 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7efd580a1d80 0x7efd580a4240 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:19.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.212+0000 7efd74710640 1 --2- 172.21.15.160:0/3194614291 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd6c102c80 0x7efd6c19ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:19.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.212+0000 7efd74710640 1 --2- 172.21.15.160:0/3194614291 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd6c1000b0 0x7efd6c197ef0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:19.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.212+0000 7efd74710640 1 --2- 172.21.15.160:0/3194614291 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd6c0ff6f0 0x7efd6c1979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:19.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.212+0000 7efd74710640 1 -- 172.21.15.160:0/3194614291 >> 172.21.15.160:0/3194614291 conn(0x7efd6c0fb250 msgr2=0x7efd6c105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:19.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.212+0000 7efd74710640 1 -- 172.21.15.160:0/3194614291 shutdown_connections 2024-03-07T07:39:19.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.212+0000 7efd74710640 1 -- 172.21.15.160:0/3194614291 wait complete. 2024-03-07T07:39:19.224 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-03-07T07:39:19.333 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.331+0000 7f495cd07640 1 -- 172.21.15.160:0/1380510851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4958102930 msgr2=0x7f4958102db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:19.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.331+0000 7f495cd07640 1 --2- 172.21.15.160:0/1380510851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4958102930 0x7f4958102db0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f494c009930 tx=0x7f494c02bc30 comp rx=0 tx=0).stop 2024-03-07T07:39:19.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.331+0000 7f495cd07640 1 -- 172.21.15.160:0/1380510851 shutdown_connections 2024-03-07T07:39:19.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.331+0000 7f495cd07640 1 --2- 172.21.15.160:0/1380510851 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49581032f0 0x7f4958109b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:19.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.331+0000 7f495cd07640 1 --2- 172.21.15.160:0/1380510851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4958102930 0x7f4958102db0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:19.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.331+0000 7f495cd07640 1 --2- 172.21.15.160:0/1380510851 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49580ff960 0x7f49580ffd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:19.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.331+0000 7f495cd07640 1 -- 172.21.15.160:0/1380510851 >> 172.21.15.160:0/1380510851 conn(0x7f49580fb290 msgr2=0x7f49580fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:19.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.331+0000 7f495cd07640 1 -- 172.21.15.160:0/1380510851 shutdown_connections 2024-03-07T07:39:19.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.331+0000 7f495cd07640 1 -- 172.21.15.160:0/1380510851 wait complete. 2024-03-07T07:39:19.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.332+0000 7f495cd07640 1 Processor -- start 2024-03-07T07:39:19.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.332+0000 7f495cd07640 1 -- start start 2024-03-07T07:39:19.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.332+0000 7f495cd07640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49580ff960 0x7f4958197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:19.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.332+0000 7f495cd07640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4958102930 0x7f4958197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:19.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.332+0000 7f495cd07640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49581032f0 0x7f495819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:19.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.332+0000 7f495cd07640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495810d340 con 0x7f49580ff960 2024-03-07T07:39:19.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.332+0000 7f495cd07640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495810d1c0 con 0x7f4958102930 2024-03-07T07:39:19.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.332+0000 7f495cd07640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495810d4c0 con 0x7f49581032f0 2024-03-07T07:39:19.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.332+0000 7f495659c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49580ff960 0x7f4958197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:19.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.332+0000 7f495659c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49580ff960 0x7f4958197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:42360/0 (socket says 172.21.15.160:42360) 2024-03-07T07:39:19.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.332+0000 7f495659c640 1 -- 172.21.15.160:0/2355313993 learned_addr learned my addr 172.21.15.160:0/2355313993 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:39:19.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.332+0000 7f4956d9d640 1 --2- 172.21.15.160:0/2355313993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49581032f0 0x7f495819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:19.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.332+0000 7f4955d9b640 1 --2- 172.21.15.160:0/2355313993 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4958102930 0x7f4958197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:19.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.332+0000 7f495659c640 1 -- 172.21.15.160:0/2355313993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49581032f0 msgr2=0x7f495819f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:19.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.332+0000 7f495659c640 1 --2- 172.21.15.160:0/2355313993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49581032f0 0x7f495819f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:19.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.332+0000 7f495659c640 1 -- 172.21.15.160:0/2355313993 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4958102930 msgr2=0x7f4958197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:19.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.332+0000 7f495659c640 1 --2- 172.21.15.160:0/2355313993 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4958102930 0x7f4958197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:19.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.332+0000 7f495659c640 1 -- 172.21.15.160:0/2355313993 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f495819f680 con 0x7f49580ff960 2024-03-07T07:39:19.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.332+0000 7f4955d9b640 1 --2- 172.21.15.160:0/2355313993 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4958102930 0x7f4958197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:39:19.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.333+0000 7f495659c640 1 --2- 172.21.15.160:0/2355313993 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49580ff960 0x7f4958197a60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f494800e9c0 tx=0x7f494800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:19.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.333+0000 7f49477fe640 1 -- 172.21.15.160:0/2355313993 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494800cd80 con 0x7f49580ff960 2024-03-07T07:39:19.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.333+0000 7f495cd07640 1 -- 172.21.15.160:0/2355313993 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f495819f970 con 0x7f49580ff960 2024-03-07T07:39:19.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.333+0000 7f49477fe640 1 -- 172.21.15.160:0/2355313993 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f49480035f0 con 0x7f49580ff960 2024-03-07T07:39:19.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.333+0000 7f49477fe640 1 -- 172.21.15.160:0/2355313993 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4948010690 con 0x7f49580ff960 2024-03-07T07:39:19.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.333+0000 7f495cd07640 1 -- 172.21.15.160:0/2355313993 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f495819fe40 con 0x7f49580ff960 2024-03-07T07:39:19.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.334+0000 7f495cd07640 1 -- 172.21.15.160:0/2355313993 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4958108b30 con 0x7f49580ff960 2024-03-07T07:39:19.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.335+0000 7f49477fe640 1 -- 172.21.15.160:0/2355313993 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f49480037a0 con 0x7f49580ff960 2024-03-07T07:39:19.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.335+0000 7f49477fe640 1 --2- 172.21.15.160:0/2355313993 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f49400a1d80 0x7f49400a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:19.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.335+0000 7f4955d9b640 1 --2- 172.21.15.160:0/2355313993 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f49400a1d80 0x7f49400a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:19.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.335+0000 7f49477fe640 1 -- 172.21.15.160:0/2355313993 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f49480ce060 con 0x7f49580ff960 2024-03-07T07:39:19.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.336+0000 7f4955d9b640 1 --2- 172.21.15.160:0/2355313993 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f49400a1d80 0x7f49400a4240 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f494c009900 tx=0x7f494c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:19.340 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.340+0000 7f49477fe640 1 -- 172.21.15.160:0/2355313993 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f494809b5e0 con 0x7f49580ff960 2024-03-07T07:39:19.540 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.540+0000 7f495cd07640 1 -- 172.21.15.160:0/2355313993 --> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f4958061aa0 con 0x7f49400a1d80 2024-03-07T07:39:19.562 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.562+0000 7f49477fe640 1 -- 172.21.15.160:0/2355313993 <== mgr.14152 v2:172.21.15.160:6800/419630619 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f4958061aa0 con 0x7f49400a1d80 2024-03-07T07:39:19.564 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.564+0000 7f495cd07640 1 -- 172.21.15.160:0/2355313993 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f49400a1d80 msgr2=0x7f49400a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:19.565 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.564+0000 7f495cd07640 1 --2- 172.21.15.160:0/2355313993 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f49400a1d80 0x7f49400a4240 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f494c009900 tx=0x7f494c005fd0 comp rx=0 tx=0).stop 2024-03-07T07:39:19.565 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.564+0000 7f495cd07640 1 -- 172.21.15.160:0/2355313993 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49580ff960 msgr2=0x7f4958197a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:19.565 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.564+0000 7f495cd07640 1 --2- 172.21.15.160:0/2355313993 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49580ff960 0x7f4958197a60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f494800e9c0 tx=0x7f494800ee80 comp rx=0 tx=0).stop 2024-03-07T07:39:19.565 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.564+0000 7f495cd07640 1 -- 172.21.15.160:0/2355313993 shutdown_connections 2024-03-07T07:39:19.565 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.564+0000 7f495cd07640 1 --2- 172.21.15.160:0/2355313993 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f49400a1d80 0x7f49400a4240 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:19.565 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.564+0000 7f495cd07640 1 --2- 172.21.15.160:0/2355313993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49581032f0 0x7f495819f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:19.565 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.564+0000 7f495cd07640 1 --2- 172.21.15.160:0/2355313993 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4958102930 0x7f4958197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:19.565 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.564+0000 7f495cd07640 1 --2- 172.21.15.160:0/2355313993 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49580ff960 0x7f4958197a60 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:19.565 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.564+0000 7f495cd07640 1 -- 172.21.15.160:0/2355313993 >> 172.21.15.160:0/2355313993 conn(0x7f49580fb290 msgr2=0x7f4958105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:19.565 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.564+0000 7f495cd07640 1 -- 172.21.15.160:0/2355313993 shutdown_connections 2024-03-07T07:39:19.565 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.564+0000 7f495cd07640 1 -- 172.21.15.160:0/2355313993 wait complete. 2024-03-07T07:39:19.574 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:39:19 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:39:19.194+0000 7fc506a65700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-07T07:39:19.575 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:39:19 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:39:19.194+0000 7fc506a65700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-07T07:39:19.575 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:39:19 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:39:19.194+0000 7fc506a65700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-07T07:39:19.575 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:39:19 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:39:19.194+0000 7fc506a65700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-07T07:39:19.575 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:39:19 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:39:19.194+0000 7fc506a65700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-07T07:39:19.576 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-03-07T07:39:19.686 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.682+0000 7f7faaf1d640 1 -- 172.21.15.160:0/3631287837 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7fa41073c0 msgr2=0x7f7fa41097b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:19.686 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.682+0000 7f7faaf1d640 1 --2- 172.21.15.160:0/3631287837 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7fa41073c0 0x7f7fa41097b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c009ca0 tx=0x7f7f9c02c0f0 comp rx=0 tx=0).stop 2024-03-07T07:39:19.686 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.683+0000 7f7faaf1d640 1 -- 172.21.15.160:0/3631287837 shutdown_connections 2024-03-07T07:39:19.686 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.683+0000 7f7faaf1d640 1 --2- 172.21.15.160:0/3631287837 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7fa41073c0 0x7f7fa41097b0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:19.686 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.683+0000 7f7faaf1d640 1 --2- 172.21.15.160:0/3631287837 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7fa4104a90 0x7f7fa4106e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:19.686 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.683+0000 7f7faaf1d640 1 --2- 172.21.15.160:0/3631287837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fa4102160 0x7f7fa4104550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:19.686 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.683+0000 7f7faaf1d640 1 -- 172.21.15.160:0/3631287837 >> 172.21.15.160:0/3631287837 conn(0x7f7fa40fbad0 msgr2=0x7f7fa40fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:19.686 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.683+0000 7f7faaf1d640 1 -- 172.21.15.160:0/3631287837 shutdown_connections 2024-03-07T07:39:19.686 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.683+0000 7f7faaf1d640 1 -- 172.21.15.160:0/3631287837 wait complete. 2024-03-07T07:39:19.686 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.683+0000 7f7faaf1d640 1 Processor -- start 2024-03-07T07:39:19.687 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.683+0000 7f7faaf1d640 1 -- start start 2024-03-07T07:39:19.687 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.683+0000 7f7faaf1d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7fa4102160 0x7f7fa41979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:19.687 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.683+0000 7f7faaf1d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fa4104a90 0x7f7fa4197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:19.687 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.683+0000 7f7faaf1d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7fa41073c0 0x7f7fa419ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:19.687 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.683+0000 7f7faaf1d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa410d300 con 0x7f7fa41073c0 2024-03-07T07:39:19.687 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.683+0000 7f7faaf1d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa410d180 con 0x7f7fa4102160 2024-03-07T07:39:19.687 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.683+0000 7f7faaf1d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa410d480 con 0x7f7fa4104a90 2024-03-07T07:39:19.687 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.683+0000 7f7fa94ba640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7fa41073c0 0x7f7fa419ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:19.687 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.683+0000 7f7fa3fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fa4104a90 0x7f7fa4197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:19.687 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.683+0000 7f7fa94ba640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7fa41073c0 0x7f7fa419ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:49634/0 (socket says 172.21.15.160:49634) 2024-03-07T07:39:19.688 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.683+0000 7f7fa94ba640 1 -- 172.21.15.160:0/1557377620 learned_addr learned my addr 172.21.15.160:0/1557377620 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:39:19.688 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.683+0000 7f7fa3fff640 1 --2- 172.21.15.160:0/1557377620 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fa4104a90 0x7f7fa4197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.160:57962/0 (socket says 172.21.15.160:57962) 2024-03-07T07:39:19.688 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.683+0000 7f7fa8cb9640 1 --2- 172.21.15.160:0/1557377620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7fa4102160 0x7f7fa41979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:19.688 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.684+0000 7f7fa94ba640 1 -- 172.21.15.160:0/1557377620 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fa4104a90 msgr2=0x7f7fa4197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:19.688 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.684+0000 7f7fa94ba640 1 --2- 172.21.15.160:0/1557377620 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fa4104a90 0x7f7fa4197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:19.688 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.684+0000 7f7fa94ba640 1 -- 172.21.15.160:0/1557377620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7fa4102160 msgr2=0x7f7fa41979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:19.688 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.684+0000 7f7fa94ba640 1 --2- 172.21.15.160:0/1557377620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7fa4102160 0x7f7fa41979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:19.688 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.684+0000 7f7fa94ba640 1 -- 172.21.15.160:0/1557377620 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa419f5c0 con 0x7f7fa41073c0 2024-03-07T07:39:19.688 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.684+0000 7f7fa3fff640 1 --2- 172.21.15.160:0/1557377620 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fa4104a90 0x7f7fa4197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:39:19.688 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.684+0000 7f7fa94ba640 1 --2- 172.21.15.160:0/1557377620 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7fa41073c0 0x7f7fa419ef50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c037040 tx=0x7f7f9c032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:19.688 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.684+0000 7f7fa1ffb640 1 -- 172.21.15.160:0/1557377620 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c041020 con 0x7f7fa41073c0 2024-03-07T07:39:19.688 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.684+0000 7f7fa1ffb640 1 -- 172.21.15.160:0/1557377620 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f9c034cd0 con 0x7f7fa41073c0 2024-03-07T07:39:19.688 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.684+0000 7f7faaf1d640 1 -- 172.21.15.160:0/1557377620 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fa419f850 con 0x7f7fa41073c0 2024-03-07T07:39:19.689 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.684+0000 7f7fa1ffb640 1 -- 172.21.15.160:0/1557377620 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c039040 con 0x7f7fa41073c0 2024-03-07T07:39:19.689 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.684+0000 7f7faaf1d640 1 -- 172.21.15.160:0/1557377620 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fa419fd40 con 0x7f7fa41073c0 2024-03-07T07:39:19.689 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.685+0000 7f7faaf1d640 1 -- 172.21.15.160:0/1557377620 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fa40fd5c0 con 0x7f7fa41073c0 2024-03-07T07:39:19.690 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.686+0000 7f7fa1ffb640 1 -- 172.21.15.160:0/1557377620 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f7f9c044020 con 0x7f7fa41073c0 2024-03-07T07:39:19.690 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.687+0000 7f7fa1ffb640 1 --2- 172.21.15.160:0/1557377620 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f7f8c0a1d80 0x7f7f8c0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:19.690 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.687+0000 7f7fa8cb9640 1 --2- 172.21.15.160:0/1557377620 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f7f8c0a1d80 0x7f7f8c0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:19.690 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.687+0000 7f7fa1ffb640 1 -- 172.21.15.160:0/1557377620 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f7f9c0efab0 con 0x7f7fa41073c0 2024-03-07T07:39:19.690 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.687+0000 7f7fa8cb9640 1 --2- 172.21.15.160:0/1557377620 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f7f8c0a1d80 0x7f7f8c0a4240 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7f94009bd0 tx=0x7f7f94005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:19.691 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.691+0000 7f7fa1ffb640 1 -- 172.21.15.160:0/1557377620 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7f9c04b020 con 0x7f7fa41073c0 2024-03-07T07:39:19.897 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.896+0000 7f7faaf1d640 1 -- 172.21.15.160:0/1557377620 --> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f7fa4061b30 con 0x7f7f8c0a1d80 2024-03-07T07:39:19.900 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.900+0000 7f7fa1ffb640 1 -- 172.21.15.160:0/1557377620 <== mgr.14152 v2:172.21.15.160:6800/419630619 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f7fa4061b30 con 0x7f7f8c0a1d80 2024-03-07T07:39:19.901 INFO:teuthology.orchestra.run.smithi160.stdout:/volumes/qa/sv_0/54303028-814d-418f-9a63-596c0cbbc6ed 2024-03-07T07:39:19.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.902+0000 7f7faaf1d640 1 -- 172.21.15.160:0/1557377620 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f7f8c0a1d80 msgr2=0x7f7f8c0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:19.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.902+0000 7f7faaf1d640 1 --2- 172.21.15.160:0/1557377620 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f7f8c0a1d80 0x7f7f8c0a4240 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7f94009bd0 tx=0x7f7f94005f90 comp rx=0 tx=0).stop 2024-03-07T07:39:19.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.902+0000 7f7faaf1d640 1 -- 172.21.15.160:0/1557377620 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7fa41073c0 msgr2=0x7f7fa419ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:19.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.902+0000 7f7faaf1d640 1 --2- 172.21.15.160:0/1557377620 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7fa41073c0 0x7f7fa419ef50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c037040 tx=0x7f7f9c032f40 comp rx=0 tx=0).stop 2024-03-07T07:39:19.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.902+0000 7f7faaf1d640 1 -- 172.21.15.160:0/1557377620 shutdown_connections 2024-03-07T07:39:19.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.902+0000 7f7faaf1d640 1 --2- 172.21.15.160:0/1557377620 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f7f8c0a1d80 0x7f7f8c0a4240 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:19.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.902+0000 7f7faaf1d640 1 --2- 172.21.15.160:0/1557377620 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7fa41073c0 0x7f7fa419ef50 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:19.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.902+0000 7f7faaf1d640 1 --2- 172.21.15.160:0/1557377620 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fa4104a90 0x7f7fa4197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:19.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.902+0000 7f7faaf1d640 1 --2- 172.21.15.160:0/1557377620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7fa4102160 0x7f7fa41979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:19.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.902+0000 7f7faaf1d640 1 -- 172.21.15.160:0/1557377620 >> 172.21.15.160:0/1557377620 conn(0x7f7fa40fbad0 msgr2=0x7f7fa4102ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:19.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.902+0000 7f7faaf1d640 1 -- 172.21.15.160:0/1557377620 shutdown_connections 2024-03-07T07:39:19.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:19.902+0000 7f7faaf1d640 1 -- 172.21.15.160:0/1557377620 wait complete. 2024-03-07T07:39:19.914 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-03-07T07:39:20.023 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.021+0000 7fb0409c9640 1 -- 172.21.15.160:0/3123835650 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb03c102130 msgr2=0x7fb03c1089c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:20.023 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.021+0000 7fb0409c9640 1 --2- 172.21.15.160:0/3123835650 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb03c102130 0x7fb03c1089c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb034009ca0 tx=0x7fb03402c0f0 comp rx=0 tx=0).stop 2024-03-07T07:39:20.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.021+0000 7fb0409c9640 1 -- 172.21.15.160:0/3123835650 shutdown_connections 2024-03-07T07:39:20.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.021+0000 7fb0409c9640 1 --2- 172.21.15.160:0/3123835650 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb03c102130 0x7fb03c1089c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:20.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.021+0000 7fb0409c9640 1 --2- 172.21.15.160:0/3123835650 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb03c101770 0x7fb03c101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:20.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.021+0000 7fb0409c9640 1 --2- 172.21.15.160:0/3123835650 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb03c100580 0x7fb03c1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:20.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.021+0000 7fb0409c9640 1 -- 172.21.15.160:0/3123835650 >> 172.21.15.160:0/3123835650 conn(0x7fb03c0fbb10 msgr2=0x7fb03c0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:20.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.021+0000 7fb0409c9640 1 -- 172.21.15.160:0/3123835650 shutdown_connections 2024-03-07T07:39:20.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.021+0000 7fb0409c9640 1 -- 172.21.15.160:0/3123835650 wait complete. 2024-03-07T07:39:20.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.022+0000 7fb0409c9640 1 Processor -- start 2024-03-07T07:39:20.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.022+0000 7fb0409c9640 1 -- start start 2024-03-07T07:39:20.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.022+0000 7fb0409c9640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb03c100580 0x7fb03c196810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:20.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.022+0000 7fb0409c9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb03c101770 0x7fb03c196d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:20.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.022+0000 7fb0409c9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb03c102130 0x7fb03c19ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:20.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.022+0000 7fb0409c9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c10c210 con 0x7fb03c102130 2024-03-07T07:39:20.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.022+0000 7fb0409c9640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c10c090 con 0x7fb03c100580 2024-03-07T07:39:20.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.022+0000 7fb0409c9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c10c390 con 0x7fb03c101770 2024-03-07T07:39:20.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.022+0000 7fb03ad9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb03c102130 0x7fb03c19ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:20.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.022+0000 7fb03a59c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb03c100580 0x7fb03c196810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:20.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.022+0000 7fb03ad9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb03c102130 0x7fb03c19ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:49646/0 (socket says 172.21.15.160:49646) 2024-03-07T07:39:20.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.022+0000 7fb03ad9d640 1 -- 172.21.15.160:0/834596745 learned_addr learned my addr 172.21.15.160:0/834596745 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:39:20.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.022+0000 7fb031d9b640 1 --2- 172.21.15.160:0/834596745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb03c101770 0x7fb03c196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:20.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.022+0000 7fb03ad9d640 1 -- 172.21.15.160:0/834596745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb03c101770 msgr2=0x7fb03c196d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:20.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.022+0000 7fb03ad9d640 1 --2- 172.21.15.160:0/834596745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb03c101770 0x7fb03c196d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:20.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.022+0000 7fb03ad9d640 1 -- 172.21.15.160:0/834596745 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb03c100580 msgr2=0x7fb03c196810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:20.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.022+0000 7fb03ad9d640 1 --2- 172.21.15.160:0/834596745 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb03c100580 0x7fb03c196810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:20.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.022+0000 7fb03ad9d640 1 -- 172.21.15.160:0/834596745 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb03c19e430 con 0x7fb03c102130 2024-03-07T07:39:20.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.023+0000 7fb03ad9d640 1 --2- 172.21.15.160:0/834596745 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb03c102130 0x7fb03c19ddc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb034037040 tx=0x7fb034032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:20.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.023+0000 7fb033fff640 1 -- 172.21.15.160:0/834596745 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb034041020 con 0x7fb03c102130 2024-03-07T07:39:20.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.023+0000 7fb033fff640 1 -- 172.21.15.160:0/834596745 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb034034cd0 con 0x7fb03c102130 2024-03-07T07:39:20.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.023+0000 7fb0409c9640 1 -- 172.21.15.160:0/834596745 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb03c19e6c0 con 0x7fb03c102130 2024-03-07T07:39:20.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.023+0000 7fb033fff640 1 -- 172.21.15.160:0/834596745 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb034039040 con 0x7fb03c102130 2024-03-07T07:39:20.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.023+0000 7fb0409c9640 1 -- 172.21.15.160:0/834596745 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb03c19ebb0 con 0x7fb03c102130 2024-03-07T07:39:20.027 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.024+0000 7fb0409c9640 1 -- 172.21.15.160:0/834596745 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb03c1078a0 con 0x7fb03c102130 2024-03-07T07:39:20.027 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.025+0000 7fb033fff640 1 -- 172.21.15.160:0/834596745 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7fb034044020 con 0x7fb03c102130 2024-03-07T07:39:20.027 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.025+0000 7fb033fff640 1 --2- 172.21.15.160:0/834596745 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fb0280a1d80 0x7fb0280a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:20.027 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.025+0000 7fb03a59c640 1 --2- 172.21.15.160:0/834596745 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fb0280a1d80 0x7fb0280a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:20.027 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.025+0000 7fb033fff640 1 -- 172.21.15.160:0/834596745 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb0340efab0 con 0x7fb03c102130 2024-03-07T07:39:20.027 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.026+0000 7fb03a59c640 1 --2- 172.21.15.160:0/834596745 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fb0280a1d80 0x7fb0280a4240 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb03c1015d0 tx=0x7fb02c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:20.029 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.029+0000 7fb033fff640 1 -- 172.21.15.160:0/834596745 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb03404b020 con 0x7fb03c102130 2024-03-07T07:39:20.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:19 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/3494546573' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-07T07:39:20.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:19 smithi160 ceph-mon[31963]: mds.? [v2:172.21.15.178:6836/1247829154,v1:172.21.15.178:6837/1247829154] up:active 2024-03-07T07:39:20.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:19 smithi160 ceph-mon[31963]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-03-07T07:39:20.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:19 smithi160 ceph-mon[31963]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-03-07T07:39:20.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:19 smithi160 ceph-mon[31963]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:39:20.232 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.232+0000 7fb0409c9640 1 -- 172.21.15.160:0/834596745 --> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fb03c061b30 con 0x7fb0280a1d80 2024-03-07T07:39:20.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.253+0000 7fb033fff640 1 -- 172.21.15.160:0/834596745 <== mgr.14152 v2:172.21.15.160:6800/419630619 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fb03c061b30 con 0x7fb0280a1d80 2024-03-07T07:39:20.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.255+0000 7fb0409c9640 1 -- 172.21.15.160:0/834596745 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fb0280a1d80 msgr2=0x7fb0280a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:20.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.255+0000 7fb0409c9640 1 --2- 172.21.15.160:0/834596745 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fb0280a1d80 0x7fb0280a4240 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb03c1015d0 tx=0x7fb02c005f90 comp rx=0 tx=0).stop 2024-03-07T07:39:20.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.255+0000 7fb0409c9640 1 -- 172.21.15.160:0/834596745 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb03c102130 msgr2=0x7fb03c19ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:20.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.255+0000 7fb0409c9640 1 --2- 172.21.15.160:0/834596745 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb03c102130 0x7fb03c19ddc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb034037040 tx=0x7fb034032f40 comp rx=0 tx=0).stop 2024-03-07T07:39:20.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.255+0000 7fb0409c9640 1 -- 172.21.15.160:0/834596745 shutdown_connections 2024-03-07T07:39:20.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.255+0000 7fb0409c9640 1 --2- 172.21.15.160:0/834596745 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fb0280a1d80 0x7fb0280a4240 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:20.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.255+0000 7fb0409c9640 1 --2- 172.21.15.160:0/834596745 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb03c102130 0x7fb03c19ddc0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:20.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.255+0000 7fb0409c9640 1 --2- 172.21.15.160:0/834596745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb03c101770 0x7fb03c196d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:20.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.255+0000 7fb0409c9640 1 --2- 172.21.15.160:0/834596745 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb03c100580 0x7fb03c196810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:20.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.255+0000 7fb0409c9640 1 -- 172.21.15.160:0/834596745 >> 172.21.15.160:0/834596745 conn(0x7fb03c0fbb10 msgr2=0x7fb03c104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:20.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.255+0000 7fb0409c9640 1 -- 172.21.15.160:0/834596745 shutdown_connections 2024-03-07T07:39:20.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.255+0000 7fb0409c9640 1 -- 172.21.15.160:0/834596745 wait complete. 2024-03-07T07:39:20.267 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-03-07T07:39:20.303 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:19 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/3494546573' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-07T07:39:20.303 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:19 smithi177 ceph-mon[36578]: mds.? [v2:172.21.15.178:6836/1247829154,v1:172.21.15.178:6837/1247829154] up:active 2024-03-07T07:39:20.303 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:19 smithi177 ceph-mon[36578]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-03-07T07:39:20.304 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:19 smithi177 ceph-mon[36578]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-03-07T07:39:20.304 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:19 smithi177 ceph-mon[36578]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:39:20.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:19 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/3494546573' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-07T07:39:20.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:19 smithi178 ceph-mon[36650]: mds.? [v2:172.21.15.178:6836/1247829154,v1:172.21.15.178:6837/1247829154] up:active 2024-03-07T07:39:20.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:19 smithi178 ceph-mon[36650]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-03-07T07:39:20.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:19 smithi178 ceph-mon[36650]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-03-07T07:39:20.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:19 smithi178 ceph-mon[36650]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:39:20.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.375+0000 7f1e8ceb6640 1 -- 172.21.15.160:0/1499495917 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e881011d0 msgr2=0x7f1e88108a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:20.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.375+0000 7f1e8ceb6640 1 --2- 172.21.15.160:0/1499495917 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e881011d0 0x7f1e88108a70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1e80009ca0 tx=0x7f1e8002c0f0 comp rx=0 tx=0).stop 2024-03-07T07:39:20.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.375+0000 7f1e8ceb6640 1 -- 172.21.15.160:0/1499495917 shutdown_connections 2024-03-07T07:39:20.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.375+0000 7f1e8ceb6640 1 --2- 172.21.15.160:0/1499495917 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e881011d0 0x7f1e88108a70 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:20.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.375+0000 7f1e8ceb6640 1 --2- 172.21.15.160:0/1499495917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e88100810 0x7f1e88100c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:20.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.375+0000 7f1e8ceb6640 1 --2- 172.21.15.160:0/1499495917 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e88101bc0 0x7f1e88101fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:20.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.375+0000 7f1e8ceb6640 1 -- 172.21.15.160:0/1499495917 >> 172.21.15.160:0/1499495917 conn(0x7f1e880fb2a0 msgr2=0x7f1e880fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:20.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.375+0000 7f1e8ceb6640 1 -- 172.21.15.160:0/1499495917 shutdown_connections 2024-03-07T07:39:20.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.376+0000 7f1e8ceb6640 1 -- 172.21.15.160:0/1499495917 wait complete. 2024-03-07T07:39:20.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.376+0000 7f1e8ceb6640 1 Processor -- start 2024-03-07T07:39:20.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.376+0000 7f1e8ceb6640 1 -- start start 2024-03-07T07:39:20.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.376+0000 7f1e8ceb6640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e88100810 0x7f1e88100270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:20.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.376+0000 7f1e8ceb6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e881011d0 0x7f1e8819c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:20.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.376+0000 7f1e8ceb6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e88101bc0 0x7f1e8819ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:20.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.376+0000 7f1e8ceb6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8810c240 con 0x7f1e881011d0 2024-03-07T07:39:20.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.376+0000 7f1e8ceb6640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8810c0c0 con 0x7f1e88100810 2024-03-07T07:39:20.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.376+0000 7f1e8ceb6640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8810c3c0 con 0x7f1e88101bc0 2024-03-07T07:39:20.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.376+0000 7f1e85d9b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e881011d0 0x7f1e8819c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:20.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.377+0000 7f1e85d9b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e881011d0 0x7f1e8819c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:49674/0 (socket says 172.21.15.160:49674) 2024-03-07T07:39:20.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.377+0000 7f1e85d9b640 1 -- 172.21.15.160:0/2064354288 learned_addr learned my addr 172.21.15.160:0/2064354288 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:39:20.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.377+0000 7f1e8659c640 1 --2- 172.21.15.160:0/2064354288 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e88100810 0x7f1e88100270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:20.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.377+0000 7f1e86d9d640 1 --2- 172.21.15.160:0/2064354288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e88101bc0 0x7f1e8819ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:20.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.377+0000 7f1e85d9b640 1 -- 172.21.15.160:0/2064354288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e88101bc0 msgr2=0x7f1e8819ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:20.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.377+0000 7f1e85d9b640 1 --2- 172.21.15.160:0/2064354288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e88101bc0 0x7f1e8819ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:20.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.377+0000 7f1e85d9b640 1 -- 172.21.15.160:0/2064354288 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e88100810 msgr2=0x7f1e88100270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:20.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.377+0000 7f1e85d9b640 1 --2- 172.21.15.160:0/2064354288 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e88100810 0x7f1e88100270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:20.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.377+0000 7f1e85d9b640 1 -- 172.21.15.160:0/2064354288 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e8819f600 con 0x7f1e881011d0 2024-03-07T07:39:20.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.377+0000 7f1e85d9b640 1 --2- 172.21.15.160:0/2064354288 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e881011d0 0x7f1e8819c6d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c00e9c0 tx=0x7f1e7c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:20.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.377+0000 7f1e777fe640 1 -- 172.21.15.160:0/2064354288 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c00cd80 con 0x7f1e881011d0 2024-03-07T07:39:20.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.377+0000 7f1e777fe640 1 -- 172.21.15.160:0/2064354288 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1e7c0035f0 con 0x7f1e881011d0 2024-03-07T07:39:20.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.377+0000 7f1e8ceb6640 1 -- 172.21.15.160:0/2064354288 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e8819f890 con 0x7f1e881011d0 2024-03-07T07:39:20.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.377+0000 7f1e777fe640 1 -- 172.21.15.160:0/2064354288 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c010690 con 0x7f1e881011d0 2024-03-07T07:39:20.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.377+0000 7f1e8ceb6640 1 -- 172.21.15.160:0/2064354288 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e8819fd90 con 0x7f1e881011d0 2024-03-07T07:39:20.382 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.378+0000 7f1e8ceb6640 1 -- 172.21.15.160:0/2064354288 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e88107950 con 0x7f1e881011d0 2024-03-07T07:39:20.382 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.379+0000 7f1e777fe640 1 -- 172.21.15.160:0/2064354288 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f1e7c0037a0 con 0x7f1e881011d0 2024-03-07T07:39:20.382 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.380+0000 7f1e777fe640 1 --2- 172.21.15.160:0/2064354288 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f1e700a1d80 0x7f1e700a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:20.382 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.380+0000 7f1e8659c640 1 --2- 172.21.15.160:0/2064354288 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f1e700a1d80 0x7f1e700a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:20.382 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.380+0000 7f1e777fe640 1 -- 172.21.15.160:0/2064354288 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1e7c0cdfb0 con 0x7f1e881011d0 2024-03-07T07:39:20.382 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.380+0000 7f1e8659c640 1 --2- 172.21.15.160:0/2064354288 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f1e700a1d80 0x7f1e700a4240 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1e88069a90 tx=0x7f1e78005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:20.384 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.384+0000 7f1e777fe640 1 -- 172.21.15.160:0/2064354288 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1e7c09b530 con 0x7f1e881011d0 2024-03-07T07:39:20.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.586+0000 7f1e8ceb6640 1 -- 172.21.15.160:0/2064354288 --> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e88061b30 con 0x7f1e700a1d80 2024-03-07T07:39:20.589 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.589+0000 7f1e777fe640 1 -- 172.21.15.160:0/2064354288 <== mgr.14152 v2:172.21.15.160:6800/419630619 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f1e88061b30 con 0x7f1e700a1d80 2024-03-07T07:39:20.590 INFO:teuthology.orchestra.run.smithi160.stdout:/volumes/qa/sv_1/be5ac2bd-d227-43bd-bead-10930cde3bb5 2024-03-07T07:39:20.592 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.591+0000 7f1e8ceb6640 1 -- 172.21.15.160:0/2064354288 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f1e700a1d80 msgr2=0x7f1e700a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:20.592 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.591+0000 7f1e8ceb6640 1 --2- 172.21.15.160:0/2064354288 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f1e700a1d80 0x7f1e700a4240 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1e88069a90 tx=0x7f1e78005f90 comp rx=0 tx=0).stop 2024-03-07T07:39:20.592 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.591+0000 7f1e8ceb6640 1 -- 172.21.15.160:0/2064354288 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e881011d0 msgr2=0x7f1e8819c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:20.592 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.591+0000 7f1e8ceb6640 1 --2- 172.21.15.160:0/2064354288 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e881011d0 0x7f1e8819c6d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c00e9c0 tx=0x7f1e7c00ee80 comp rx=0 tx=0).stop 2024-03-07T07:39:20.592 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.591+0000 7f1e8ceb6640 1 -- 172.21.15.160:0/2064354288 shutdown_connections 2024-03-07T07:39:20.592 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.591+0000 7f1e8ceb6640 1 --2- 172.21.15.160:0/2064354288 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f1e700a1d80 0x7f1e700a4240 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:20.593 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.591+0000 7f1e8ceb6640 1 --2- 172.21.15.160:0/2064354288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e88101bc0 0x7f1e8819ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:20.593 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.591+0000 7f1e8ceb6640 1 --2- 172.21.15.160:0/2064354288 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e881011d0 0x7f1e8819c6d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:20.593 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.591+0000 7f1e8ceb6640 1 --2- 172.21.15.160:0/2064354288 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e88100810 0x7f1e88100270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:20.593 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.591+0000 7f1e8ceb6640 1 -- 172.21.15.160:0/2064354288 >> 172.21.15.160:0/2064354288 conn(0x7f1e880fb2a0 msgr2=0x7f1e88104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:20.593 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.591+0000 7f1e8ceb6640 1 -- 172.21.15.160:0/2064354288 shutdown_connections 2024-03-07T07:39:20.593 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.592+0000 7f1e8ceb6640 1 -- 172.21.15.160:0/2064354288 wait complete. 2024-03-07T07:39:20.603 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T07:39:20.707 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.704+0000 7efdf2989640 1 -- 172.21.15.160:0/2596150552 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdec101770 msgr2=0x7efdec101bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:20.707 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.704+0000 7efdf2989640 1 --2- 172.21.15.160:0/2596150552 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdec101770 0x7efdec101bf0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efde0009860 tx=0x7efde002bb70 comp rx=0 tx=0).stop 2024-03-07T07:39:20.707 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.704+0000 7efdf2989640 1 -- 172.21.15.160:0/2596150552 shutdown_connections 2024-03-07T07:39:20.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.704+0000 7efdf2989640 1 --2- 172.21.15.160:0/2596150552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efdec102130 0x7efdec1089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:20.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.704+0000 7efdf2989640 1 --2- 172.21.15.160:0/2596150552 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdec101770 0x7efdec101bf0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:20.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.704+0000 7efdf2989640 1 --2- 172.21.15.160:0/2596150552 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efdec100580 0x7efdec1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:20.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.704+0000 7efdf2989640 1 -- 172.21.15.160:0/2596150552 >> 172.21.15.160:0/2596150552 conn(0x7efdec0fbaf0 msgr2=0x7efdec0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:20.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.704+0000 7efdf2989640 1 -- 172.21.15.160:0/2596150552 shutdown_connections 2024-03-07T07:39:20.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.704+0000 7efdf2989640 1 -- 172.21.15.160:0/2596150552 wait complete. 2024-03-07T07:39:20.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.705+0000 7efdf2989640 1 Processor -- start 2024-03-07T07:39:20.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.705+0000 7efdf2989640 1 -- start start 2024-03-07T07:39:20.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.705+0000 7efdf2989640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efdec100580 0x7efdec196870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:20.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.705+0000 7efdf2989640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdec101770 0x7efdec196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:20.709 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.705+0000 7efdf2989640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efdec102130 0x7efdec19de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:20.709 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.705+0000 7efdf2989640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdec10c210 con 0x7efdec100580 2024-03-07T07:39:20.709 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.705+0000 7efdf2989640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdec10c090 con 0x7efdec101770 2024-03-07T07:39:20.709 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.705+0000 7efdf2989640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdec10c390 con 0x7efdec102130 2024-03-07T07:39:20.709 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.705+0000 7efdebfff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efdec100580 0x7efdec196870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:20.709 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.705+0000 7efdebfff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efdec100580 0x7efdec196870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:49680/0 (socket says 172.21.15.160:49680) 2024-03-07T07:39:20.709 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.705+0000 7efdebfff640 1 -- 172.21.15.160:0/1717047867 learned_addr learned my addr 172.21.15.160:0/1717047867 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:39:20.709 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.705+0000 7efdeb7fe640 1 --2- 172.21.15.160:0/1717047867 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdec101770 0x7efdec196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:20.709 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.705+0000 7efdf0f26640 1 --2- 172.21.15.160:0/1717047867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efdec102130 0x7efdec19de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:20.709 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.706+0000 7efdebfff640 1 -- 172.21.15.160:0/1717047867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efdec102130 msgr2=0x7efdec19de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:20.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.706+0000 7efdebfff640 1 --2- 172.21.15.160:0/1717047867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efdec102130 0x7efdec19de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:20.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.706+0000 7efdebfff640 1 -- 172.21.15.160:0/1717047867 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdec101770 msgr2=0x7efdec196db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:20.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.706+0000 7efdebfff640 1 --2- 172.21.15.160:0/1717047867 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdec101770 0x7efdec196db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:20.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.706+0000 7efdebfff640 1 -- 172.21.15.160:0/1717047867 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdec19e490 con 0x7efdec100580 2024-03-07T07:39:20.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.706+0000 7efdebfff640 1 --2- 172.21.15.160:0/1717047867 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efdec100580 0x7efdec196870 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efddc00e9a0 tx=0x7efddc00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:20.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.706+0000 7efde97fa640 1 -- 172.21.15.160:0/1717047867 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efddc00cd80 con 0x7efdec100580 2024-03-07T07:39:20.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.706+0000 7efdf2989640 1 -- 172.21.15.160:0/1717047867 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdec19e780 con 0x7efdec100580 2024-03-07T07:39:20.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.706+0000 7efde97fa640 1 -- 172.21.15.160:0/1717047867 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efddc010910 con 0x7efdec100580 2024-03-07T07:39:20.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.706+0000 7efde97fa640 1 -- 172.21.15.160:0/1717047867 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efddc018a40 con 0x7efdec100580 2024-03-07T07:39:20.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.706+0000 7efdf2989640 1 -- 172.21.15.160:0/1717047867 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdec19ecf0 con 0x7efdec100580 2024-03-07T07:39:20.710 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.707+0000 7efdf2989640 1 -- 172.21.15.160:0/1717047867 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdec1078a0 con 0x7efdec100580 2024-03-07T07:39:20.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.708+0000 7efde97fa640 1 -- 172.21.15.160:0/1717047867 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 136754+0+0 (secure 0 0 0) 0x7efddc010ac0 con 0x7efdec100580 2024-03-07T07:39:20.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.709+0000 7efde97fa640 1 --2- 172.21.15.160:0/1717047867 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7efdd40a1d80 0x7efdd40a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:20.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.709+0000 7efdeb7fe640 1 --2- 172.21.15.160:0/1717047867 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7efdd40a1d80 0x7efdd40a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:20.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.709+0000 7efde97fa640 1 -- 172.21.15.160:0/1717047867 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7efddc0ce590 con 0x7efdec100580 2024-03-07T07:39:20.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.709+0000 7efdeb7fe640 1 --2- 172.21.15.160:0/1717047867 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7efdd40a1d80 0x7efdd40a4240 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7efde002c180 tx=0x7efde0034ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:20.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.712+0000 7efde97fa640 1 -- 172.21.15.160:0/1717047867 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7efddc026020 con 0x7efdec100580 2024-03-07T07:39:20.806 INFO:journalctl@ceph.mds.i.smithi178.stdout:Mar 07 07:39:20 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-i[68618]: 2024-03-07T07:39:20.344+0000 7fc63d8f6700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-07T07:39:20.806 INFO:journalctl@ceph.mds.i.smithi178.stdout:Mar 07 07:39:20 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-i[68618]: 2024-03-07T07:39:20.344+0000 7fc63d8f6700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-07T07:39:20.806 INFO:journalctl@ceph.mds.i.smithi178.stdout:Mar 07 07:39:20 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-i[68618]: 2024-03-07T07:39:20.344+0000 7fc63d8f6700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-03-07T07:39:20.876 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.875+0000 7efde97fa640 1 -- 172.21.15.160:0/1717047867 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mgrmap(e 17) v1 ==== 136805+0+0 (secure 0 0 0) 0x7efddc0979e0 con 0x7efdec100580 2024-03-07T07:39:20.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.930+0000 7efdf2989640 1 -- 172.21.15.160:0/1717047867 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efdec02d030 con 0x7efdec100580 2024-03-07T07:39:20.931 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.930+0000 7efde97fa640 1 -- 172.21.15.160:0/1717047867 <== mon.0 v2:172.21.15.160:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+11411 (secure 0 0 0) 0x7efddc09ba70 con 0x7efdec100580 2024-03-07T07:39:20.931 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:39:20.931 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/2615961083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2615961083},{"type":"v1","addr":"172.21.15.160:6835","nonce":2615961083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1759006033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1759006033},{"type":"v1","addr":"172.21.15.160:6837","nonce":1759006033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6839/3763944721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":3763944721},{"type":"v1","addr":"172.21.15.160:6839","nonce":3763944721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6841/3611426637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6840","nonce":3611426637},{"type":"v1","addr":"172.21.15.160:6841","nonce":3611426637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1457364761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1457364761},{"type":"v1","addr":"172.21.15.177:6833","nonce":1457364761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/170428292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":170428292},{"type":"v1","addr":"172.21.15.177:6835","nonce":170428292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-07T07:39:14.164243+0000","modified":"2024-03-07T07:39:18.861903+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24511,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/645365846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":645365846},{"type":"v1","addr":"172.21.15.178:6833","nonce":645365846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6837/4088036516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":4088036516},{"type":"v1","addr":"172.21.15.177:6837","nonce":4088036516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/647840547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":647840547},{"type":"v1","addr":"172.21.15.178:6835","nonce":647840547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.178:6837/1247829154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1247829154},{"type":"v1","addr":"172.21.15.178:6837","nonce":1247829154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24497":{"gid":24497,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/3329773838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3329773838},{"type":"v1","addr":"172.21.15.177:6839","nonce":3329773838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24511":{"gid":24511,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.178:6839/2947354024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":2947354024},{"type":"v1","addr":"172.21.15.178:6839","nonce":2947354024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24511,"qdb_cluster":[24487,24511,24497]},"id":1}]} 2024-03-07T07:39:20.931 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 21 2024-03-07T07:39:20.933 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.932+0000 7efdf2989640 1 -- 172.21.15.160:0/1717047867 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7efdd40a1d80 msgr2=0x7efdd40a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:20.933 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.932+0000 7efdf2989640 1 --2- 172.21.15.160:0/1717047867 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7efdd40a1d80 0x7efdd40a4240 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7efde002c180 tx=0x7efde0034ad0 comp rx=0 tx=0).stop 2024-03-07T07:39:20.933 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.933+0000 7efdf2989640 1 -- 172.21.15.160:0/1717047867 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efdec100580 msgr2=0x7efdec196870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:20.933 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.933+0000 7efdf2989640 1 --2- 172.21.15.160:0/1717047867 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efdec100580 0x7efdec196870 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efddc00e9a0 tx=0x7efddc00ee60 comp rx=0 tx=0).stop 2024-03-07T07:39:20.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.933+0000 7efdf2989640 1 -- 172.21.15.160:0/1717047867 shutdown_connections 2024-03-07T07:39:20.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.933+0000 7efdf2989640 1 --2- 172.21.15.160:0/1717047867 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7efdd40a1d80 0x7efdd40a4240 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:20.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.933+0000 7efdf2989640 1 --2- 172.21.15.160:0/1717047867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efdec102130 0x7efdec19de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:20.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.933+0000 7efdf2989640 1 --2- 172.21.15.160:0/1717047867 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdec101770 0x7efdec196db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:20.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.933+0000 7efdf2989640 1 --2- 172.21.15.160:0/1717047867 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efdec100580 0x7efdec196870 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:20.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.933+0000 7efdf2989640 1 -- 172.21.15.160:0/1717047867 >> 172.21.15.160:0/1717047867 conn(0x7efdec0fbaf0 msgr2=0x7efdec104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:20.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.933+0000 7efdf2989640 1 -- 172.21.15.160:0/1717047867 shutdown_connections 2024-03-07T07:39:20.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:20.933+0000 7efdf2989640 1 -- 172.21.15.160:0/1717047867 wait complete. 2024-03-07T07:39:20.944 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T07:39:21.049 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.046+0000 7f1c61ed4640 1 -- 172.21.15.160:0/900339204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c5c102110 msgr2=0x7f1c5c104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:21.049 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.046+0000 7f1c61ed4640 1 --2- 172.21.15.160:0/900339204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c5c102110 0x7f1c5c104500 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1c50009860 tx=0x7f1c5002bb70 comp rx=0 tx=0).stop 2024-03-07T07:39:21.049 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.046+0000 7f1c61ed4640 1 -- 172.21.15.160:0/900339204 shutdown_connections 2024-03-07T07:39:21.049 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.046+0000 7f1c61ed4640 1 --2- 172.21.15.160:0/900339204 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c5c107370 0x7f1c5c1097c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:21.049 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.046+0000 7f1c61ed4640 1 --2- 172.21.15.160:0/900339204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c5c104a40 0x7f1c5c106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:21.049 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.046+0000 7f1c61ed4640 1 --2- 172.21.15.160:0/900339204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c5c102110 0x7f1c5c104500 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:21.050 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.046+0000 7f1c61ed4640 1 -- 172.21.15.160:0/900339204 >> 172.21.15.160:0/900339204 conn(0x7f1c5c0fbb40 msgr2=0x7f1c5c0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:21.050 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.046+0000 7f1c61ed4640 1 -- 172.21.15.160:0/900339204 shutdown_connections 2024-03-07T07:39:21.050 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.046+0000 7f1c61ed4640 1 -- 172.21.15.160:0/900339204 wait complete. 2024-03-07T07:39:21.050 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.046+0000 7f1c61ed4640 1 Processor -- start 2024-03-07T07:39:21.050 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.047+0000 7f1c61ed4640 1 -- start start 2024-03-07T07:39:21.050 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.047+0000 7f1c61ed4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c5c102110 0x7f1c5c1979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:21.050 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.047+0000 7f1c61ed4640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c5c104a40 0x7f1c5c197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:21.050 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.047+0000 7f1c61ed4640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c5c107370 0x7f1c5c19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:21.050 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.047+0000 7f1c61ed4640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c10d310 con 0x7f1c5c104a40 2024-03-07T07:39:21.050 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.047+0000 7f1c61ed4640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c10d190 con 0x7f1c5c107370 2024-03-07T07:39:21.050 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.047+0000 7f1c61ed4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c10d490 con 0x7f1c5c102110 2024-03-07T07:39:21.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.047+0000 7f1c5affd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c5c104a40 0x7f1c5c197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:21.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.047+0000 7f1c5affd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c5c104a40 0x7f1c5c197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:49706/0 (socket says 172.21.15.160:49706) 2024-03-07T07:39:21.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.047+0000 7f1c5affd640 1 -- 172.21.15.160:0/24870448 learned_addr learned my addr 172.21.15.160:0/24870448 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:39:21.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.047+0000 7f1c5b7fe640 1 --2- 172.21.15.160:0/24870448 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c5c102110 0x7f1c5c1979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:21.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.047+0000 7f1c5bfff640 1 --2- 172.21.15.160:0/24870448 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c5c107370 0x7f1c5c19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:21.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.047+0000 7f1c5affd640 1 -- 172.21.15.160:0/24870448 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c5c102110 msgr2=0x7f1c5c1979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:21.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.047+0000 7f1c5affd640 1 --2- 172.21.15.160:0/24870448 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c5c102110 0x7f1c5c1979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:21.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.047+0000 7f1c5affd640 1 -- 172.21.15.160:0/24870448 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c5c107370 msgr2=0x7f1c5c19ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:21.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.047+0000 7f1c5affd640 1 --2- 172.21.15.160:0/24870448 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c5c107370 0x7f1c5c19ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:21.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.047+0000 7f1c5affd640 1 -- 172.21.15.160:0/24870448 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c5c19f5d0 con 0x7f1c5c104a40 2024-03-07T07:39:21.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.047+0000 7f1c5affd640 1 --2- 172.21.15.160:0/24870448 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c5c104a40 0x7f1c5c197ef0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c00e9f0 tx=0x7f1c4c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:21.052 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.047+0000 7f1c58ff9640 1 -- 172.21.15.160:0/24870448 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c00ccc0 con 0x7f1c5c104a40 2024-03-07T07:39:21.052 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.047+0000 7f1c58ff9640 1 -- 172.21.15.160:0/24870448 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1c4c010910 con 0x7f1c5c104a40 2024-03-07T07:39:21.052 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.048+0000 7f1c61ed4640 1 -- 172.21.15.160:0/24870448 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c5c19f8c0 con 0x7f1c5c104a40 2024-03-07T07:39:21.052 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.048+0000 7f1c58ff9640 1 -- 172.21.15.160:0/24870448 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c0188c0 con 0x7f1c5c104a40 2024-03-07T07:39:21.052 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.048+0000 7f1c61ed4640 1 -- 172.21.15.160:0/24870448 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c5c19fe10 con 0x7f1c5c104a40 2024-03-07T07:39:21.052 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.048+0000 7f1c61ed4640 1 -- 172.21.15.160:0/24870448 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c5c0fd630 con 0x7f1c5c104a40 2024-03-07T07:39:21.053 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.049+0000 7f1c58ff9640 1 -- 172.21.15.160:0/24870448 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) v1 ==== 136805+0+0 (secure 0 0 0) 0x7f1c4c010ac0 con 0x7f1c5c104a40 2024-03-07T07:39:21.053 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.050+0000 7f1c58ff9640 1 --2- 172.21.15.160:0/24870448 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f1c440a1e10 0x7f1c440a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:21.053 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.050+0000 7f1c5b7fe640 1 --2- 172.21.15.160:0/24870448 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f1c440a1e10 0x7f1c440a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:21.053 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.050+0000 7f1c58ff9640 1 -- 172.21.15.160:0/24870448 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1c4c0ce060 con 0x7f1c5c104a40 2024-03-07T07:39:21.053 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.050+0000 7f1c5b7fe640 1 --2- 172.21.15.160:0/24870448 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f1c440a1e10 0x7f1c440a42d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1c5002c180 tx=0x7f1c50034ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:21.055 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.054+0000 7f1c58ff9640 1 -- 172.21.15.160:0/24870448 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1c4c09b4e0 con 0x7f1c5c104a40 2024-03-07T07:39:21.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:20 smithi160 ceph-mon[31963]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:39:21.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:20 smithi160 ceph-mon[31963]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:39:21.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:20 smithi160 ceph-mon[31963]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:39:21.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:20 smithi160 ceph-mon[31963]: pgmap v291: 97 pgs: 97 active+clean; 601 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 9.6 KiB/s wr, 23 op/s 2024-03-07T07:39:21.231 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.231+0000 7f1c61ed4640 1 -- 172.21.15.160:0/24870448 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c5c02d280 con 0x7f1c5c104a40 2024-03-07T07:39:21.232 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.232+0000 7f1c58ff9640 1 -- 172.21.15.160:0/24870448 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22242 (secure 0 0 0) 0x7f1c4c09ade0 con 0x7f1c5c104a40 2024-03-07T07:39:21.233 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:39:21.233 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":85,"fsid":"826f6582-dc54-11ee-95c6-87774f69a715","created":"2024-03-07T07:31:14.229288+0000","modified":"2024-03-07T07:39:15.600463+0000","last_up_change":"2024-03-07T07:38:20.776225+0000","last_in_change":"2024-03-07T07:38:00.508917+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T07:34:34.529424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T07:39:12.075432+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T07:39:13.367820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bee66c67-fb6e-4717-9685-ca62587d5f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6803","nonce":1368482926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6805","nonce":1368482926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6809","nonce":1368482926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6807","nonce":1368482926}]},"public_addr":"172.21.15.160:6803/1368482926","cluster_addr":"172.21.15.160:6805/1368482926","heartbeat_back_addr":"172.21.15.160:6809/1368482926","heartbeat_front_addr":"172.21.15.160:6807/1368482926","state":["exists","up"]},{"osd":1,"uuid":"49236251-f750-424e-992b-bddf55762556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6801","nonce":2922391884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6803","nonce":2922391884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6807","nonce":2922391884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6805","nonce":2922391884}]},"public_addr":"172.21.15.177:6801/2922391884","cluster_addr":"172.21.15.177:6803/2922391884","heartbeat_back_addr":"172.21.15.177:6807/2922391884","heartbeat_front_addr":"172.21.15.177:6805/2922391884","state":["exists","up"]},{"osd":2,"uuid":"7a432b28-f7ee-43fe-9196-7aaeccfecf77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6801","nonce":1217901551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6803","nonce":1217901551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6807","nonce":1217901551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6805","nonce":1217901551}]},"public_addr":"172.21.15.178:6801/1217901551","cluster_addr":"172.21.15.178:6803/1217901551","heartbeat_back_addr":"172.21.15.178:6807/1217901551","heartbeat_front_addr":"172.21.15.178:6805/1217901551","state":["exists","up"]},{"osd":3,"uuid":"4ad22a20-f1be-4002-a48d-ecd3953e63f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6811","nonce":42543989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6813","nonce":42543989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6817","nonce":42543989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6815","nonce":42543989}]},"public_addr":"172.21.15.160:6811/42543989","cluster_addr":"172.21.15.160:6813/42543989","heartbeat_back_addr":"172.21.15.160:6817/42543989","heartbeat_front_addr":"172.21.15.160:6815/42543989","state":["exists","up"]},{"osd":4,"uuid":"2b65b104-ac82-4ce3-b6c5-11a399103045","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6809","nonce":88178553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6811","nonce":88178553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6815","nonce":88178553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6813","nonce":88178553}]},"public_addr":"172.21.15.177:6809/88178553","cluster_addr":"172.21.15.177:6811/88178553","heartbeat_back_addr":"172.21.15.177:6815/88178553","heartbeat_front_addr":"172.21.15.177:6813/88178553","state":["exists","up"]},{"osd":5,"uuid":"30a1e714-08ff-44e8-9264-9ef164604b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6809","nonce":964732891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6811","nonce":964732891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6815","nonce":964732891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6813","nonce":964732891}]},"public_addr":"172.21.15.178:6809/964732891","cluster_addr":"172.21.15.178:6811/964732891","heartbeat_back_addr":"172.21.15.178:6815/964732891","heartbeat_front_addr":"172.21.15.178:6813/964732891","state":["exists","up"]},{"osd":6,"uuid":"b1e1646e-f8a9-42b8-b62d-9d7b5d6a1ff4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6819","nonce":3251327001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6821","nonce":3251327001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6825","nonce":3251327001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6823","nonce":3251327001}]},"public_addr":"172.21.15.160:6819/3251327001","cluster_addr":"172.21.15.160:6821/3251327001","heartbeat_back_addr":"172.21.15.160:6825/3251327001","heartbeat_front_addr":"172.21.15.160:6823/3251327001","state":["exists","up"]},{"osd":7,"uuid":"e66b400b-9747-4f6d-b05b-db22468e2750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6817","nonce":1175500850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6819","nonce":1175500850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6823","nonce":1175500850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6821","nonce":1175500850}]},"public_addr":"172.21.15.177:6817/1175500850","cluster_addr":"172.21.15.177:6819/1175500850","heartbeat_back_addr":"172.21.15.177:6823/1175500850","heartbeat_front_addr":"172.21.15.177:6821/1175500850","state":["exists","up"]},{"osd":8,"uuid":"3ee0f2c4-fca4-49fc-a98b-07aa0fcba0e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6817","nonce":688015531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6819","nonce":688015531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6823","nonce":688015531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6821","nonce":688015531}]},"public_addr":"172.21.15.178:6817/688015531","cluster_addr":"172.21.15.178:6819/688015531","heartbeat_back_addr":"172.21.15.178:6823/688015531","heartbeat_front_addr":"172.21.15.178:6821/688015531","state":["exists","up"]},{"osd":9,"uuid":"856039d7-f7ee-4dbd-b019-0073e705ce0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6827","nonce":3265797851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6829","nonce":3265797851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6833","nonce":3265797851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6831","nonce":3265797851}]},"public_addr":"172.21.15.160:6827/3265797851","cluster_addr":"172.21.15.160:6829/3265797851","heartbeat_back_addr":"172.21.15.160:6833/3265797851","heartbeat_front_addr":"172.21.15.160:6831/3265797851","state":["exists","up"]},{"osd":10,"uuid":"bdc9899d-d720-4429-ad53-4c388a4dbe75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6825","nonce":2867473412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6827","nonce":2867473412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6831","nonce":2867473412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6829","nonce":2867473412}]},"public_addr":"172.21.15.177:6825/2867473412","cluster_addr":"172.21.15.177:6827/2867473412","heartbeat_back_addr":"172.21.15.177:6831/2867473412","heartbeat_front_addr":"172.21.15.177:6829/2867473412","state":["exists","up"]},{"osd":11,"uuid":"cc9adb54-ef49-47c6-90e3-fc03bc0f5cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6825","nonce":4068724731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6827","nonce":4068724731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6831","nonce":4068724731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6829","nonce":4068724731}]},"public_addr":"172.21.15.178:6825/4068724731","cluster_addr":"172.21.15.178:6827/4068724731","heartbeat_back_addr":"172.21.15.178:6831/4068724731","heartbeat_front_addr":"172.21.15.178:6829/4068724731","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:33:44.141585+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:07.685585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:31.044386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:56.308925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:35:21.582097+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:35:45.983540+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:36:11.530242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:36:36.844681+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:02.303437+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:27.629867+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:53.289495+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:38:18.536169+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/3027450612":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/874709722":"2024-03-08T07:32:08.190837+0000","172.21.15.160:6800/1130675225":"2024-03-08T07:31:41.166858+0000","172.21.15.160:0/3857193980":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/3861968297":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6800/3859174315":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/3013534743":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6801/3859174315":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/1310937667":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6801/1130675225":"2024-03-08T07:31:41.166858+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T07:39:21.235 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.234+0000 7f1c61ed4640 1 -- 172.21.15.160:0/24870448 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f1c440a1e10 msgr2=0x7f1c440a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:21.235 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.234+0000 7f1c61ed4640 1 --2- 172.21.15.160:0/24870448 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f1c440a1e10 0x7f1c440a42d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1c5002c180 tx=0x7f1c50034ad0 comp rx=0 tx=0).stop 2024-03-07T07:39:21.235 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.234+0000 7f1c61ed4640 1 -- 172.21.15.160:0/24870448 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c5c104a40 msgr2=0x7f1c5c197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:21.235 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.234+0000 7f1c61ed4640 1 --2- 172.21.15.160:0/24870448 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c5c104a40 0x7f1c5c197ef0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c00e9f0 tx=0x7f1c4c00edf0 comp rx=0 tx=0).stop 2024-03-07T07:39:21.235 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.234+0000 7f1c61ed4640 1 -- 172.21.15.160:0/24870448 shutdown_connections 2024-03-07T07:39:21.235 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.234+0000 7f1c61ed4640 1 --2- 172.21.15.160:0/24870448 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f1c440a1e10 0x7f1c440a42d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:21.235 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.234+0000 7f1c61ed4640 1 --2- 172.21.15.160:0/24870448 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c5c107370 0x7f1c5c19ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:21.235 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.234+0000 7f1c61ed4640 1 --2- 172.21.15.160:0/24870448 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c5c104a40 0x7f1c5c197ef0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:21.236 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.234+0000 7f1c61ed4640 1 --2- 172.21.15.160:0/24870448 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c5c102110 0x7f1c5c1979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:21.236 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.234+0000 7f1c61ed4640 1 -- 172.21.15.160:0/24870448 >> 172.21.15.160:0/24870448 conn(0x7f1c5c0fbb40 msgr2=0x7f1c5c102a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:21.236 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.235+0000 7f1c61ed4640 1 -- 172.21.15.160:0/24870448 shutdown_connections 2024-03-07T07:39:21.236 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.235+0000 7f1c61ed4640 1 -- 172.21.15.160:0/24870448 wait complete. 2024-03-07T07:39:21.247 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-07T07:39:21.247 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-07T07:39:21.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:20 smithi178 ceph-mon[36650]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:39:21.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:20 smithi178 ceph-mon[36650]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:39:21.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:20 smithi178 ceph-mon[36650]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:39:21.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:20 smithi178 ceph-mon[36650]: pgmap v291: 97 pgs: 97 active+clean; 601 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 9.6 KiB/s wr, 23 op/s 2024-03-07T07:39:21.351 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.349+0000 7f5338d81640 1 -- 172.21.15.160:0/1294924646 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5334101770 msgr2=0x7f5334101bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:21.352 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.349+0000 7f5338d81640 1 --2- 172.21.15.160:0/1294924646 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5334101770 0x7f5334101bf0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f531c009860 tx=0x7f531c02bb70 comp rx=0 tx=0).stop 2024-03-07T07:39:21.352 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.349+0000 7f5338d81640 1 -- 172.21.15.160:0/1294924646 shutdown_connections 2024-03-07T07:39:21.352 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.349+0000 7f5338d81640 1 --2- 172.21.15.160:0/1294924646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5334102130 0x7f53341089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:21.352 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.349+0000 7f5338d81640 1 --2- 172.21.15.160:0/1294924646 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5334101770 0x7f5334101bf0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:21.352 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.349+0000 7f5338d81640 1 --2- 172.21.15.160:0/1294924646 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5334100580 0x7f53341009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:21.352 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.349+0000 7f5338d81640 1 -- 172.21.15.160:0/1294924646 >> 172.21.15.160:0/1294924646 conn(0x7f53340fbaf0 msgr2=0x7f53340fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:21.352 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.349+0000 7f5338d81640 1 -- 172.21.15.160:0/1294924646 shutdown_connections 2024-03-07T07:39:21.352 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.349+0000 7f5338d81640 1 -- 172.21.15.160:0/1294924646 wait complete. 2024-03-07T07:39:21.352 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.350+0000 7f5338d81640 1 Processor -- start 2024-03-07T07:39:21.352 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.350+0000 7f5338d81640 1 -- start start 2024-03-07T07:39:21.352 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.350+0000 7f5338d81640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5334100580 0x7f5334196890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:21.353 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.350+0000 7f5338d81640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5334101770 0x7f5334196dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:21.353 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.350+0000 7f5338d81640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5334102130 0x7f533419de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:21.353 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.350+0000 7f5338d81640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533410c210 con 0x7f5334102130 2024-03-07T07:39:21.353 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.350+0000 7f5338d81640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533410c090 con 0x7f5334101770 2024-03-07T07:39:21.353 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.350+0000 7f5338d81640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533410c390 con 0x7f5334100580 2024-03-07T07:39:21.353 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.350+0000 7f5332d9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5334102130 0x7f533419de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:21.353 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.350+0000 7f5332d9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5334102130 0x7f533419de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:49722/0 (socket says 172.21.15.160:49722) 2024-03-07T07:39:21.353 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.350+0000 7f533259c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5334100580 0x7f5334196890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:21.353 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.350+0000 7f5332d9d640 1 -- 172.21.15.160:0/3808742525 learned_addr learned my addr 172.21.15.160:0/3808742525 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:39:21.353 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.350+0000 7f5329d9b640 1 --2- 172.21.15.160:0/3808742525 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5334101770 0x7f5334196dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:21.353 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.350+0000 7f5332d9d640 1 -- 172.21.15.160:0/3808742525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5334100580 msgr2=0x7f5334196890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:21.354 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.350+0000 7f5332d9d640 1 --2- 172.21.15.160:0/3808742525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5334100580 0x7f5334196890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:21.354 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.350+0000 7f5332d9d640 1 -- 172.21.15.160:0/3808742525 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5334101770 msgr2=0x7f5334196dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:21.354 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.350+0000 7f5332d9d640 1 --2- 172.21.15.160:0/3808742525 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5334101770 0x7f5334196dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:21.354 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.350+0000 7f5332d9d640 1 -- 172.21.15.160:0/3808742525 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f533419e4b0 con 0x7f5334102130 2024-03-07T07:39:21.354 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.350+0000 7f5329d9b640 1 --2- 172.21.15.160:0/3808742525 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5334101770 0x7f5334196dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:39:21.354 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.350+0000 7f533259c640 1 --2- 172.21.15.160:0/3808742525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5334100580 0x7f5334196890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:39:21.354 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.351+0000 7f5332d9d640 1 --2- 172.21.15.160:0/3808742525 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5334102130 0x7f533419de40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f532c00eca0 tx=0x7f532c00b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:21.354 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.351+0000 7f532bfff640 1 -- 172.21.15.160:0/3808742525 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532c004b20 con 0x7f5334102130 2024-03-07T07:39:21.354 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.351+0000 7f5338d81640 1 -- 172.21.15.160:0/3808742525 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5334068820 con 0x7f5334102130 2024-03-07T07:39:21.354 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.351+0000 7f532bfff640 1 -- 172.21.15.160:0/3808742525 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f532c004cc0 con 0x7f5334102130 2024-03-07T07:39:21.354 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.351+0000 7f532bfff640 1 -- 172.21.15.160:0/3808742525 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532c017670 con 0x7f5334102130 2024-03-07T07:39:21.354 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.351+0000 7f5338d81640 1 -- 172.21.15.160:0/3808742525 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5334068d90 con 0x7f5334102130 2024-03-07T07:39:21.355 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.352+0000 7f5338d81640 1 -- 172.21.15.160:0/3808742525 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5334068450 con 0x7f5334102130 2024-03-07T07:39:21.355 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.353+0000 7f532bfff640 1 -- 172.21.15.160:0/3808742525 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) v1 ==== 136805+0+0 (secure 0 0 0) 0x7f532c005020 con 0x7f5334102130 2024-03-07T07:39:21.355 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.353+0000 7f532bfff640 1 --2- 172.21.15.160:0/3808742525 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f53200a1e10 0x7f53200a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:21.356 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.353+0000 7f533259c640 1 --2- 172.21.15.160:0/3808742525 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f53200a1e10 0x7f53200a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:21.356 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.353+0000 7f532bfff640 1 -- 172.21.15.160:0/3808742525 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f532c0cf7d0 con 0x7f5334102130 2024-03-07T07:39:21.356 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.354+0000 7f533259c640 1 --2- 172.21.15.160:0/3808742525 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f53200a1e10 0x7f53200a42d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f53240096c0 tx=0x7f5324009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:21.356 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:20 smithi177 ceph-mon[36578]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:39:21.356 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:20 smithi177 ceph-mon[36578]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:39:21.356 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:20 smithi177 ceph-mon[36578]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:39:21.356 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:20 smithi177 ceph-mon[36578]: pgmap v291: 97 pgs: 97 active+clean; 601 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 9.6 KiB/s wr, 23 op/s 2024-03-07T07:39:21.358 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.358+0000 7f532bfff640 1 -- 172.21.15.160:0/3808742525 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f532c09c600 con 0x7f5334102130 2024-03-07T07:39:21.530 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.529+0000 7f5338d81640 1 -- 172.21.15.160:0/3808742525 --> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5334068ff0 con 0x7f53200a1e10 2024-03-07T07:39:21.537 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.537+0000 7f532bfff640 1 -- 172.21.15.160:0/3808742525 <== mgr.14152 v2:172.21.15.160:6800/419630619 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+258939 (secure 0 0 0) 0x7f5334068ff0 con 0x7f53200a1e10 2024-03-07T07:39:21.540 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:39:21.542 INFO:teuthology.orchestra.run.smithi160.stderr:dumped all 2024-03-07T07:39:21.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.540+0000 7f5338d81640 1 -- 172.21.15.160:0/3808742525 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f53200a1e10 msgr2=0x7f53200a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:21.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.540+0000 7f5338d81640 1 --2- 172.21.15.160:0/3808742525 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f53200a1e10 0x7f53200a42d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f53240096c0 tx=0x7f5324009260 comp rx=0 tx=0).stop 2024-03-07T07:39:21.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.540+0000 7f5338d81640 1 -- 172.21.15.160:0/3808742525 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5334102130 msgr2=0x7f533419de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:21.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.540+0000 7f5338d81640 1 --2- 172.21.15.160:0/3808742525 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5334102130 0x7f533419de40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f532c00eca0 tx=0x7f532c00b690 comp rx=0 tx=0).stop 2024-03-07T07:39:21.544 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.540+0000 7f5338d81640 1 -- 172.21.15.160:0/3808742525 shutdown_connections 2024-03-07T07:39:21.544 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.540+0000 7f5338d81640 1 --2- 172.21.15.160:0/3808742525 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f53200a1e10 0x7f53200a42d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:21.544 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.540+0000 7f5338d81640 1 --2- 172.21.15.160:0/3808742525 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5334102130 0x7f533419de40 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:21.544 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.540+0000 7f5338d81640 1 --2- 172.21.15.160:0/3808742525 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5334101770 0x7f5334196dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:21.544 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.540+0000 7f5338d81640 1 --2- 172.21.15.160:0/3808742525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5334100580 0x7f5334196890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:21.544 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.540+0000 7f5338d81640 1 -- 172.21.15.160:0/3808742525 >> 172.21.15.160:0/3808742525 conn(0x7f53340fbaf0 msgr2=0x7f5334104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:21.544 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.540+0000 7f5338d81640 1 -- 172.21.15.160:0/3808742525 shutdown_connections 2024-03-07T07:39:21.544 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.540+0000 7f5338d81640 1 -- 172.21.15.160:0/3808742525 wait complete. 2024-03-07T07:39:21.552 INFO:teuthology.orchestra.run.smithi160.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2024-03-07T07:39:20.271777+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":615381,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":144,"num_read_kb":124,"num_write":313,"num_write_kb":3000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":253,"ondisk_log_size":253,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":259,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360224,"kb_used_data":11032,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581600,"statfs":{"total":1151940427776,"available":1151571558400,"internally_reserved":0,"allocated":11296768,"data_stored":7029715,"data_compressed":107820,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25013,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":132,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.455863"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.604658+0000","last_change":"2024-03-07T07:39:14.614168+0000","last_active":"2024-03-07T07:39:15.604658+0000","last_peered":"2024-03-07T07:39:15.604658+0000","last_clean":"2024-03-07T07:39:15.604658+0000","last_became_active":"2024-03-07T07:39:14.613708+0000","last_became_peered":"2024-03-07T07:39:14.613708+0000","last_unstale":"2024-03-07T07:39:15.604658+0000","last_undegraded":"2024-03-07T07:39:15.604658+0000","last_fullsized":"2024-03-07T07:39:15.604658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:08:07.286470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.125350+0000","last_change":"2024-03-07T07:39:14.609425+0000","last_active":"2024-03-07T07:39:16.125350+0000","last_peered":"2024-03-07T07:39:16.125350+0000","last_clean":"2024-03-07T07:39:16.125350+0000","last_became_active":"2024-03-07T07:39:14.609060+0000","last_became_peered":"2024-03-07T07:39:14.609060+0000","last_unstale":"2024-03-07T07:39:16.125350+0000","last_undegraded":"2024-03-07T07:39:16.125350+0000","last_fullsized":"2024-03-07T07:39:16.125350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:37:33.773358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.125303+0000","last_change":"2024-03-07T07:39:14.609985+0000","last_active":"2024-03-07T07:39:16.125303+0000","last_peered":"2024-03-07T07:39:16.125303+0000","last_clean":"2024-03-07T07:39:16.125303+0000","last_became_active":"2024-03-07T07:39:14.609490+0000","last_became_peered":"2024-03-07T07:39:14.609490+0000","last_unstale":"2024-03-07T07:39:16.125303+0000","last_undegraded":"2024-03-07T07:39:16.125303+0000","last_fullsized":"2024-03-07T07:39:16.125303+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T19:09:24.134718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.126341+0000","last_change":"2024-03-07T07:39:14.620482+0000","last_active":"2024-03-07T07:39:16.126341+0000","last_peered":"2024-03-07T07:39:16.126341+0000","last_clean":"2024-03-07T07:39:16.126341+0000","last_became_active":"2024-03-07T07:39:14.620100+0000","last_became_peered":"2024-03-07T07:39:14.620100+0000","last_unstale":"2024-03-07T07:39:16.126341+0000","last_undegraded":"2024-03-07T07:39:16.126341+0000","last_fullsized":"2024-03-07T07:39:16.126341+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:03:20.291833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.838708+0000","last_change":"2024-03-07T07:39:14.603228+0000","last_active":"2024-03-07T07:39:15.838708+0000","last_peered":"2024-03-07T07:39:15.838708+0000","last_clean":"2024-03-07T07:39:15.838708+0000","last_became_active":"2024-03-07T07:39:14.602874+0000","last_became_peered":"2024-03-07T07:39:14.602874+0000","last_unstale":"2024-03-07T07:39:15.838708+0000","last_undegraded":"2024-03-07T07:39:15.838708+0000","last_fullsized":"2024-03-07T07:39:15.838708+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:26:17.762271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.856483+0000","last_change":"2024-03-07T07:39:14.611556+0000","last_active":"2024-03-07T07:39:15.856483+0000","last_peered":"2024-03-07T07:39:15.856483+0000","last_clean":"2024-03-07T07:39:15.856483+0000","last_became_active":"2024-03-07T07:39:14.609938+0000","last_became_peered":"2024-03-07T07:39:14.609938+0000","last_unstale":"2024-03-07T07:39:15.856483+0000","last_undegraded":"2024-03-07T07:39:15.856483+0000","last_fullsized":"2024-03-07T07:39:15.856483+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:51:49.013178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.125796+0000","last_change":"2024-03-07T07:39:14.615259+0000","last_active":"2024-03-07T07:39:16.125796+0000","last_peered":"2024-03-07T07:39:16.125796+0000","last_clean":"2024-03-07T07:39:16.125796+0000","last_became_active":"2024-03-07T07:39:14.614878+0000","last_became_peered":"2024-03-07T07:39:14.614878+0000","last_unstale":"2024-03-07T07:39:16.125796+0000","last_undegraded":"2024-03-07T07:39:16.125796+0000","last_fullsized":"2024-03-07T07:39:16.125796+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:22:28.905360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.838241+0000","last_change":"2024-03-07T07:39:14.614518+0000","last_active":"2024-03-07T07:39:15.838241+0000","last_peered":"2024-03-07T07:39:15.838241+0000","last_clean":"2024-03-07T07:39:15.838241+0000","last_became_active":"2024-03-07T07:39:14.614308+0000","last_became_peered":"2024-03-07T07:39:14.614308+0000","last_unstale":"2024-03-07T07:39:15.838241+0000","last_undegraded":"2024-03-07T07:39:15.838241+0000","last_fullsized":"2024-03-07T07:39:15.838241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:54:54.908066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-07T07:39:14.617630+0000","last_change":"2024-03-07T07:39:14.615360+0000","last_active":"2024-03-07T07:39:14.617630+0000","last_peered":"2024-03-07T07:39:14.617630+0000","last_clean":"2024-03-07T07:39:14.617630+0000","last_became_active":"2024-03-07T07:39:14.615036+0000","last_became_peered":"2024-03-07T07:39:14.615036+0000","last_unstale":"2024-03-07T07:39:14.617630+0000","last_undegraded":"2024-03-07T07:39:14.617630+0000","last_fullsized":"2024-03-07T07:39:14.617630+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:24:36.633406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.856814+0000","last_change":"2024-03-07T07:39:14.610450+0000","last_active":"2024-03-07T07:39:15.856814+0000","last_peered":"2024-03-07T07:39:15.856814+0000","last_clean":"2024-03-07T07:39:15.856814+0000","last_became_active":"2024-03-07T07:39:14.609483+0000","last_became_peered":"2024-03-07T07:39:14.609483+0000","last_unstale":"2024-03-07T07:39:15.856814+0000","last_undegraded":"2024-03-07T07:39:15.856814+0000","last_fullsized":"2024-03-07T07:39:15.856814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:37:59.323371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.126110+0000","last_change":"2024-03-07T07:39:14.611800+0000","last_active":"2024-03-07T07:39:16.126110+0000","last_peered":"2024-03-07T07:39:16.126110+0000","last_clean":"2024-03-07T07:39:16.126110+0000","last_became_active":"2024-03-07T07:39:14.611482+0000","last_became_peered":"2024-03-07T07:39:14.611482+0000","last_unstale":"2024-03-07T07:39:16.126110+0000","last_undegraded":"2024-03-07T07:39:16.126110+0000","last_fullsized":"2024-03-07T07:39:16.126110+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:26:36.361716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.126171+0000","last_change":"2024-03-07T07:39:14.621686+0000","last_active":"2024-03-07T07:39:16.126171+0000","last_peered":"2024-03-07T07:39:16.126171+0000","last_clean":"2024-03-07T07:39:16.126171+0000","last_became_active":"2024-03-07T07:39:14.620815+0000","last_became_peered":"2024-03-07T07:39:14.620815+0000","last_unstale":"2024-03-07T07:39:16.126171+0000","last_undegraded":"2024-03-07T07:39:16.126171+0000","last_fullsized":"2024-03-07T07:39:16.126171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:33:59.972215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.603717+0000","last_change":"2024-03-07T07:39:14.606045+0000","last_active":"2024-03-07T07:39:15.603717+0000","last_peered":"2024-03-07T07:39:15.603717+0000","last_clean":"2024-03-07T07:39:15.603717+0000","last_became_active":"2024-03-07T07:39:14.605726+0000","last_became_peered":"2024-03-07T07:39:14.605726+0000","last_unstale":"2024-03-07T07:39:15.603717+0000","last_undegraded":"2024-03-07T07:39:15.603717+0000","last_fullsized":"2024-03-07T07:39:15.603717+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:13:51.899507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.125192+0000","last_change":"2024-03-07T07:39:14.605435+0000","last_active":"2024-03-07T07:39:16.125192+0000","last_peered":"2024-03-07T07:39:16.125192+0000","last_clean":"2024-03-07T07:39:16.125192+0000","last_became_active":"2024-03-07T07:39:14.605127+0000","last_became_peered":"2024-03-07T07:39:14.605127+0000","last_unstale":"2024-03-07T07:39:16.125192+0000","last_undegraded":"2024-03-07T07:39:16.125192+0000","last_fullsized":"2024-03-07T07:39:16.125192+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:15:58.206921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.125256+0000","last_change":"2024-03-07T07:39:14.617166+0000","last_active":"2024-03-07T07:39:16.125256+0000","last_peered":"2024-03-07T07:39:16.125256+0000","last_clean":"2024-03-07T07:39:16.125256+0000","last_became_active":"2024-03-07T07:39:14.616615+0000","last_became_peered":"2024-03-07T07:39:14.616615+0000","last_unstale":"2024-03-07T07:39:16.125256+0000","last_undegraded":"2024-03-07T07:39:16.125256+0000","last_fullsized":"2024-03-07T07:39:16.125256+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:25:05.086822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-07T07:39:14.613395+0000","last_change":"2024-03-07T07:39:14.611990+0000","last_active":"2024-03-07T07:39:14.613395+0000","last_peered":"2024-03-07T07:39:14.613395+0000","last_clean":"2024-03-07T07:39:14.613395+0000","last_became_active":"2024-03-07T07:39:14.610728+0000","last_became_peered":"2024-03-07T07:39:14.610728+0000","last_unstale":"2024-03-07T07:39:14.613395+0000","last_undegraded":"2024-03-07T07:39:14.613395+0000","last_fullsized":"2024-03-07T07:39:14.613395+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T19:21:42.252823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.604704+0000","last_change":"2024-03-07T07:39:14.615409+0000","last_active":"2024-03-07T07:39:15.604704+0000","last_peered":"2024-03-07T07:39:15.604704+0000","last_clean":"2024-03-07T07:39:15.604704+0000","last_became_active":"2024-03-07T07:39:14.615003+0000","last_became_peered":"2024-03-07T07:39:14.615003+0000","last_unstale":"2024-03-07T07:39:15.604704+0000","last_undegraded":"2024-03-07T07:39:15.604704+0000","last_fullsized":"2024-03-07T07:39:15.604704+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:33:56.464718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.836417+0000","last_change":"2024-03-07T07:39:14.610501+0000","last_active":"2024-03-07T07:39:15.836417+0000","last_peered":"2024-03-07T07:39:15.836417+0000","last_clean":"2024-03-07T07:39:15.836417+0000","last_became_active":"2024-03-07T07:39:14.610190+0000","last_became_peered":"2024-03-07T07:39:14.610190+0000","last_unstale":"2024-03-07T07:39:15.836417+0000","last_undegraded":"2024-03-07T07:39:15.836417+0000","last_fullsized":"2024-03-07T07:39:15.836417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:05:31.360270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.125986+0000","last_change":"2024-03-07T07:39:14.609373+0000","last_active":"2024-03-07T07:39:16.125986+0000","last_peered":"2024-03-07T07:39:16.125986+0000","last_clean":"2024-03-07T07:39:16.125986+0000","last_became_active":"2024-03-07T07:39:14.608174+0000","last_became_peered":"2024-03-07T07:39:14.608174+0000","last_unstale":"2024-03-07T07:39:16.125986+0000","last_undegraded":"2024-03-07T07:39:16.125986+0000","last_fullsized":"2024-03-07T07:39:16.125986+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:25:18.704239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:17.861977+0000","last_change":"2024-03-07T07:39:13.830499+0000","last_active":"2024-03-07T07:39:17.861977+0000","last_peered":"2024-03-07T07:39:17.861977+0000","last_clean":"2024-03-07T07:39:17.861977+0000","last_became_active":"2024-03-07T07:39:13.830131+0000","last_became_peered":"2024-03-07T07:39:13.830131+0000","last_unstale":"2024-03-07T07:39:17.861977+0000","last_undegraded":"2024-03-07T07:39:17.861977+0000","last_fullsized":"2024-03-07T07:39:17.861977+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:48:14.338758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.838489+0000","last_change":"2024-03-07T07:39:14.612715+0000","last_active":"2024-03-07T07:39:15.838489+0000","last_peered":"2024-03-07T07:39:15.838489+0000","last_clean":"2024-03-07T07:39:15.838489+0000","last_became_active":"2024-03-07T07:39:14.612383+0000","last_became_peered":"2024-03-07T07:39:14.612383+0000","last_unstale":"2024-03-07T07:39:15.838489+0000","last_undegraded":"2024-03-07T07:39:15.838489+0000","last_fullsized":"2024-03-07T07:39:15.838489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:17:36.800396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.838702+0000","last_change":"2024-03-07T07:39:13.830901+0000","last_active":"2024-03-07T07:39:15.838702+0000","last_peered":"2024-03-07T07:39:15.838702+0000","last_clean":"2024-03-07T07:39:15.838702+0000","last_became_active":"2024-03-07T07:39:13.830581+0000","last_became_peered":"2024-03-07T07:39:13.830581+0000","last_unstale":"2024-03-07T07:39:15.838702+0000","last_undegraded":"2024-03-07T07:39:15.838702+0000","last_fullsized":"2024-03-07T07:39:15.838702+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:25:15.428381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.125398+0000","last_change":"2024-03-07T07:39:14.610876+0000","last_active":"2024-03-07T07:39:16.125398+0000","last_peered":"2024-03-07T07:39:16.125398+0000","last_clean":"2024-03-07T07:39:16.125398+0000","last_became_active":"2024-03-07T07:39:14.610584+0000","last_became_peered":"2024-03-07T07:39:14.610584+0000","last_unstale":"2024-03-07T07:39:16.125398+0000","last_undegraded":"2024-03-07T07:39:16.125398+0000","last_fullsized":"2024-03-07T07:39:16.125398+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:55:11.818462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:17.861707+0000","last_change":"2024-03-07T07:39:13.838602+0000","last_active":"2024-03-07T07:39:17.861707+0000","last_peered":"2024-03-07T07:39:17.861707+0000","last_clean":"2024-03-07T07:39:17.861707+0000","last_became_active":"2024-03-07T07:39:13.838061+0000","last_became_peered":"2024-03-07T07:39:13.838061+0000","last_unstale":"2024-03-07T07:39:17.861707+0000","last_undegraded":"2024-03-07T07:39:17.861707+0000","last_fullsized":"2024-03-07T07:39:17.861707+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:42:33.589280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-07T07:39:14.613125+0000","last_change":"2024-03-07T07:39:14.607666+0000","last_active":"2024-03-07T07:39:14.613125+0000","last_peered":"2024-03-07T07:39:14.613125+0000","last_clean":"2024-03-07T07:39:14.613125+0000","last_became_active":"2024-03-07T07:39:14.607339+0000","last_became_peered":"2024-03-07T07:39:14.607339+0000","last_unstale":"2024-03-07T07:39:14.613125+0000","last_undegraded":"2024-03-07T07:39:14.613125+0000","last_fullsized":"2024-03-07T07:39:14.613125+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:57:46.319015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.839017+0000","last_change":"2024-03-07T07:39:13.829179+0000","last_active":"2024-03-07T07:39:15.839017+0000","last_peered":"2024-03-07T07:39:15.839017+0000","last_clean":"2024-03-07T07:39:15.839017+0000","last_became_active":"2024-03-07T07:39:13.828162+0000","last_became_peered":"2024-03-07T07:39:13.828162+0000","last_unstale":"2024-03-07T07:39:15.839017+0000","last_undegraded":"2024-03-07T07:39:15.839017+0000","last_fullsized":"2024-03-07T07:39:15.839017+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:38:00.330980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.125759+0000","last_change":"2024-03-07T07:39:14.621496+0000","last_active":"2024-03-07T07:39:16.125759+0000","last_peered":"2024-03-07T07:39:16.125759+0000","last_clean":"2024-03-07T07:39:16.125759+0000","last_became_active":"2024-03-07T07:39:14.621139+0000","last_became_peered":"2024-03-07T07:39:14.621139+0000","last_unstale":"2024-03-07T07:39:16.125759+0000","last_undegraded":"2024-03-07T07:39:16.125759+0000","last_fullsized":"2024-03-07T07:39:16.125759+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:24:45.553536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:18.869990+0000","last_change":"2024-03-07T07:39:13.834999+0000","last_active":"2024-03-07T07:39:18.869990+0000","last_peered":"2024-03-07T07:39:18.869990+0000","last_clean":"2024-03-07T07:39:18.869990+0000","last_became_active":"2024-03-07T07:39:13.834672+0000","last_became_peered":"2024-03-07T07:39:13.834672+0000","last_unstale":"2024-03-07T07:39:18.869990+0000","last_undegraded":"2024-03-07T07:39:18.869990+0000","last_fullsized":"2024-03-07T07:39:18.869990+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:55:57.944992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.838954+0000","last_change":"2024-03-07T07:39:14.605519+0000","last_active":"2024-03-07T07:39:15.838954+0000","last_peered":"2024-03-07T07:39:15.838954+0000","last_clean":"2024-03-07T07:39:15.838954+0000","last_became_active":"2024-03-07T07:39:14.605185+0000","last_became_peered":"2024-03-07T07:39:14.605185+0000","last_unstale":"2024-03-07T07:39:15.838954+0000","last_undegraded":"2024-03-07T07:39:15.838954+0000","last_fullsized":"2024-03-07T07:39:15.838954+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:54:33.622751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:17.861391+0000","last_change":"2024-03-07T07:39:13.837186+0000","last_active":"2024-03-07T07:39:17.861391+0000","last_peered":"2024-03-07T07:39:17.861391+0000","last_clean":"2024-03-07T07:39:17.861391+0000","last_became_active":"2024-03-07T07:39:13.836807+0000","last_became_peered":"2024-03-07T07:39:13.836807+0000","last_unstale":"2024-03-07T07:39:17.861391+0000","last_undegraded":"2024-03-07T07:39:17.861391+0000","last_fullsized":"2024-03-07T07:39:17.861391+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:32:39.574197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.125881+0000","last_change":"2024-03-07T07:39:14.621503+0000","last_active":"2024-03-07T07:39:16.125881+0000","last_peered":"2024-03-07T07:39:16.125881+0000","last_clean":"2024-03-07T07:39:16.125881+0000","last_became_active":"2024-03-07T07:39:14.621146+0000","last_became_peered":"2024-03-07T07:39:14.621146+0000","last_unstale":"2024-03-07T07:39:16.125881+0000","last_undegraded":"2024-03-07T07:39:16.125881+0000","last_fullsized":"2024-03-07T07:39:16.125881+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:50:15.270355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:17.862334+0000","last_change":"2024-03-07T07:39:13.838146+0000","last_active":"2024-03-07T07:39:17.862334+0000","last_peered":"2024-03-07T07:39:17.862334+0000","last_clean":"2024-03-07T07:39:17.862334+0000","last_became_active":"2024-03-07T07:39:13.837788+0000","last_became_peered":"2024-03-07T07:39:13.837788+0000","last_unstale":"2024-03-07T07:39:17.862334+0000","last_undegraded":"2024-03-07T07:39:17.862334+0000","last_fullsized":"2024-03-07T07:39:17.862334+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:50:52.154146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.838105+0000","last_change":"2024-03-07T07:39:14.605069+0000","last_active":"2024-03-07T07:39:15.838105+0000","last_peered":"2024-03-07T07:39:15.838105+0000","last_clean":"2024-03-07T07:39:15.838105+0000","last_became_active":"2024-03-07T07:39:14.604733+0000","last_became_peered":"2024-03-07T07:39:14.604733+0000","last_unstale":"2024-03-07T07:39:15.838105+0000","last_undegraded":"2024-03-07T07:39:15.838105+0000","last_fullsized":"2024-03-07T07:39:15.838105+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:49:08.312385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.125163+0000","last_change":"2024-03-07T07:39:13.837854+0000","last_active":"2024-03-07T07:39:16.125163+0000","last_peered":"2024-03-07T07:39:16.125163+0000","last_clean":"2024-03-07T07:39:16.125163+0000","last_became_active":"2024-03-07T07:39:13.837428+0000","last_became_peered":"2024-03-07T07:39:13.837428+0000","last_unstale":"2024-03-07T07:39:16.125163+0000","last_undegraded":"2024-03-07T07:39:16.125163+0000","last_fullsized":"2024-03-07T07:39:16.125163+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:35:15.369936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.836296+0000","last_change":"2024-03-07T07:39:14.606299+0000","last_active":"2024-03-07T07:39:15.836296+0000","last_peered":"2024-03-07T07:39:15.836296+0000","last_clean":"2024-03-07T07:39:15.836296+0000","last_became_active":"2024-03-07T07:39:14.605983+0000","last_became_peered":"2024-03-07T07:39:14.605983+0000","last_unstale":"2024-03-07T07:39:15.836296+0000","last_undegraded":"2024-03-07T07:39:15.836296+0000","last_fullsized":"2024-03-07T07:39:15.836296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:53:16.658862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.859391+0000","last_change":"2024-03-07T07:39:13.840129+0000","last_active":"2024-03-07T07:39:16.859391+0000","last_peered":"2024-03-07T07:39:16.859391+0000","last_clean":"2024-03-07T07:39:16.859391+0000","last_became_active":"2024-03-07T07:39:13.839735+0000","last_became_peered":"2024-03-07T07:39:13.839735+0000","last_unstale":"2024-03-07T07:39:16.859391+0000","last_undegraded":"2024-03-07T07:39:16.859391+0000","last_fullsized":"2024-03-07T07:39:16.859391+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:31:38.711810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-07T07:39:14.613145+0000","last_change":"2024-03-07T07:39:14.606449+0000","last_active":"2024-03-07T07:39:14.613145+0000","last_peered":"2024-03-07T07:39:14.613145+0000","last_clean":"2024-03-07T07:39:14.613145+0000","last_became_active":"2024-03-07T07:39:14.606009+0000","last_became_peered":"2024-03-07T07:39:14.606009+0000","last_unstale":"2024-03-07T07:39:14.613145+0000","last_undegraded":"2024-03-07T07:39:14.613145+0000","last_fullsized":"2024-03-07T07:39:14.613145+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:33:31.383436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.857460+0000","last_change":"2024-03-07T07:39:13.834097+0000","last_active":"2024-03-07T07:39:16.857460+0000","last_peered":"2024-03-07T07:39:16.857460+0000","last_clean":"2024-03-07T07:39:16.857460+0000","last_became_active":"2024-03-07T07:39:13.833749+0000","last_became_peered":"2024-03-07T07:39:13.833749+0000","last_unstale":"2024-03-07T07:39:16.857460+0000","last_undegraded":"2024-03-07T07:39:16.857460+0000","last_fullsized":"2024-03-07T07:39:16.857460+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:12:22.471596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.125477+0000","last_change":"2024-03-07T07:39:14.621000+0000","last_active":"2024-03-07T07:39:16.125477+0000","last_peered":"2024-03-07T07:39:16.125477+0000","last_clean":"2024-03-07T07:39:16.125477+0000","last_became_active":"2024-03-07T07:39:14.620708+0000","last_became_peered":"2024-03-07T07:39:14.620708+0000","last_unstale":"2024-03-07T07:39:16.125477+0000","last_undegraded":"2024-03-07T07:39:16.125477+0000","last_fullsized":"2024-03-07T07:39:16.125477+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:26:48.363406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.858243+0000","last_change":"2024-03-07T07:39:13.837442+0000","last_active":"2024-03-07T07:39:16.858243+0000","last_peered":"2024-03-07T07:39:16.858243+0000","last_clean":"2024-03-07T07:39:16.858243+0000","last_became_active":"2024-03-07T07:39:13.836515+0000","last_became_peered":"2024-03-07T07:39:13.836515+0000","last_unstale":"2024-03-07T07:39:16.858243+0000","last_undegraded":"2024-03-07T07:39:16.858243+0000","last_fullsized":"2024-03-07T07:39:16.858243+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:19:08.312359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-07T07:39:14.620098+0000","last_change":"2024-03-07T07:39:14.619130+0000","last_active":"2024-03-07T07:39:14.620098+0000","last_peered":"2024-03-07T07:39:14.620098+0000","last_clean":"2024-03-07T07:39:14.620098+0000","last_became_active":"2024-03-07T07:39:14.618814+0000","last_became_peered":"2024-03-07T07:39:14.618814+0000","last_unstale":"2024-03-07T07:39:14.620098+0000","last_undegraded":"2024-03-07T07:39:14.620098+0000","last_fullsized":"2024-03-07T07:39:14.620098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:56:06.691897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:19.554520+0000","last_change":"2024-03-07T07:39:13.841051+0000","last_active":"2024-03-07T07:39:19.554520+0000","last_peered":"2024-03-07T07:39:19.554520+0000","last_clean":"2024-03-07T07:39:19.554520+0000","last_became_active":"2024-03-07T07:39:13.840719+0000","last_became_peered":"2024-03-07T07:39:13.840719+0000","last_unstale":"2024-03-07T07:39:19.554520+0000","last_undegraded":"2024-03-07T07:39:19.554520+0000","last_fullsized":"2024-03-07T07:39:19.554520+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T19:33:57.327673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20895,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":16,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.839154+0000","last_change":"2024-03-07T07:39:14.606539+0000","last_active":"2024-03-07T07:39:15.839154+0000","last_peered":"2024-03-07T07:39:15.839154+0000","last_clean":"2024-03-07T07:39:15.839154+0000","last_became_active":"2024-03-07T07:39:14.605832+0000","last_became_peered":"2024-03-07T07:39:14.605832+0000","last_unstale":"2024-03-07T07:39:15.839154+0000","last_undegraded":"2024-03-07T07:39:15.839154+0000","last_fullsized":"2024-03-07T07:39:15.839154+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:08:10.781906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:17.862546+0000","last_change":"2024-03-07T07:39:13.841526+0000","last_active":"2024-03-07T07:39:17.862546+0000","last_peered":"2024-03-07T07:39:17.862546+0000","last_clean":"2024-03-07T07:39:17.862546+0000","last_became_active":"2024-03-07T07:39:13.841153+0000","last_became_peered":"2024-03-07T07:39:13.841153+0000","last_unstale":"2024-03-07T07:39:17.862546+0000","last_undegraded":"2024-03-07T07:39:17.862546+0000","last_fullsized":"2024-03-07T07:39:17.862546+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T19:38:41.264033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.603494+0000","last_change":"2024-03-07T07:39:14.603948+0000","last_active":"2024-03-07T07:39:15.603494+0000","last_peered":"2024-03-07T07:39:15.603494+0000","last_clean":"2024-03-07T07:39:15.603494+0000","last_became_active":"2024-03-07T07:39:14.603724+0000","last_became_peered":"2024-03-07T07:39:14.603724+0000","last_unstale":"2024-03-07T07:39:15.603494+0000","last_undegraded":"2024-03-07T07:39:15.603494+0000","last_fullsized":"2024-03-07T07:39:15.603494+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T19:29:59.296537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.857443+0000","last_change":"2024-03-07T07:39:13.831965+0000","last_active":"2024-03-07T07:39:16.857443+0000","last_peered":"2024-03-07T07:39:16.857443+0000","last_clean":"2024-03-07T07:39:16.857443+0000","last_became_active":"2024-03-07T07:39:13.831617+0000","last_became_peered":"2024-03-07T07:39:13.831617+0000","last_unstale":"2024-03-07T07:39:16.857443+0000","last_undegraded":"2024-03-07T07:39:16.857443+0000","last_fullsized":"2024-03-07T07:39:16.857443+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:46:49.060697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.126415+0000","last_change":"2024-03-07T07:39:14.616265+0000","last_active":"2024-03-07T07:39:16.126415+0000","last_peered":"2024-03-07T07:39:16.126415+0000","last_clean":"2024-03-07T07:39:16.126415+0000","last_became_active":"2024-03-07T07:39:14.615206+0000","last_became_peered":"2024-03-07T07:39:14.615206+0000","last_unstale":"2024-03-07T07:39:16.126415+0000","last_undegraded":"2024-03-07T07:39:16.126415+0000","last_fullsized":"2024-03-07T07:39:16.126415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:39:21.997327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'4","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:18.873027+0000","last_change":"2024-03-07T07:39:13.833311+0000","last_active":"2024-03-07T07:39:18.873027+0000","last_peered":"2024-03-07T07:39:18.873027+0000","last_clean":"2024-03-07T07:39:18.873027+0000","last_became_active":"2024-03-07T07:39:13.832863+0000","last_became_peered":"2024-03-07T07:39:13.832863+0000","last_unstale":"2024-03-07T07:39:18.873027+0000","last_undegraded":"2024-03-07T07:39:18.873027+0000","last_fullsized":"2024-03-07T07:39:18.873027+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:43:50.988905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.125661+0000","last_change":"2024-03-07T07:39:14.615740+0000","last_active":"2024-03-07T07:39:16.125661+0000","last_peered":"2024-03-07T07:39:16.125661+0000","last_clean":"2024-03-07T07:39:16.125661+0000","last_became_active":"2024-03-07T07:39:14.615394+0000","last_became_peered":"2024-03-07T07:39:14.615394+0000","last_unstale":"2024-03-07T07:39:16.125661+0000","last_undegraded":"2024-03-07T07:39:16.125661+0000","last_fullsized":"2024-03-07T07:39:16.125661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:17:19.107374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:17.863654+0000","last_change":"2024-03-07T07:39:13.838231+0000","last_active":"2024-03-07T07:39:17.863654+0000","last_peered":"2024-03-07T07:39:17.863654+0000","last_clean":"2024-03-07T07:39:17.863654+0000","last_became_active":"2024-03-07T07:39:13.837895+0000","last_became_peered":"2024-03-07T07:39:13.837895+0000","last_unstale":"2024-03-07T07:39:17.863654+0000","last_undegraded":"2024-03-07T07:39:17.863654+0000","last_fullsized":"2024-03-07T07:39:17.863654+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:32:48.381613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.125232+0000","last_change":"2024-03-07T07:39:14.617134+0000","last_active":"2024-03-07T07:39:16.125232+0000","last_peered":"2024-03-07T07:39:16.125232+0000","last_clean":"2024-03-07T07:39:16.125232+0000","last_became_active":"2024-03-07T07:39:14.616813+0000","last_became_peered":"2024-03-07T07:39:14.616813+0000","last_unstale":"2024-03-07T07:39:16.125232+0000","last_undegraded":"2024-03-07T07:39:16.125232+0000","last_fullsized":"2024-03-07T07:39:16.125232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:12:18.477309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.839078+0000","last_change":"2024-03-07T07:39:13.828783+0000","last_active":"2024-03-07T07:39:15.839078+0000","last_peered":"2024-03-07T07:39:15.839078+0000","last_clean":"2024-03-07T07:39:15.839078+0000","last_became_active":"2024-03-07T07:39:13.828427+0000","last_became_peered":"2024-03-07T07:39:13.828427+0000","last_unstale":"2024-03-07T07:39:15.839078+0000","last_undegraded":"2024-03-07T07:39:15.839078+0000","last_fullsized":"2024-03-07T07:39:15.839078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:49:21.074014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.604553+0000","last_change":"2024-03-07T07:39:14.612765+0000","last_active":"2024-03-07T07:39:15.604553+0000","last_peered":"2024-03-07T07:39:15.604553+0000","last_clean":"2024-03-07T07:39:15.604553+0000","last_became_active":"2024-03-07T07:39:14.612381+0000","last_became_peered":"2024-03-07T07:39:14.612381+0000","last_unstale":"2024-03-07T07:39:15.604553+0000","last_undegraded":"2024-03-07T07:39:15.604553+0000","last_fullsized":"2024-03-07T07:39:15.604553+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:57:13.912594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.836495+0000","last_change":"2024-03-07T07:39:13.833342+0000","last_active":"2024-03-07T07:39:15.836495+0000","last_peered":"2024-03-07T07:39:15.836495+0000","last_clean":"2024-03-07T07:39:15.836495+0000","last_became_active":"2024-03-07T07:39:13.833013+0000","last_became_peered":"2024-03-07T07:39:13.833013+0000","last_unstale":"2024-03-07T07:39:15.836495+0000","last_undegraded":"2024-03-07T07:39:15.836495+0000","last_fullsized":"2024-03-07T07:39:15.836495+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:39:59.821206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.856881+0000","last_change":"2024-03-07T07:39:14.604744+0000","last_active":"2024-03-07T07:39:15.856881+0000","last_peered":"2024-03-07T07:39:15.856881+0000","last_clean":"2024-03-07T07:39:15.856881+0000","last_became_active":"2024-03-07T07:39:14.604364+0000","last_became_peered":"2024-03-07T07:39:14.604364+0000","last_unstale":"2024-03-07T07:39:15.856881+0000","last_undegraded":"2024-03-07T07:39:15.856881+0000","last_fullsized":"2024-03-07T07:39:15.856881+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:38:57.975024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.126211+0000","last_change":"2024-03-07T07:39:13.836434+0000","last_active":"2024-03-07T07:39:16.126211+0000","last_peered":"2024-03-07T07:39:16.126211+0000","last_clean":"2024-03-07T07:39:16.126211+0000","last_became_active":"2024-03-07T07:39:13.835953+0000","last_became_peered":"2024-03-07T07:39:13.835953+0000","last_unstale":"2024-03-07T07:39:16.126211+0000","last_undegraded":"2024-03-07T07:39:16.126211+0000","last_fullsized":"2024-03-07T07:39:16.126211+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:23:20.941845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.856762+0000","last_change":"2024-03-07T07:39:14.618062+0000","last_active":"2024-03-07T07:39:15.856762+0000","last_peered":"2024-03-07T07:39:15.856762+0000","last_clean":"2024-03-07T07:39:15.856762+0000","last_became_active":"2024-03-07T07:39:14.617715+0000","last_became_peered":"2024-03-07T07:39:14.617715+0000","last_unstale":"2024-03-07T07:39:15.856762+0000","last_undegraded":"2024-03-07T07:39:15.856762+0000","last_fullsized":"2024-03-07T07:39:15.856762+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:51:29.200771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.125393+0000","last_change":"2024-03-07T07:39:14.617522+0000","last_active":"2024-03-07T07:39:16.125393+0000","last_peered":"2024-03-07T07:39:16.125393+0000","last_clean":"2024-03-07T07:39:16.125393+0000","last_became_active":"2024-03-07T07:39:14.616862+0000","last_became_peered":"2024-03-07T07:39:14.616862+0000","last_unstale":"2024-03-07T07:39:16.125393+0000","last_undegraded":"2024-03-07T07:39:16.125393+0000","last_fullsized":"2024-03-07T07:39:16.125393+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:37:58.473662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:17.863277+0000","last_change":"2024-03-07T07:39:13.835090+0000","last_active":"2024-03-07T07:39:17.863277+0000","last_peered":"2024-03-07T07:39:17.863277+0000","last_clean":"2024-03-07T07:39:17.863277+0000","last_became_active":"2024-03-07T07:39:13.834765+0000","last_became_peered":"2024-03-07T07:39:13.834765+0000","last_unstale":"2024-03-07T07:39:17.863277+0000","last_undegraded":"2024-03-07T07:39:17.863277+0000","last_fullsized":"2024-03-07T07:39:17.863277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:50:41.530959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.838765+0000","last_change":"2024-03-07T07:39:14.615224+0000","last_active":"2024-03-07T07:39:15.838765+0000","last_peered":"2024-03-07T07:39:15.838765+0000","last_clean":"2024-03-07T07:39:15.838765+0000","last_became_active":"2024-03-07T07:39:14.614942+0000","last_became_peered":"2024-03-07T07:39:14.614942+0000","last_unstale":"2024-03-07T07:39:15.838765+0000","last_undegraded":"2024-03-07T07:39:15.838765+0000","last_fullsized":"2024-03-07T07:39:15.838765+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:30:48.321950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:18.868397+0000","last_change":"2024-03-07T07:39:13.833245+0000","last_active":"2024-03-07T07:39:18.868397+0000","last_peered":"2024-03-07T07:39:18.868397+0000","last_clean":"2024-03-07T07:39:18.868397+0000","last_became_active":"2024-03-07T07:39:13.832837+0000","last_became_peered":"2024-03-07T07:39:13.832837+0000","last_unstale":"2024-03-07T07:39:18.868397+0000","last_undegraded":"2024-03-07T07:39:18.868397+0000","last_fullsized":"2024-03-07T07:39:18.868397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:47:28.204394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":339,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.125613+0000","last_change":"2024-03-07T07:37:58.585313+0000","last_active":"2024-03-07T07:39:16.125613+0000","last_peered":"2024-03-07T07:39:16.125613+0000","last_clean":"2024-03-07T07:39:16.125613+0000","last_became_active":"2024-03-07T07:37:58.584808+0000","last_became_peered":"2024-03-07T07:37:58.584808+0000","last_unstale":"2024-03-07T07:39:16.125613+0000","last_undegraded":"2024-03-07T07:39:16.125613+0000","last_fullsized":"2024-03-07T07:39:16.125613+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:34:34.612837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:34:34.612837+0000","last_clean_scrub_stamp":"2024-03-07T07:34:34.612837+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:56:56.998601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.603377+0000","last_change":"2024-03-07T07:39:14.615907+0000","last_active":"2024-03-07T07:39:15.603377+0000","last_peered":"2024-03-07T07:39:15.603377+0000","last_clean":"2024-03-07T07:39:15.603377+0000","last_became_active":"2024-03-07T07:39:14.615587+0000","last_became_peered":"2024-03-07T07:39:14.615587+0000","last_unstale":"2024-03-07T07:39:15.603377+0000","last_undegraded":"2024-03-07T07:39:15.603377+0000","last_fullsized":"2024-03-07T07:39:15.603377+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:08:20.363895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:17.861503+0000","last_change":"2024-03-07T07:39:13.832143+0000","last_active":"2024-03-07T07:39:17.861503+0000","last_peered":"2024-03-07T07:39:17.861503+0000","last_clean":"2024-03-07T07:39:17.861503+0000","last_became_active":"2024-03-07T07:39:13.831299+0000","last_became_peered":"2024-03-07T07:39:13.831299+0000","last_unstale":"2024-03-07T07:39:17.861503+0000","last_undegraded":"2024-03-07T07:39:17.861503+0000","last_fullsized":"2024-03-07T07:39:17.861503+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:25:08.965498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.839180+0000","last_change":"2024-03-07T07:39:13.827617+0000","last_active":"2024-03-07T07:39:15.839180+0000","last_peered":"2024-03-07T07:39:15.839180+0000","last_clean":"2024-03-07T07:39:15.839180+0000","last_became_active":"2024-03-07T07:39:13.827188+0000","last_became_peered":"2024-03-07T07:39:13.827188+0000","last_unstale":"2024-03-07T07:39:15.839180+0000","last_undegraded":"2024-03-07T07:39:15.839180+0000","last_fullsized":"2024-03-07T07:39:15.839180+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:35:27.634460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.603768+0000","last_change":"2024-03-07T07:39:14.608307+0000","last_active":"2024-03-07T07:39:15.603768+0000","last_peered":"2024-03-07T07:39:15.603768+0000","last_clean":"2024-03-07T07:39:15.603768+0000","last_became_active":"2024-03-07T07:39:14.607948+0000","last_became_peered":"2024-03-07T07:39:14.607948+0000","last_unstale":"2024-03-07T07:39:15.603768+0000","last_undegraded":"2024-03-07T07:39:15.603768+0000","last_fullsized":"2024-03-07T07:39:15.603768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:36:27.294925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:17.861560+0000","last_change":"2024-03-07T07:39:13.831123+0000","last_active":"2024-03-07T07:39:17.861560+0000","last_peered":"2024-03-07T07:39:17.861560+0000","last_clean":"2024-03-07T07:39:17.861560+0000","last_became_active":"2024-03-07T07:39:13.830801+0000","last_became_peered":"2024-03-07T07:39:13.830801+0000","last_unstale":"2024-03-07T07:39:17.861560+0000","last_undegraded":"2024-03-07T07:39:17.861560+0000","last_fullsized":"2024-03-07T07:39:17.861560+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:19:54.346449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.838544+0000","last_change":"2024-03-07T07:39:14.610510+0000","last_active":"2024-03-07T07:39:15.838544+0000","last_peered":"2024-03-07T07:39:15.838544+0000","last_clean":"2024-03-07T07:39:15.838544+0000","last_became_active":"2024-03-07T07:39:14.609908+0000","last_became_peered":"2024-03-07T07:39:14.609908+0000","last_unstale":"2024-03-07T07:39:15.838544+0000","last_undegraded":"2024-03-07T07:39:15.838544+0000","last_fullsized":"2024-03-07T07:39:15.838544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:51:32.045296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.857044+0000","last_change":"2024-03-07T07:39:13.838605+0000","last_active":"2024-03-07T07:39:16.857044+0000","last_peered":"2024-03-07T07:39:16.857044+0000","last_clean":"2024-03-07T07:39:16.857044+0000","last_became_active":"2024-03-07T07:39:13.838362+0000","last_became_peered":"2024-03-07T07:39:13.838362+0000","last_unstale":"2024-03-07T07:39:16.857044+0000","last_undegraded":"2024-03-07T07:39:16.857044+0000","last_fullsized":"2024-03-07T07:39:16.857044+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:23:47.039496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.125389+0000","last_change":"2024-03-07T07:39:14.615902+0000","last_active":"2024-03-07T07:39:16.125389+0000","last_peered":"2024-03-07T07:39:16.125389+0000","last_clean":"2024-03-07T07:39:16.125389+0000","last_became_active":"2024-03-07T07:39:14.615578+0000","last_became_peered":"2024-03-07T07:39:14.615578+0000","last_unstale":"2024-03-07T07:39:16.125389+0000","last_undegraded":"2024-03-07T07:39:16.125389+0000","last_fullsized":"2024-03-07T07:39:16.125389+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:30:34.371625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-07T07:39:14.597264+0000","last_change":"2024-03-07T07:39:13.834640+0000","last_active":"2024-03-07T07:39:14.597264+0000","last_peered":"2024-03-07T07:39:14.597264+0000","last_clean":"2024-03-07T07:39:14.597264+0000","last_became_active":"2024-03-07T07:39:13.834247+0000","last_became_peered":"2024-03-07T07:39:13.834247+0000","last_unstale":"2024-03-07T07:39:14.597264+0000","last_undegraded":"2024-03-07T07:39:14.597264+0000","last_fullsized":"2024-03-07T07:39:14.597264+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:40:55.487824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.838687+0000","last_change":"2024-03-07T07:39:14.603762+0000","last_active":"2024-03-07T07:39:15.838687+0000","last_peered":"2024-03-07T07:39:15.838687+0000","last_clean":"2024-03-07T07:39:15.838687+0000","last_became_active":"2024-03-07T07:39:14.603444+0000","last_became_peered":"2024-03-07T07:39:14.603444+0000","last_unstale":"2024-03-07T07:39:15.838687+0000","last_undegraded":"2024-03-07T07:39:15.838687+0000","last_fullsized":"2024-03-07T07:39:15.838687+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:18:03.075996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.857806+0000","last_change":"2024-03-07T07:39:13.833864+0000","last_active":"2024-03-07T07:39:16.857806+0000","last_peered":"2024-03-07T07:39:16.857806+0000","last_clean":"2024-03-07T07:39:16.857806+0000","last_became_active":"2024-03-07T07:39:13.833570+0000","last_became_peered":"2024-03-07T07:39:13.833570+0000","last_unstale":"2024-03-07T07:39:16.857806+0000","last_undegraded":"2024-03-07T07:39:16.857806+0000","last_fullsized":"2024-03-07T07:39:16.857806+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:20:16.387319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.604294+0000","last_change":"2024-03-07T07:39:14.610023+0000","last_active":"2024-03-07T07:39:15.604294+0000","last_peered":"2024-03-07T07:39:15.604294+0000","last_clean":"2024-03-07T07:39:15.604294+0000","last_became_active":"2024-03-07T07:39:14.609671+0000","last_became_peered":"2024-03-07T07:39:14.609671+0000","last_unstale":"2024-03-07T07:39:15.604294+0000","last_undegraded":"2024-03-07T07:39:15.604294+0000","last_fullsized":"2024-03-07T07:39:15.604294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:41:48.384795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.125609+0000","last_change":"2024-03-07T07:39:13.841326+0000","last_active":"2024-03-07T07:39:16.125609+0000","last_peered":"2024-03-07T07:39:16.125609+0000","last_clean":"2024-03-07T07:39:16.125609+0000","last_became_active":"2024-03-07T07:39:13.840955+0000","last_became_peered":"2024-03-07T07:39:13.840955+0000","last_unstale":"2024-03-07T07:39:16.125609+0000","last_undegraded":"2024-03-07T07:39:16.125609+0000","last_fullsized":"2024-03-07T07:39:16.125609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:56:25.944653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.838682+0000","last_change":"2024-03-07T07:39:14.617783+0000","last_active":"2024-03-07T07:39:15.838682+0000","last_peered":"2024-03-07T07:39:15.838682+0000","last_clean":"2024-03-07T07:39:15.838682+0000","last_became_active":"2024-03-07T07:39:14.617472+0000","last_became_peered":"2024-03-07T07:39:14.617472+0000","last_unstale":"2024-03-07T07:39:15.838682+0000","last_undegraded":"2024-03-07T07:39:15.838682+0000","last_fullsized":"2024-03-07T07:39:15.838682+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:35:48.618321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.856171+0000","last_change":"2024-03-07T07:39:13.837602+0000","last_active":"2024-03-07T07:39:16.856171+0000","last_peered":"2024-03-07T07:39:16.856171+0000","last_clean":"2024-03-07T07:39:16.856171+0000","last_became_active":"2024-03-07T07:39:13.837275+0000","last_became_peered":"2024-03-07T07:39:13.837275+0000","last_unstale":"2024-03-07T07:39:16.856171+0000","last_undegraded":"2024-03-07T07:39:16.856171+0000","last_fullsized":"2024-03-07T07:39:16.856171+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:40:43.267842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.126518+0000","last_change":"2024-03-07T07:39:14.616702+0000","last_active":"2024-03-07T07:39:16.126518+0000","last_peered":"2024-03-07T07:39:16.126518+0000","last_clean":"2024-03-07T07:39:16.126518+0000","last_became_active":"2024-03-07T07:39:14.614720+0000","last_became_peered":"2024-03-07T07:39:14.614720+0000","last_unstale":"2024-03-07T07:39:16.126518+0000","last_undegraded":"2024-03-07T07:39:16.126518+0000","last_fullsized":"2024-03-07T07:39:16.126518+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:14:45.648231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:17.863211+0000","last_change":"2024-03-07T07:39:13.836641+0000","last_active":"2024-03-07T07:39:17.863211+0000","last_peered":"2024-03-07T07:39:17.863211+0000","last_clean":"2024-03-07T07:39:17.863211+0000","last_became_active":"2024-03-07T07:39:13.834012+0000","last_became_peered":"2024-03-07T07:39:13.834012+0000","last_unstale":"2024-03-07T07:39:17.863211+0000","last_undegraded":"2024-03-07T07:39:17.863211+0000","last_fullsized":"2024-03-07T07:39:17.863211+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:40:49.596816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.126259+0000","last_change":"2024-03-07T07:39:14.611074+0000","last_active":"2024-03-07T07:39:16.126259+0000","last_peered":"2024-03-07T07:39:16.126259+0000","last_clean":"2024-03-07T07:39:16.126259+0000","last_became_active":"2024-03-07T07:39:14.610651+0000","last_became_peered":"2024-03-07T07:39:14.610651+0000","last_unstale":"2024-03-07T07:39:16.126259+0000","last_undegraded":"2024-03-07T07:39:16.126259+0000","last_fullsized":"2024-03-07T07:39:16.126259+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:18:29.684266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.125661+0000","last_change":"2024-03-07T07:39:13.839743+0000","last_active":"2024-03-07T07:39:16.125661+0000","last_peered":"2024-03-07T07:39:16.125661+0000","last_clean":"2024-03-07T07:39:16.125661+0000","last_became_active":"2024-03-07T07:39:13.839079+0000","last_became_peered":"2024-03-07T07:39:13.839079+0000","last_unstale":"2024-03-07T07:39:16.125661+0000","last_undegraded":"2024-03-07T07:39:16.125661+0000","last_fullsized":"2024-03-07T07:39:16.125661+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:11:00.757666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.125867+0000","last_change":"2024-03-07T07:39:14.615941+0000","last_active":"2024-03-07T07:39:16.125867+0000","last_peered":"2024-03-07T07:39:16.125867+0000","last_clean":"2024-03-07T07:39:16.125867+0000","last_became_active":"2024-03-07T07:39:14.615577+0000","last_became_peered":"2024-03-07T07:39:14.615577+0000","last_unstale":"2024-03-07T07:39:16.125867+0000","last_undegraded":"2024-03-07T07:39:16.125867+0000","last_fullsized":"2024-03-07T07:39:16.125867+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:06:05.457297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.603805+0000","last_change":"2024-03-07T07:39:13.842002+0000","last_active":"2024-03-07T07:39:15.603805+0000","last_peered":"2024-03-07T07:39:15.603805+0000","last_clean":"2024-03-07T07:39:15.603805+0000","last_became_active":"2024-03-07T07:39:13.841660+0000","last_became_peered":"2024-03-07T07:39:13.841660+0000","last_unstale":"2024-03-07T07:39:15.603805+0000","last_undegraded":"2024-03-07T07:39:15.603805+0000","last_fullsized":"2024-03-07T07:39:15.603805+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T19:14:58.009793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.126449+0000","last_change":"2024-03-07T07:39:14.611652+0000","last_active":"2024-03-07T07:39:16.126449+0000","last_peered":"2024-03-07T07:39:16.126449+0000","last_clean":"2024-03-07T07:39:16.126449+0000","last_became_active":"2024-03-07T07:39:14.611300+0000","last_became_peered":"2024-03-07T07:39:14.611300+0000","last_unstale":"2024-03-07T07:39:16.126449+0000","last_undegraded":"2024-03-07T07:39:16.126449+0000","last_fullsized":"2024-03-07T07:39:16.126449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:30:44.240560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.604514+0000","last_change":"2024-03-07T07:39:14.617630+0000","last_active":"2024-03-07T07:39:15.604514+0000","last_peered":"2024-03-07T07:39:15.604514+0000","last_clean":"2024-03-07T07:39:15.604514+0000","last_became_active":"2024-03-07T07:39:14.616897+0000","last_became_peered":"2024-03-07T07:39:14.616897+0000","last_unstale":"2024-03-07T07:39:15.604514+0000","last_undegraded":"2024-03-07T07:39:15.604514+0000","last_fullsized":"2024-03-07T07:39:15.604514+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T19:24:35.522835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.125448+0000","last_change":"2024-03-07T07:39:14.616670+0000","last_active":"2024-03-07T07:39:16.125448+0000","last_peered":"2024-03-07T07:39:16.125448+0000","last_clean":"2024-03-07T07:39:16.125448+0000","last_became_active":"2024-03-07T07:39:14.615720+0000","last_became_peered":"2024-03-07T07:39:14.615720+0000","last_unstale":"2024-03-07T07:39:16.125448+0000","last_undegraded":"2024-03-07T07:39:16.125448+0000","last_fullsized":"2024-03-07T07:39:16.125448+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:51:59.533172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.125591+0000","last_change":"2024-03-07T07:39:14.606794+0000","last_active":"2024-03-07T07:39:16.125591+0000","last_peered":"2024-03-07T07:39:16.125591+0000","last_clean":"2024-03-07T07:39:16.125591+0000","last_became_active":"2024-03-07T07:39:14.606395+0000","last_became_peered":"2024-03-07T07:39:14.606395+0000","last_unstale":"2024-03-07T07:39:16.125591+0000","last_undegraded":"2024-03-07T07:39:16.125591+0000","last_fullsized":"2024-03-07T07:39:16.125591+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:54:26.263361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.125250+0000","last_change":"2024-03-07T07:39:14.616627+0000","last_active":"2024-03-07T07:39:16.125250+0000","last_peered":"2024-03-07T07:39:16.125250+0000","last_clean":"2024-03-07T07:39:16.125250+0000","last_became_active":"2024-03-07T07:39:14.615865+0000","last_became_peered":"2024-03-07T07:39:14.615865+0000","last_unstale":"2024-03-07T07:39:16.125250+0000","last_undegraded":"2024-03-07T07:39:16.125250+0000","last_fullsized":"2024-03-07T07:39:16.125250+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:15:46.997786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.603625+0000","last_change":"2024-03-07T07:39:14.616493+0000","last_active":"2024-03-07T07:39:15.603625+0000","last_peered":"2024-03-07T07:39:15.603625+0000","last_clean":"2024-03-07T07:39:15.603625+0000","last_became_active":"2024-03-07T07:39:14.616185+0000","last_became_peered":"2024-03-07T07:39:14.616185+0000","last_unstale":"2024-03-07T07:39:15.603625+0000","last_undegraded":"2024-03-07T07:39:15.603625+0000","last_fullsized":"2024-03-07T07:39:15.603625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:51:57.888472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.604540+0000","last_change":"2024-03-07T07:39:14.614720+0000","last_active":"2024-03-07T07:39:15.604540+0000","last_peered":"2024-03-07T07:39:15.604540+0000","last_clean":"2024-03-07T07:39:15.604540+0000","last_became_active":"2024-03-07T07:39:14.614256+0000","last_became_peered":"2024-03-07T07:39:14.614256+0000","last_unstale":"2024-03-07T07:39:15.604540+0000","last_undegraded":"2024-03-07T07:39:15.604540+0000","last_fullsized":"2024-03-07T07:39:15.604540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:58:24.497664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.126074+0000","last_change":"2024-03-07T07:39:14.611152+0000","last_active":"2024-03-07T07:39:16.126074+0000","last_peered":"2024-03-07T07:39:16.126074+0000","last_clean":"2024-03-07T07:39:16.126074+0000","last_became_active":"2024-03-07T07:39:14.610616+0000","last_became_peered":"2024-03-07T07:39:14.610616+0000","last_unstale":"2024-03-07T07:39:16.126074+0000","last_undegraded":"2024-03-07T07:39:16.126074+0000","last_fullsized":"2024-03-07T07:39:16.126074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:59:36.283163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.838308+0000","last_change":"2024-03-07T07:39:14.610762+0000","last_active":"2024-03-07T07:39:15.838308+0000","last_peered":"2024-03-07T07:39:15.838308+0000","last_clean":"2024-03-07T07:39:15.838308+0000","last_became_active":"2024-03-07T07:39:14.610440+0000","last_became_peered":"2024-03-07T07:39:14.610440+0000","last_unstale":"2024-03-07T07:39:15.838308+0000","last_undegraded":"2024-03-07T07:39:15.838308+0000","last_fullsized":"2024-03-07T07:39:15.838308+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:45:18.046694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.125272+0000","last_change":"2024-03-07T07:39:14.610322+0000","last_active":"2024-03-07T07:39:16.125272+0000","last_peered":"2024-03-07T07:39:16.125272+0000","last_clean":"2024-03-07T07:39:16.125272+0000","last_became_active":"2024-03-07T07:39:14.610008+0000","last_became_peered":"2024-03-07T07:39:14.610008+0000","last_unstale":"2024-03-07T07:39:16.125272+0000","last_undegraded":"2024-03-07T07:39:16.125272+0000","last_fullsized":"2024-03-07T07:39:16.125272+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:35:46.071475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.838058+0000","last_change":"2024-03-07T07:39:14.610084+0000","last_active":"2024-03-07T07:39:15.838058+0000","last_peered":"2024-03-07T07:39:15.838058+0000","last_clean":"2024-03-07T07:39:15.838058+0000","last_became_active":"2024-03-07T07:39:14.609085+0000","last_became_peered":"2024-03-07T07:39:14.609085+0000","last_unstale":"2024-03-07T07:39:15.838058+0000","last_undegraded":"2024-03-07T07:39:15.838058+0000","last_fullsized":"2024-03-07T07:39:15.838058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:24:31.206928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-07T07:39:14.613432+0000","last_change":"2024-03-07T07:39:14.611486+0000","last_active":"2024-03-07T07:39:14.613432+0000","last_peered":"2024-03-07T07:39:14.613432+0000","last_clean":"2024-03-07T07:39:14.613432+0000","last_became_active":"2024-03-07T07:39:14.611128+0000","last_became_peered":"2024-03-07T07:39:14.611128+0000","last_unstale":"2024-03-07T07:39:14.613432+0000","last_undegraded":"2024-03-07T07:39:14.613432+0000","last_fullsized":"2024-03-07T07:39:14.613432+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:34:25.068523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.603721+0000","last_change":"2024-03-07T07:39:14.604771+0000","last_active":"2024-03-07T07:39:15.603721+0000","last_peered":"2024-03-07T07:39:15.603721+0000","last_clean":"2024-03-07T07:39:15.603721+0000","last_became_active":"2024-03-07T07:39:14.604494+0000","last_became_peered":"2024-03-07T07:39:14.604494+0000","last_unstale":"2024-03-07T07:39:15.603721+0000","last_undegraded":"2024-03-07T07:39:15.603721+0000","last_fullsized":"2024-03-07T07:39:15.603721+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:42:45.141833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.604524+0000","last_change":"2024-03-07T07:39:14.618689+0000","last_active":"2024-03-07T07:39:15.604524+0000","last_peered":"2024-03-07T07:39:15.604524+0000","last_clean":"2024-03-07T07:39:15.604524+0000","last_became_active":"2024-03-07T07:39:14.618330+0000","last_became_peered":"2024-03-07T07:39:14.618330+0000","last_unstale":"2024-03-07T07:39:15.604524+0000","last_undegraded":"2024-03-07T07:39:15.604524+0000","last_fullsized":"2024-03-07T07:39:15.604524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:23:07.431656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282597,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":24896,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":131,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":233472,"data_stored":33761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383694,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":901120,"data_stored":429951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645331,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":1032079,"data_compressed":35940,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808856,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":439339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070493,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":439259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332130,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":1030093,"data_compressed":35940,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495655,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":917504,"data_stored":438611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":431563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":917504,"data_stored":438611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":950272,"data_stored":441483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182455,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":438805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920818,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":439556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757293,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":1030365,"data_compressed":35940,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34299,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34299,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34299,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35940,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35940,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35940,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35940,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35940,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":22864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-07T07:39:21.560 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-07T07:39:21.700 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.694+0000 7f752bed4640 1 -- 172.21.15.160:0/1819891241 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7524101770 msgr2=0x7f7524101bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:21.700 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.694+0000 7f752bed4640 1 --2- 172.21.15.160:0/1819891241 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7524101770 0x7f7524101bf0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7514009900 tx=0x7f751402bc30 comp rx=0 tx=0).stop 2024-03-07T07:39:21.700 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.694+0000 7f752bed4640 1 -- 172.21.15.160:0/1819891241 shutdown_connections 2024-03-07T07:39:21.700 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.694+0000 7f752bed4640 1 --2- 172.21.15.160:0/1819891241 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7524102130 0x7f75241089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:21.701 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.694+0000 7f752bed4640 1 --2- 172.21.15.160:0/1819891241 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7524101770 0x7f7524101bf0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:21.701 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.694+0000 7f752bed4640 1 --2- 172.21.15.160:0/1819891241 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7524100580 0x7f75241009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:21.701 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.694+0000 7f752bed4640 1 -- 172.21.15.160:0/1819891241 >> 172.21.15.160:0/1819891241 conn(0x7f75240fbaf0 msgr2=0x7f75240fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:21.701 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.694+0000 7f752bed4640 1 -- 172.21.15.160:0/1819891241 shutdown_connections 2024-03-07T07:39:21.701 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.694+0000 7f752bed4640 1 -- 172.21.15.160:0/1819891241 wait complete. 2024-03-07T07:39:21.702 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.695+0000 7f752bed4640 1 Processor -- start 2024-03-07T07:39:21.702 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.695+0000 7f752bed4640 1 -- start start 2024-03-07T07:39:21.702 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.695+0000 7f752bed4640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7524100580 0x7f752406a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:21.702 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.695+0000 7f752bed4640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7524101770 0x7f752419c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:21.702 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.695+0000 7f752bed4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7524102130 0x7f752419ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:21.702 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.695+0000 7f752bed4640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752410c210 con 0x7f7524101770 2024-03-07T07:39:21.703 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.695+0000 7f752bed4640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752410c090 con 0x7f7524100580 2024-03-07T07:39:21.703 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.695+0000 7f752bed4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752410c390 con 0x7f7524102130 2024-03-07T07:39:21.703 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.695+0000 7f752946f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7524101770 0x7f752419c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:21.703 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.695+0000 7f752946f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7524101770 0x7f752419c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:49744/0 (socket says 172.21.15.160:49744) 2024-03-07T07:39:21.703 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.695+0000 7f752946f640 1 -- 172.21.15.160:0/2012727665 learned_addr learned my addr 172.21.15.160:0/2012727665 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:39:21.703 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.695+0000 7f752a471640 1 --2- 172.21.15.160:0/2012727665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7524102130 0x7f752419ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:21.704 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.695+0000 7f7529c70640 1 --2- 172.21.15.160:0/2012727665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7524100580 0x7f752406a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:21.704 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.695+0000 7f752946f640 1 -- 172.21.15.160:0/2012727665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7524102130 msgr2=0x7f752419ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:21.704 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.695+0000 7f752946f640 1 --2- 172.21.15.160:0/2012727665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7524102130 0x7f752419ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:21.704 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.695+0000 7f752946f640 1 -- 172.21.15.160:0/2012727665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7524100580 msgr2=0x7f752406a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:21.704 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.695+0000 7f752946f640 1 --2- 172.21.15.160:0/2012727665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7524100580 0x7f752406a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:21.704 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.695+0000 7f752946f640 1 -- 172.21.15.160:0/2012727665 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f752419f590 con 0x7f7524101770 2024-03-07T07:39:21.705 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.695+0000 7f752946f640 1 --2- 172.21.15.160:0/2012727665 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7524101770 0x7f752419c690 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7514009230 tx=0x7f751402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:21.705 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.695+0000 7f751affd640 1 -- 172.21.15.160:0/2012727665 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7514041070 con 0x7f7524101770 2024-03-07T07:39:21.705 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.695+0000 7f751affd640 1 -- 172.21.15.160:0/2012727665 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7514004450 con 0x7f7524101770 2024-03-07T07:39:21.705 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.695+0000 7f752bed4640 1 -- 172.21.15.160:0/2012727665 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f752419f820 con 0x7f7524101770 2024-03-07T07:39:21.705 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.695+0000 7f751affd640 1 -- 172.21.15.160:0/2012727665 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7514003e20 con 0x7f7524101770 2024-03-07T07:39:21.705 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.696+0000 7f752bed4640 1 -- 172.21.15.160:0/2012727665 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f752419fd10 con 0x7f7524101770 2024-03-07T07:39:21.706 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.696+0000 7f752bed4640 1 -- 172.21.15.160:0/2012727665 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75241078c0 con 0x7f7524101770 2024-03-07T07:39:21.706 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.697+0000 7f751affd640 1 -- 172.21.15.160:0/2012727665 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) v1 ==== 136805+0+0 (secure 0 0 0) 0x7f7514044070 con 0x7f7524101770 2024-03-07T07:39:21.706 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.698+0000 7f751affd640 1 --2- 172.21.15.160:0/2012727665 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f75100a1e10 0x7f75100a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:21.706 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.698+0000 7f7529c70640 1 --2- 172.21.15.160:0/2012727665 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f75100a1e10 0x7f75100a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:21.706 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.698+0000 7f751affd640 1 -- 172.21.15.160:0/2012727665 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f75140ef7f0 con 0x7f7524101770 2024-03-07T07:39:21.706 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.698+0000 7f7529c70640 1 --2- 172.21.15.160:0/2012727665 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f75100a1e10 0x7f75100a42d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f75241015d0 tx=0x7f7520005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:21.707 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.702+0000 7f751affd640 1 -- 172.21.15.160:0/2012727665 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f75140bc730 con 0x7f7524101770 2024-03-07T07:39:21.876 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.876+0000 7f752bed4640 1 -- 172.21.15.160:0/2012727665 --> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7524061aa0 con 0x7f75100a1e10 2024-03-07T07:39:21.883 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.883+0000 7f751affd640 1 -- 172.21.15.160:0/2012727665 <== mgr.14152 v2:172.21.15.160:6800/419630619 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+258939 (secure 0 0 0) 0x7f7524061aa0 con 0x7f75100a1e10 2024-03-07T07:39:21.886 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:39:21.887 INFO:teuthology.orchestra.run.smithi160.stderr:dumped all 2024-03-07T07:39:21.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.886+0000 7f752bed4640 1 -- 172.21.15.160:0/2012727665 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f75100a1e10 msgr2=0x7f75100a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:21.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.886+0000 7f752bed4640 1 --2- 172.21.15.160:0/2012727665 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f75100a1e10 0x7f75100a42d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f75241015d0 tx=0x7f7520005f90 comp rx=0 tx=0).stop 2024-03-07T07:39:21.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.886+0000 7f752bed4640 1 -- 172.21.15.160:0/2012727665 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7524101770 msgr2=0x7f752419c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:21.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.886+0000 7f752bed4640 1 --2- 172.21.15.160:0/2012727665 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7524101770 0x7f752419c690 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7514009230 tx=0x7f751402c650 comp rx=0 tx=0).stop 2024-03-07T07:39:21.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.886+0000 7f752bed4640 1 -- 172.21.15.160:0/2012727665 shutdown_connections 2024-03-07T07:39:21.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.886+0000 7f752bed4640 1 --2- 172.21.15.160:0/2012727665 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f75100a1e10 0x7f75100a42d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:21.890 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.886+0000 7f752bed4640 1 --2- 172.21.15.160:0/2012727665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7524102130 0x7f752419ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:21.890 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.886+0000 7f752bed4640 1 --2- 172.21.15.160:0/2012727665 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7524101770 0x7f752419c690 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:21.890 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.886+0000 7f752bed4640 1 --2- 172.21.15.160:0/2012727665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7524100580 0x7f752406a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:21.890 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.886+0000 7f752bed4640 1 -- 172.21.15.160:0/2012727665 >> 172.21.15.160:0/2012727665 conn(0x7f75240fbaf0 msgr2=0x7f7524104a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:21.890 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.886+0000 7f752bed4640 1 -- 172.21.15.160:0/2012727665 shutdown_connections 2024-03-07T07:39:21.890 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:21.886+0000 7f752bed4640 1 -- 172.21.15.160:0/2012727665 wait complete. 2024-03-07T07:39:21.897 INFO:teuthology.orchestra.run.smithi160.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2024-03-07T07:39:20.271777+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":615381,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":144,"num_read_kb":124,"num_write":313,"num_write_kb":3000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":253,"ondisk_log_size":253,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":259,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360224,"kb_used_data":11032,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581600,"statfs":{"total":1151940427776,"available":1151571558400,"internally_reserved":0,"allocated":11296768,"data_stored":7029715,"data_compressed":107820,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25013,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":132,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.455863"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.604658+0000","last_change":"2024-03-07T07:39:14.614168+0000","last_active":"2024-03-07T07:39:15.604658+0000","last_peered":"2024-03-07T07:39:15.604658+0000","last_clean":"2024-03-07T07:39:15.604658+0000","last_became_active":"2024-03-07T07:39:14.613708+0000","last_became_peered":"2024-03-07T07:39:14.613708+0000","last_unstale":"2024-03-07T07:39:15.604658+0000","last_undegraded":"2024-03-07T07:39:15.604658+0000","last_fullsized":"2024-03-07T07:39:15.604658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:08:07.286470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.125350+0000","last_change":"2024-03-07T07:39:14.609425+0000","last_active":"2024-03-07T07:39:16.125350+0000","last_peered":"2024-03-07T07:39:16.125350+0000","last_clean":"2024-03-07T07:39:16.125350+0000","last_became_active":"2024-03-07T07:39:14.609060+0000","last_became_peered":"2024-03-07T07:39:14.609060+0000","last_unstale":"2024-03-07T07:39:16.125350+0000","last_undegraded":"2024-03-07T07:39:16.125350+0000","last_fullsized":"2024-03-07T07:39:16.125350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:37:33.773358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.125303+0000","last_change":"2024-03-07T07:39:14.609985+0000","last_active":"2024-03-07T07:39:16.125303+0000","last_peered":"2024-03-07T07:39:16.125303+0000","last_clean":"2024-03-07T07:39:16.125303+0000","last_became_active":"2024-03-07T07:39:14.609490+0000","last_became_peered":"2024-03-07T07:39:14.609490+0000","last_unstale":"2024-03-07T07:39:16.125303+0000","last_undegraded":"2024-03-07T07:39:16.125303+0000","last_fullsized":"2024-03-07T07:39:16.125303+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T19:09:24.134718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.126341+0000","last_change":"2024-03-07T07:39:14.620482+0000","last_active":"2024-03-07T07:39:16.126341+0000","last_peered":"2024-03-07T07:39:16.126341+0000","last_clean":"2024-03-07T07:39:16.126341+0000","last_became_active":"2024-03-07T07:39:14.620100+0000","last_became_peered":"2024-03-07T07:39:14.620100+0000","last_unstale":"2024-03-07T07:39:16.126341+0000","last_undegraded":"2024-03-07T07:39:16.126341+0000","last_fullsized":"2024-03-07T07:39:16.126341+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:03:20.291833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.838708+0000","last_change":"2024-03-07T07:39:14.603228+0000","last_active":"2024-03-07T07:39:15.838708+0000","last_peered":"2024-03-07T07:39:15.838708+0000","last_clean":"2024-03-07T07:39:15.838708+0000","last_became_active":"2024-03-07T07:39:14.602874+0000","last_became_peered":"2024-03-07T07:39:14.602874+0000","last_unstale":"2024-03-07T07:39:15.838708+0000","last_undegraded":"2024-03-07T07:39:15.838708+0000","last_fullsized":"2024-03-07T07:39:15.838708+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:26:17.762271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.856483+0000","last_change":"2024-03-07T07:39:14.611556+0000","last_active":"2024-03-07T07:39:15.856483+0000","last_peered":"2024-03-07T07:39:15.856483+0000","last_clean":"2024-03-07T07:39:15.856483+0000","last_became_active":"2024-03-07T07:39:14.609938+0000","last_became_peered":"2024-03-07T07:39:14.609938+0000","last_unstale":"2024-03-07T07:39:15.856483+0000","last_undegraded":"2024-03-07T07:39:15.856483+0000","last_fullsized":"2024-03-07T07:39:15.856483+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:51:49.013178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.125796+0000","last_change":"2024-03-07T07:39:14.615259+0000","last_active":"2024-03-07T07:39:16.125796+0000","last_peered":"2024-03-07T07:39:16.125796+0000","last_clean":"2024-03-07T07:39:16.125796+0000","last_became_active":"2024-03-07T07:39:14.614878+0000","last_became_peered":"2024-03-07T07:39:14.614878+0000","last_unstale":"2024-03-07T07:39:16.125796+0000","last_undegraded":"2024-03-07T07:39:16.125796+0000","last_fullsized":"2024-03-07T07:39:16.125796+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:22:28.905360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.838241+0000","last_change":"2024-03-07T07:39:14.614518+0000","last_active":"2024-03-07T07:39:15.838241+0000","last_peered":"2024-03-07T07:39:15.838241+0000","last_clean":"2024-03-07T07:39:15.838241+0000","last_became_active":"2024-03-07T07:39:14.614308+0000","last_became_peered":"2024-03-07T07:39:14.614308+0000","last_unstale":"2024-03-07T07:39:15.838241+0000","last_undegraded":"2024-03-07T07:39:15.838241+0000","last_fullsized":"2024-03-07T07:39:15.838241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:54:54.908066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-07T07:39:14.617630+0000","last_change":"2024-03-07T07:39:14.615360+0000","last_active":"2024-03-07T07:39:14.617630+0000","last_peered":"2024-03-07T07:39:14.617630+0000","last_clean":"2024-03-07T07:39:14.617630+0000","last_became_active":"2024-03-07T07:39:14.615036+0000","last_became_peered":"2024-03-07T07:39:14.615036+0000","last_unstale":"2024-03-07T07:39:14.617630+0000","last_undegraded":"2024-03-07T07:39:14.617630+0000","last_fullsized":"2024-03-07T07:39:14.617630+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:24:36.633406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.856814+0000","last_change":"2024-03-07T07:39:14.610450+0000","last_active":"2024-03-07T07:39:15.856814+0000","last_peered":"2024-03-07T07:39:15.856814+0000","last_clean":"2024-03-07T07:39:15.856814+0000","last_became_active":"2024-03-07T07:39:14.609483+0000","last_became_peered":"2024-03-07T07:39:14.609483+0000","last_unstale":"2024-03-07T07:39:15.856814+0000","last_undegraded":"2024-03-07T07:39:15.856814+0000","last_fullsized":"2024-03-07T07:39:15.856814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:37:59.323371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.126110+0000","last_change":"2024-03-07T07:39:14.611800+0000","last_active":"2024-03-07T07:39:16.126110+0000","last_peered":"2024-03-07T07:39:16.126110+0000","last_clean":"2024-03-07T07:39:16.126110+0000","last_became_active":"2024-03-07T07:39:14.611482+0000","last_became_peered":"2024-03-07T07:39:14.611482+0000","last_unstale":"2024-03-07T07:39:16.126110+0000","last_undegraded":"2024-03-07T07:39:16.126110+0000","last_fullsized":"2024-03-07T07:39:16.126110+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:26:36.361716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.126171+0000","last_change":"2024-03-07T07:39:14.621686+0000","last_active":"2024-03-07T07:39:16.126171+0000","last_peered":"2024-03-07T07:39:16.126171+0000","last_clean":"2024-03-07T07:39:16.126171+0000","last_became_active":"2024-03-07T07:39:14.620815+0000","last_became_peered":"2024-03-07T07:39:14.620815+0000","last_unstale":"2024-03-07T07:39:16.126171+0000","last_undegraded":"2024-03-07T07:39:16.126171+0000","last_fullsized":"2024-03-07T07:39:16.126171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:33:59.972215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.603717+0000","last_change":"2024-03-07T07:39:14.606045+0000","last_active":"2024-03-07T07:39:15.603717+0000","last_peered":"2024-03-07T07:39:15.603717+0000","last_clean":"2024-03-07T07:39:15.603717+0000","last_became_active":"2024-03-07T07:39:14.605726+0000","last_became_peered":"2024-03-07T07:39:14.605726+0000","last_unstale":"2024-03-07T07:39:15.603717+0000","last_undegraded":"2024-03-07T07:39:15.603717+0000","last_fullsized":"2024-03-07T07:39:15.603717+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:13:51.899507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.125192+0000","last_change":"2024-03-07T07:39:14.605435+0000","last_active":"2024-03-07T07:39:16.125192+0000","last_peered":"2024-03-07T07:39:16.125192+0000","last_clean":"2024-03-07T07:39:16.125192+0000","last_became_active":"2024-03-07T07:39:14.605127+0000","last_became_peered":"2024-03-07T07:39:14.605127+0000","last_unstale":"2024-03-07T07:39:16.125192+0000","last_undegraded":"2024-03-07T07:39:16.125192+0000","last_fullsized":"2024-03-07T07:39:16.125192+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:15:58.206921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.125256+0000","last_change":"2024-03-07T07:39:14.617166+0000","last_active":"2024-03-07T07:39:16.125256+0000","last_peered":"2024-03-07T07:39:16.125256+0000","last_clean":"2024-03-07T07:39:16.125256+0000","last_became_active":"2024-03-07T07:39:14.616615+0000","last_became_peered":"2024-03-07T07:39:14.616615+0000","last_unstale":"2024-03-07T07:39:16.125256+0000","last_undegraded":"2024-03-07T07:39:16.125256+0000","last_fullsized":"2024-03-07T07:39:16.125256+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:25:05.086822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-07T07:39:14.613395+0000","last_change":"2024-03-07T07:39:14.611990+0000","last_active":"2024-03-07T07:39:14.613395+0000","last_peered":"2024-03-07T07:39:14.613395+0000","last_clean":"2024-03-07T07:39:14.613395+0000","last_became_active":"2024-03-07T07:39:14.610728+0000","last_became_peered":"2024-03-07T07:39:14.610728+0000","last_unstale":"2024-03-07T07:39:14.613395+0000","last_undegraded":"2024-03-07T07:39:14.613395+0000","last_fullsized":"2024-03-07T07:39:14.613395+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T19:21:42.252823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.604704+0000","last_change":"2024-03-07T07:39:14.615409+0000","last_active":"2024-03-07T07:39:15.604704+0000","last_peered":"2024-03-07T07:39:15.604704+0000","last_clean":"2024-03-07T07:39:15.604704+0000","last_became_active":"2024-03-07T07:39:14.615003+0000","last_became_peered":"2024-03-07T07:39:14.615003+0000","last_unstale":"2024-03-07T07:39:15.604704+0000","last_undegraded":"2024-03-07T07:39:15.604704+0000","last_fullsized":"2024-03-07T07:39:15.604704+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:33:56.464718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.836417+0000","last_change":"2024-03-07T07:39:14.610501+0000","last_active":"2024-03-07T07:39:15.836417+0000","last_peered":"2024-03-07T07:39:15.836417+0000","last_clean":"2024-03-07T07:39:15.836417+0000","last_became_active":"2024-03-07T07:39:14.610190+0000","last_became_peered":"2024-03-07T07:39:14.610190+0000","last_unstale":"2024-03-07T07:39:15.836417+0000","last_undegraded":"2024-03-07T07:39:15.836417+0000","last_fullsized":"2024-03-07T07:39:15.836417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:05:31.360270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.125986+0000","last_change":"2024-03-07T07:39:14.609373+0000","last_active":"2024-03-07T07:39:16.125986+0000","last_peered":"2024-03-07T07:39:16.125986+0000","last_clean":"2024-03-07T07:39:16.125986+0000","last_became_active":"2024-03-07T07:39:14.608174+0000","last_became_peered":"2024-03-07T07:39:14.608174+0000","last_unstale":"2024-03-07T07:39:16.125986+0000","last_undegraded":"2024-03-07T07:39:16.125986+0000","last_fullsized":"2024-03-07T07:39:16.125986+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:25:18.704239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:17.861977+0000","last_change":"2024-03-07T07:39:13.830499+0000","last_active":"2024-03-07T07:39:17.861977+0000","last_peered":"2024-03-07T07:39:17.861977+0000","last_clean":"2024-03-07T07:39:17.861977+0000","last_became_active":"2024-03-07T07:39:13.830131+0000","last_became_peered":"2024-03-07T07:39:13.830131+0000","last_unstale":"2024-03-07T07:39:17.861977+0000","last_undegraded":"2024-03-07T07:39:17.861977+0000","last_fullsized":"2024-03-07T07:39:17.861977+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:48:14.338758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.838489+0000","last_change":"2024-03-07T07:39:14.612715+0000","last_active":"2024-03-07T07:39:15.838489+0000","last_peered":"2024-03-07T07:39:15.838489+0000","last_clean":"2024-03-07T07:39:15.838489+0000","last_became_active":"2024-03-07T07:39:14.612383+0000","last_became_peered":"2024-03-07T07:39:14.612383+0000","last_unstale":"2024-03-07T07:39:15.838489+0000","last_undegraded":"2024-03-07T07:39:15.838489+0000","last_fullsized":"2024-03-07T07:39:15.838489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:17:36.800396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.838702+0000","last_change":"2024-03-07T07:39:13.830901+0000","last_active":"2024-03-07T07:39:15.838702+0000","last_peered":"2024-03-07T07:39:15.838702+0000","last_clean":"2024-03-07T07:39:15.838702+0000","last_became_active":"2024-03-07T07:39:13.830581+0000","last_became_peered":"2024-03-07T07:39:13.830581+0000","last_unstale":"2024-03-07T07:39:15.838702+0000","last_undegraded":"2024-03-07T07:39:15.838702+0000","last_fullsized":"2024-03-07T07:39:15.838702+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:25:15.428381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.125398+0000","last_change":"2024-03-07T07:39:14.610876+0000","last_active":"2024-03-07T07:39:16.125398+0000","last_peered":"2024-03-07T07:39:16.125398+0000","last_clean":"2024-03-07T07:39:16.125398+0000","last_became_active":"2024-03-07T07:39:14.610584+0000","last_became_peered":"2024-03-07T07:39:14.610584+0000","last_unstale":"2024-03-07T07:39:16.125398+0000","last_undegraded":"2024-03-07T07:39:16.125398+0000","last_fullsized":"2024-03-07T07:39:16.125398+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:55:11.818462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:17.861707+0000","last_change":"2024-03-07T07:39:13.838602+0000","last_active":"2024-03-07T07:39:17.861707+0000","last_peered":"2024-03-07T07:39:17.861707+0000","last_clean":"2024-03-07T07:39:17.861707+0000","last_became_active":"2024-03-07T07:39:13.838061+0000","last_became_peered":"2024-03-07T07:39:13.838061+0000","last_unstale":"2024-03-07T07:39:17.861707+0000","last_undegraded":"2024-03-07T07:39:17.861707+0000","last_fullsized":"2024-03-07T07:39:17.861707+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:42:33.589280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-07T07:39:14.613125+0000","last_change":"2024-03-07T07:39:14.607666+0000","last_active":"2024-03-07T07:39:14.613125+0000","last_peered":"2024-03-07T07:39:14.613125+0000","last_clean":"2024-03-07T07:39:14.613125+0000","last_became_active":"2024-03-07T07:39:14.607339+0000","last_became_peered":"2024-03-07T07:39:14.607339+0000","last_unstale":"2024-03-07T07:39:14.613125+0000","last_undegraded":"2024-03-07T07:39:14.613125+0000","last_fullsized":"2024-03-07T07:39:14.613125+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:57:46.319015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.839017+0000","last_change":"2024-03-07T07:39:13.829179+0000","last_active":"2024-03-07T07:39:15.839017+0000","last_peered":"2024-03-07T07:39:15.839017+0000","last_clean":"2024-03-07T07:39:15.839017+0000","last_became_active":"2024-03-07T07:39:13.828162+0000","last_became_peered":"2024-03-07T07:39:13.828162+0000","last_unstale":"2024-03-07T07:39:15.839017+0000","last_undegraded":"2024-03-07T07:39:15.839017+0000","last_fullsized":"2024-03-07T07:39:15.839017+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:38:00.330980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.125759+0000","last_change":"2024-03-07T07:39:14.621496+0000","last_active":"2024-03-07T07:39:16.125759+0000","last_peered":"2024-03-07T07:39:16.125759+0000","last_clean":"2024-03-07T07:39:16.125759+0000","last_became_active":"2024-03-07T07:39:14.621139+0000","last_became_peered":"2024-03-07T07:39:14.621139+0000","last_unstale":"2024-03-07T07:39:16.125759+0000","last_undegraded":"2024-03-07T07:39:16.125759+0000","last_fullsized":"2024-03-07T07:39:16.125759+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:24:45.553536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:18.869990+0000","last_change":"2024-03-07T07:39:13.834999+0000","last_active":"2024-03-07T07:39:18.869990+0000","last_peered":"2024-03-07T07:39:18.869990+0000","last_clean":"2024-03-07T07:39:18.869990+0000","last_became_active":"2024-03-07T07:39:13.834672+0000","last_became_peered":"2024-03-07T07:39:13.834672+0000","last_unstale":"2024-03-07T07:39:18.869990+0000","last_undegraded":"2024-03-07T07:39:18.869990+0000","last_fullsized":"2024-03-07T07:39:18.869990+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:55:57.944992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.838954+0000","last_change":"2024-03-07T07:39:14.605519+0000","last_active":"2024-03-07T07:39:15.838954+0000","last_peered":"2024-03-07T07:39:15.838954+0000","last_clean":"2024-03-07T07:39:15.838954+0000","last_became_active":"2024-03-07T07:39:14.605185+0000","last_became_peered":"2024-03-07T07:39:14.605185+0000","last_unstale":"2024-03-07T07:39:15.838954+0000","last_undegraded":"2024-03-07T07:39:15.838954+0000","last_fullsized":"2024-03-07T07:39:15.838954+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:54:33.622751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:17.861391+0000","last_change":"2024-03-07T07:39:13.837186+0000","last_active":"2024-03-07T07:39:17.861391+0000","last_peered":"2024-03-07T07:39:17.861391+0000","last_clean":"2024-03-07T07:39:17.861391+0000","last_became_active":"2024-03-07T07:39:13.836807+0000","last_became_peered":"2024-03-07T07:39:13.836807+0000","last_unstale":"2024-03-07T07:39:17.861391+0000","last_undegraded":"2024-03-07T07:39:17.861391+0000","last_fullsized":"2024-03-07T07:39:17.861391+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:32:39.574197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.125881+0000","last_change":"2024-03-07T07:39:14.621503+0000","last_active":"2024-03-07T07:39:16.125881+0000","last_peered":"2024-03-07T07:39:16.125881+0000","last_clean":"2024-03-07T07:39:16.125881+0000","last_became_active":"2024-03-07T07:39:14.621146+0000","last_became_peered":"2024-03-07T07:39:14.621146+0000","last_unstale":"2024-03-07T07:39:16.125881+0000","last_undegraded":"2024-03-07T07:39:16.125881+0000","last_fullsized":"2024-03-07T07:39:16.125881+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:50:15.270355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:17.862334+0000","last_change":"2024-03-07T07:39:13.838146+0000","last_active":"2024-03-07T07:39:17.862334+0000","last_peered":"2024-03-07T07:39:17.862334+0000","last_clean":"2024-03-07T07:39:17.862334+0000","last_became_active":"2024-03-07T07:39:13.837788+0000","last_became_peered":"2024-03-07T07:39:13.837788+0000","last_unstale":"2024-03-07T07:39:17.862334+0000","last_undegraded":"2024-03-07T07:39:17.862334+0000","last_fullsized":"2024-03-07T07:39:17.862334+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:50:52.154146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.838105+0000","last_change":"2024-03-07T07:39:14.605069+0000","last_active":"2024-03-07T07:39:15.838105+0000","last_peered":"2024-03-07T07:39:15.838105+0000","last_clean":"2024-03-07T07:39:15.838105+0000","last_became_active":"2024-03-07T07:39:14.604733+0000","last_became_peered":"2024-03-07T07:39:14.604733+0000","last_unstale":"2024-03-07T07:39:15.838105+0000","last_undegraded":"2024-03-07T07:39:15.838105+0000","last_fullsized":"2024-03-07T07:39:15.838105+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:49:08.312385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.125163+0000","last_change":"2024-03-07T07:39:13.837854+0000","last_active":"2024-03-07T07:39:16.125163+0000","last_peered":"2024-03-07T07:39:16.125163+0000","last_clean":"2024-03-07T07:39:16.125163+0000","last_became_active":"2024-03-07T07:39:13.837428+0000","last_became_peered":"2024-03-07T07:39:13.837428+0000","last_unstale":"2024-03-07T07:39:16.125163+0000","last_undegraded":"2024-03-07T07:39:16.125163+0000","last_fullsized":"2024-03-07T07:39:16.125163+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:35:15.369936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.836296+0000","last_change":"2024-03-07T07:39:14.606299+0000","last_active":"2024-03-07T07:39:15.836296+0000","last_peered":"2024-03-07T07:39:15.836296+0000","last_clean":"2024-03-07T07:39:15.836296+0000","last_became_active":"2024-03-07T07:39:14.605983+0000","last_became_peered":"2024-03-07T07:39:14.605983+0000","last_unstale":"2024-03-07T07:39:15.836296+0000","last_undegraded":"2024-03-07T07:39:15.836296+0000","last_fullsized":"2024-03-07T07:39:15.836296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:53:16.658862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.859391+0000","last_change":"2024-03-07T07:39:13.840129+0000","last_active":"2024-03-07T07:39:16.859391+0000","last_peered":"2024-03-07T07:39:16.859391+0000","last_clean":"2024-03-07T07:39:16.859391+0000","last_became_active":"2024-03-07T07:39:13.839735+0000","last_became_peered":"2024-03-07T07:39:13.839735+0000","last_unstale":"2024-03-07T07:39:16.859391+0000","last_undegraded":"2024-03-07T07:39:16.859391+0000","last_fullsized":"2024-03-07T07:39:16.859391+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:31:38.711810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-07T07:39:14.613145+0000","last_change":"2024-03-07T07:39:14.606449+0000","last_active":"2024-03-07T07:39:14.613145+0000","last_peered":"2024-03-07T07:39:14.613145+0000","last_clean":"2024-03-07T07:39:14.613145+0000","last_became_active":"2024-03-07T07:39:14.606009+0000","last_became_peered":"2024-03-07T07:39:14.606009+0000","last_unstale":"2024-03-07T07:39:14.613145+0000","last_undegraded":"2024-03-07T07:39:14.613145+0000","last_fullsized":"2024-03-07T07:39:14.613145+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:33:31.383436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.857460+0000","last_change":"2024-03-07T07:39:13.834097+0000","last_active":"2024-03-07T07:39:16.857460+0000","last_peered":"2024-03-07T07:39:16.857460+0000","last_clean":"2024-03-07T07:39:16.857460+0000","last_became_active":"2024-03-07T07:39:13.833749+0000","last_became_peered":"2024-03-07T07:39:13.833749+0000","last_unstale":"2024-03-07T07:39:16.857460+0000","last_undegraded":"2024-03-07T07:39:16.857460+0000","last_fullsized":"2024-03-07T07:39:16.857460+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:12:22.471596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.125477+0000","last_change":"2024-03-07T07:39:14.621000+0000","last_active":"2024-03-07T07:39:16.125477+0000","last_peered":"2024-03-07T07:39:16.125477+0000","last_clean":"2024-03-07T07:39:16.125477+0000","last_became_active":"2024-03-07T07:39:14.620708+0000","last_became_peered":"2024-03-07T07:39:14.620708+0000","last_unstale":"2024-03-07T07:39:16.125477+0000","last_undegraded":"2024-03-07T07:39:16.125477+0000","last_fullsized":"2024-03-07T07:39:16.125477+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:26:48.363406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.858243+0000","last_change":"2024-03-07T07:39:13.837442+0000","last_active":"2024-03-07T07:39:16.858243+0000","last_peered":"2024-03-07T07:39:16.858243+0000","last_clean":"2024-03-07T07:39:16.858243+0000","last_became_active":"2024-03-07T07:39:13.836515+0000","last_became_peered":"2024-03-07T07:39:13.836515+0000","last_unstale":"2024-03-07T07:39:16.858243+0000","last_undegraded":"2024-03-07T07:39:16.858243+0000","last_fullsized":"2024-03-07T07:39:16.858243+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:19:08.312359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-07T07:39:14.620098+0000","last_change":"2024-03-07T07:39:14.619130+0000","last_active":"2024-03-07T07:39:14.620098+0000","last_peered":"2024-03-07T07:39:14.620098+0000","last_clean":"2024-03-07T07:39:14.620098+0000","last_became_active":"2024-03-07T07:39:14.618814+0000","last_became_peered":"2024-03-07T07:39:14.618814+0000","last_unstale":"2024-03-07T07:39:14.620098+0000","last_undegraded":"2024-03-07T07:39:14.620098+0000","last_fullsized":"2024-03-07T07:39:14.620098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:56:06.691897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:19.554520+0000","last_change":"2024-03-07T07:39:13.841051+0000","last_active":"2024-03-07T07:39:19.554520+0000","last_peered":"2024-03-07T07:39:19.554520+0000","last_clean":"2024-03-07T07:39:19.554520+0000","last_became_active":"2024-03-07T07:39:13.840719+0000","last_became_peered":"2024-03-07T07:39:13.840719+0000","last_unstale":"2024-03-07T07:39:19.554520+0000","last_undegraded":"2024-03-07T07:39:19.554520+0000","last_fullsized":"2024-03-07T07:39:19.554520+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T19:33:57.327673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20895,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":16,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.839154+0000","last_change":"2024-03-07T07:39:14.606539+0000","last_active":"2024-03-07T07:39:15.839154+0000","last_peered":"2024-03-07T07:39:15.839154+0000","last_clean":"2024-03-07T07:39:15.839154+0000","last_became_active":"2024-03-07T07:39:14.605832+0000","last_became_peered":"2024-03-07T07:39:14.605832+0000","last_unstale":"2024-03-07T07:39:15.839154+0000","last_undegraded":"2024-03-07T07:39:15.839154+0000","last_fullsized":"2024-03-07T07:39:15.839154+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:08:10.781906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:17.862546+0000","last_change":"2024-03-07T07:39:13.841526+0000","last_active":"2024-03-07T07:39:17.862546+0000","last_peered":"2024-03-07T07:39:17.862546+0000","last_clean":"2024-03-07T07:39:17.862546+0000","last_became_active":"2024-03-07T07:39:13.841153+0000","last_became_peered":"2024-03-07T07:39:13.841153+0000","last_unstale":"2024-03-07T07:39:17.862546+0000","last_undegraded":"2024-03-07T07:39:17.862546+0000","last_fullsized":"2024-03-07T07:39:17.862546+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T19:38:41.264033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.603494+0000","last_change":"2024-03-07T07:39:14.603948+0000","last_active":"2024-03-07T07:39:15.603494+0000","last_peered":"2024-03-07T07:39:15.603494+0000","last_clean":"2024-03-07T07:39:15.603494+0000","last_became_active":"2024-03-07T07:39:14.603724+0000","last_became_peered":"2024-03-07T07:39:14.603724+0000","last_unstale":"2024-03-07T07:39:15.603494+0000","last_undegraded":"2024-03-07T07:39:15.603494+0000","last_fullsized":"2024-03-07T07:39:15.603494+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T19:29:59.296537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.857443+0000","last_change":"2024-03-07T07:39:13.831965+0000","last_active":"2024-03-07T07:39:16.857443+0000","last_peered":"2024-03-07T07:39:16.857443+0000","last_clean":"2024-03-07T07:39:16.857443+0000","last_became_active":"2024-03-07T07:39:13.831617+0000","last_became_peered":"2024-03-07T07:39:13.831617+0000","last_unstale":"2024-03-07T07:39:16.857443+0000","last_undegraded":"2024-03-07T07:39:16.857443+0000","last_fullsized":"2024-03-07T07:39:16.857443+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:46:49.060697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.126415+0000","last_change":"2024-03-07T07:39:14.616265+0000","last_active":"2024-03-07T07:39:16.126415+0000","last_peered":"2024-03-07T07:39:16.126415+0000","last_clean":"2024-03-07T07:39:16.126415+0000","last_became_active":"2024-03-07T07:39:14.615206+0000","last_became_peered":"2024-03-07T07:39:14.615206+0000","last_unstale":"2024-03-07T07:39:16.126415+0000","last_undegraded":"2024-03-07T07:39:16.126415+0000","last_fullsized":"2024-03-07T07:39:16.126415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:39:21.997327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'4","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:18.873027+0000","last_change":"2024-03-07T07:39:13.833311+0000","last_active":"2024-03-07T07:39:18.873027+0000","last_peered":"2024-03-07T07:39:18.873027+0000","last_clean":"2024-03-07T07:39:18.873027+0000","last_became_active":"2024-03-07T07:39:13.832863+0000","last_became_peered":"2024-03-07T07:39:13.832863+0000","last_unstale":"2024-03-07T07:39:18.873027+0000","last_undegraded":"2024-03-07T07:39:18.873027+0000","last_fullsized":"2024-03-07T07:39:18.873027+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:43:50.988905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.125661+0000","last_change":"2024-03-07T07:39:14.615740+0000","last_active":"2024-03-07T07:39:16.125661+0000","last_peered":"2024-03-07T07:39:16.125661+0000","last_clean":"2024-03-07T07:39:16.125661+0000","last_became_active":"2024-03-07T07:39:14.615394+0000","last_became_peered":"2024-03-07T07:39:14.615394+0000","last_unstale":"2024-03-07T07:39:16.125661+0000","last_undegraded":"2024-03-07T07:39:16.125661+0000","last_fullsized":"2024-03-07T07:39:16.125661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:17:19.107374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:17.863654+0000","last_change":"2024-03-07T07:39:13.838231+0000","last_active":"2024-03-07T07:39:17.863654+0000","last_peered":"2024-03-07T07:39:17.863654+0000","last_clean":"2024-03-07T07:39:17.863654+0000","last_became_active":"2024-03-07T07:39:13.837895+0000","last_became_peered":"2024-03-07T07:39:13.837895+0000","last_unstale":"2024-03-07T07:39:17.863654+0000","last_undegraded":"2024-03-07T07:39:17.863654+0000","last_fullsized":"2024-03-07T07:39:17.863654+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:32:48.381613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.125232+0000","last_change":"2024-03-07T07:39:14.617134+0000","last_active":"2024-03-07T07:39:16.125232+0000","last_peered":"2024-03-07T07:39:16.125232+0000","last_clean":"2024-03-07T07:39:16.125232+0000","last_became_active":"2024-03-07T07:39:14.616813+0000","last_became_peered":"2024-03-07T07:39:14.616813+0000","last_unstale":"2024-03-07T07:39:16.125232+0000","last_undegraded":"2024-03-07T07:39:16.125232+0000","last_fullsized":"2024-03-07T07:39:16.125232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:12:18.477309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.839078+0000","last_change":"2024-03-07T07:39:13.828783+0000","last_active":"2024-03-07T07:39:15.839078+0000","last_peered":"2024-03-07T07:39:15.839078+0000","last_clean":"2024-03-07T07:39:15.839078+0000","last_became_active":"2024-03-07T07:39:13.828427+0000","last_became_peered":"2024-03-07T07:39:13.828427+0000","last_unstale":"2024-03-07T07:39:15.839078+0000","last_undegraded":"2024-03-07T07:39:15.839078+0000","last_fullsized":"2024-03-07T07:39:15.839078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:49:21.074014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.604553+0000","last_change":"2024-03-07T07:39:14.612765+0000","last_active":"2024-03-07T07:39:15.604553+0000","last_peered":"2024-03-07T07:39:15.604553+0000","last_clean":"2024-03-07T07:39:15.604553+0000","last_became_active":"2024-03-07T07:39:14.612381+0000","last_became_peered":"2024-03-07T07:39:14.612381+0000","last_unstale":"2024-03-07T07:39:15.604553+0000","last_undegraded":"2024-03-07T07:39:15.604553+0000","last_fullsized":"2024-03-07T07:39:15.604553+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:57:13.912594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.836495+0000","last_change":"2024-03-07T07:39:13.833342+0000","last_active":"2024-03-07T07:39:15.836495+0000","last_peered":"2024-03-07T07:39:15.836495+0000","last_clean":"2024-03-07T07:39:15.836495+0000","last_became_active":"2024-03-07T07:39:13.833013+0000","last_became_peered":"2024-03-07T07:39:13.833013+0000","last_unstale":"2024-03-07T07:39:15.836495+0000","last_undegraded":"2024-03-07T07:39:15.836495+0000","last_fullsized":"2024-03-07T07:39:15.836495+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:39:59.821206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.856881+0000","last_change":"2024-03-07T07:39:14.604744+0000","last_active":"2024-03-07T07:39:15.856881+0000","last_peered":"2024-03-07T07:39:15.856881+0000","last_clean":"2024-03-07T07:39:15.856881+0000","last_became_active":"2024-03-07T07:39:14.604364+0000","last_became_peered":"2024-03-07T07:39:14.604364+0000","last_unstale":"2024-03-07T07:39:15.856881+0000","last_undegraded":"2024-03-07T07:39:15.856881+0000","last_fullsized":"2024-03-07T07:39:15.856881+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:38:57.975024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.126211+0000","last_change":"2024-03-07T07:39:13.836434+0000","last_active":"2024-03-07T07:39:16.126211+0000","last_peered":"2024-03-07T07:39:16.126211+0000","last_clean":"2024-03-07T07:39:16.126211+0000","last_became_active":"2024-03-07T07:39:13.835953+0000","last_became_peered":"2024-03-07T07:39:13.835953+0000","last_unstale":"2024-03-07T07:39:16.126211+0000","last_undegraded":"2024-03-07T07:39:16.126211+0000","last_fullsized":"2024-03-07T07:39:16.126211+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:23:20.941845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.856762+0000","last_change":"2024-03-07T07:39:14.618062+0000","last_active":"2024-03-07T07:39:15.856762+0000","last_peered":"2024-03-07T07:39:15.856762+0000","last_clean":"2024-03-07T07:39:15.856762+0000","last_became_active":"2024-03-07T07:39:14.617715+0000","last_became_peered":"2024-03-07T07:39:14.617715+0000","last_unstale":"2024-03-07T07:39:15.856762+0000","last_undegraded":"2024-03-07T07:39:15.856762+0000","last_fullsized":"2024-03-07T07:39:15.856762+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:51:29.200771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.125393+0000","last_change":"2024-03-07T07:39:14.617522+0000","last_active":"2024-03-07T07:39:16.125393+0000","last_peered":"2024-03-07T07:39:16.125393+0000","last_clean":"2024-03-07T07:39:16.125393+0000","last_became_active":"2024-03-07T07:39:14.616862+0000","last_became_peered":"2024-03-07T07:39:14.616862+0000","last_unstale":"2024-03-07T07:39:16.125393+0000","last_undegraded":"2024-03-07T07:39:16.125393+0000","last_fullsized":"2024-03-07T07:39:16.125393+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:37:58.473662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:17.863277+0000","last_change":"2024-03-07T07:39:13.835090+0000","last_active":"2024-03-07T07:39:17.863277+0000","last_peered":"2024-03-07T07:39:17.863277+0000","last_clean":"2024-03-07T07:39:17.863277+0000","last_became_active":"2024-03-07T07:39:13.834765+0000","last_became_peered":"2024-03-07T07:39:13.834765+0000","last_unstale":"2024-03-07T07:39:17.863277+0000","last_undegraded":"2024-03-07T07:39:17.863277+0000","last_fullsized":"2024-03-07T07:39:17.863277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:50:41.530959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.838765+0000","last_change":"2024-03-07T07:39:14.615224+0000","last_active":"2024-03-07T07:39:15.838765+0000","last_peered":"2024-03-07T07:39:15.838765+0000","last_clean":"2024-03-07T07:39:15.838765+0000","last_became_active":"2024-03-07T07:39:14.614942+0000","last_became_peered":"2024-03-07T07:39:14.614942+0000","last_unstale":"2024-03-07T07:39:15.838765+0000","last_undegraded":"2024-03-07T07:39:15.838765+0000","last_fullsized":"2024-03-07T07:39:15.838765+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:30:48.321950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:18.868397+0000","last_change":"2024-03-07T07:39:13.833245+0000","last_active":"2024-03-07T07:39:18.868397+0000","last_peered":"2024-03-07T07:39:18.868397+0000","last_clean":"2024-03-07T07:39:18.868397+0000","last_became_active":"2024-03-07T07:39:13.832837+0000","last_became_peered":"2024-03-07T07:39:13.832837+0000","last_unstale":"2024-03-07T07:39:18.868397+0000","last_undegraded":"2024-03-07T07:39:18.868397+0000","last_fullsized":"2024-03-07T07:39:18.868397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:47:28.204394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":339,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.125613+0000","last_change":"2024-03-07T07:37:58.585313+0000","last_active":"2024-03-07T07:39:16.125613+0000","last_peered":"2024-03-07T07:39:16.125613+0000","last_clean":"2024-03-07T07:39:16.125613+0000","last_became_active":"2024-03-07T07:37:58.584808+0000","last_became_peered":"2024-03-07T07:37:58.584808+0000","last_unstale":"2024-03-07T07:39:16.125613+0000","last_undegraded":"2024-03-07T07:39:16.125613+0000","last_fullsized":"2024-03-07T07:39:16.125613+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:34:34.612837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:34:34.612837+0000","last_clean_scrub_stamp":"2024-03-07T07:34:34.612837+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:56:56.998601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.603377+0000","last_change":"2024-03-07T07:39:14.615907+0000","last_active":"2024-03-07T07:39:15.603377+0000","last_peered":"2024-03-07T07:39:15.603377+0000","last_clean":"2024-03-07T07:39:15.603377+0000","last_became_active":"2024-03-07T07:39:14.615587+0000","last_became_peered":"2024-03-07T07:39:14.615587+0000","last_unstale":"2024-03-07T07:39:15.603377+0000","last_undegraded":"2024-03-07T07:39:15.603377+0000","last_fullsized":"2024-03-07T07:39:15.603377+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:08:20.363895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:17.861503+0000","last_change":"2024-03-07T07:39:13.832143+0000","last_active":"2024-03-07T07:39:17.861503+0000","last_peered":"2024-03-07T07:39:17.861503+0000","last_clean":"2024-03-07T07:39:17.861503+0000","last_became_active":"2024-03-07T07:39:13.831299+0000","last_became_peered":"2024-03-07T07:39:13.831299+0000","last_unstale":"2024-03-07T07:39:17.861503+0000","last_undegraded":"2024-03-07T07:39:17.861503+0000","last_fullsized":"2024-03-07T07:39:17.861503+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:25:08.965498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.839180+0000","last_change":"2024-03-07T07:39:13.827617+0000","last_active":"2024-03-07T07:39:15.839180+0000","last_peered":"2024-03-07T07:39:15.839180+0000","last_clean":"2024-03-07T07:39:15.839180+0000","last_became_active":"2024-03-07T07:39:13.827188+0000","last_became_peered":"2024-03-07T07:39:13.827188+0000","last_unstale":"2024-03-07T07:39:15.839180+0000","last_undegraded":"2024-03-07T07:39:15.839180+0000","last_fullsized":"2024-03-07T07:39:15.839180+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:35:27.634460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.603768+0000","last_change":"2024-03-07T07:39:14.608307+0000","last_active":"2024-03-07T07:39:15.603768+0000","last_peered":"2024-03-07T07:39:15.603768+0000","last_clean":"2024-03-07T07:39:15.603768+0000","last_became_active":"2024-03-07T07:39:14.607948+0000","last_became_peered":"2024-03-07T07:39:14.607948+0000","last_unstale":"2024-03-07T07:39:15.603768+0000","last_undegraded":"2024-03-07T07:39:15.603768+0000","last_fullsized":"2024-03-07T07:39:15.603768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:36:27.294925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:17.861560+0000","last_change":"2024-03-07T07:39:13.831123+0000","last_active":"2024-03-07T07:39:17.861560+0000","last_peered":"2024-03-07T07:39:17.861560+0000","last_clean":"2024-03-07T07:39:17.861560+0000","last_became_active":"2024-03-07T07:39:13.830801+0000","last_became_peered":"2024-03-07T07:39:13.830801+0000","last_unstale":"2024-03-07T07:39:17.861560+0000","last_undegraded":"2024-03-07T07:39:17.861560+0000","last_fullsized":"2024-03-07T07:39:17.861560+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:19:54.346449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.838544+0000","last_change":"2024-03-07T07:39:14.610510+0000","last_active":"2024-03-07T07:39:15.838544+0000","last_peered":"2024-03-07T07:39:15.838544+0000","last_clean":"2024-03-07T07:39:15.838544+0000","last_became_active":"2024-03-07T07:39:14.609908+0000","last_became_peered":"2024-03-07T07:39:14.609908+0000","last_unstale":"2024-03-07T07:39:15.838544+0000","last_undegraded":"2024-03-07T07:39:15.838544+0000","last_fullsized":"2024-03-07T07:39:15.838544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:51:32.045296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.857044+0000","last_change":"2024-03-07T07:39:13.838605+0000","last_active":"2024-03-07T07:39:16.857044+0000","last_peered":"2024-03-07T07:39:16.857044+0000","last_clean":"2024-03-07T07:39:16.857044+0000","last_became_active":"2024-03-07T07:39:13.838362+0000","last_became_peered":"2024-03-07T07:39:13.838362+0000","last_unstale":"2024-03-07T07:39:16.857044+0000","last_undegraded":"2024-03-07T07:39:16.857044+0000","last_fullsized":"2024-03-07T07:39:16.857044+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:23:47.039496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.125389+0000","last_change":"2024-03-07T07:39:14.615902+0000","last_active":"2024-03-07T07:39:16.125389+0000","last_peered":"2024-03-07T07:39:16.125389+0000","last_clean":"2024-03-07T07:39:16.125389+0000","last_became_active":"2024-03-07T07:39:14.615578+0000","last_became_peered":"2024-03-07T07:39:14.615578+0000","last_unstale":"2024-03-07T07:39:16.125389+0000","last_undegraded":"2024-03-07T07:39:16.125389+0000","last_fullsized":"2024-03-07T07:39:16.125389+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:30:34.371625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-07T07:39:14.597264+0000","last_change":"2024-03-07T07:39:13.834640+0000","last_active":"2024-03-07T07:39:14.597264+0000","last_peered":"2024-03-07T07:39:14.597264+0000","last_clean":"2024-03-07T07:39:14.597264+0000","last_became_active":"2024-03-07T07:39:13.834247+0000","last_became_peered":"2024-03-07T07:39:13.834247+0000","last_unstale":"2024-03-07T07:39:14.597264+0000","last_undegraded":"2024-03-07T07:39:14.597264+0000","last_fullsized":"2024-03-07T07:39:14.597264+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:40:55.487824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.838687+0000","last_change":"2024-03-07T07:39:14.603762+0000","last_active":"2024-03-07T07:39:15.838687+0000","last_peered":"2024-03-07T07:39:15.838687+0000","last_clean":"2024-03-07T07:39:15.838687+0000","last_became_active":"2024-03-07T07:39:14.603444+0000","last_became_peered":"2024-03-07T07:39:14.603444+0000","last_unstale":"2024-03-07T07:39:15.838687+0000","last_undegraded":"2024-03-07T07:39:15.838687+0000","last_fullsized":"2024-03-07T07:39:15.838687+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:18:03.075996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.857806+0000","last_change":"2024-03-07T07:39:13.833864+0000","last_active":"2024-03-07T07:39:16.857806+0000","last_peered":"2024-03-07T07:39:16.857806+0000","last_clean":"2024-03-07T07:39:16.857806+0000","last_became_active":"2024-03-07T07:39:13.833570+0000","last_became_peered":"2024-03-07T07:39:13.833570+0000","last_unstale":"2024-03-07T07:39:16.857806+0000","last_undegraded":"2024-03-07T07:39:16.857806+0000","last_fullsized":"2024-03-07T07:39:16.857806+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:20:16.387319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.604294+0000","last_change":"2024-03-07T07:39:14.610023+0000","last_active":"2024-03-07T07:39:15.604294+0000","last_peered":"2024-03-07T07:39:15.604294+0000","last_clean":"2024-03-07T07:39:15.604294+0000","last_became_active":"2024-03-07T07:39:14.609671+0000","last_became_peered":"2024-03-07T07:39:14.609671+0000","last_unstale":"2024-03-07T07:39:15.604294+0000","last_undegraded":"2024-03-07T07:39:15.604294+0000","last_fullsized":"2024-03-07T07:39:15.604294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:41:48.384795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.125609+0000","last_change":"2024-03-07T07:39:13.841326+0000","last_active":"2024-03-07T07:39:16.125609+0000","last_peered":"2024-03-07T07:39:16.125609+0000","last_clean":"2024-03-07T07:39:16.125609+0000","last_became_active":"2024-03-07T07:39:13.840955+0000","last_became_peered":"2024-03-07T07:39:13.840955+0000","last_unstale":"2024-03-07T07:39:16.125609+0000","last_undegraded":"2024-03-07T07:39:16.125609+0000","last_fullsized":"2024-03-07T07:39:16.125609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:56:25.944653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.838682+0000","last_change":"2024-03-07T07:39:14.617783+0000","last_active":"2024-03-07T07:39:15.838682+0000","last_peered":"2024-03-07T07:39:15.838682+0000","last_clean":"2024-03-07T07:39:15.838682+0000","last_became_active":"2024-03-07T07:39:14.617472+0000","last_became_peered":"2024-03-07T07:39:14.617472+0000","last_unstale":"2024-03-07T07:39:15.838682+0000","last_undegraded":"2024-03-07T07:39:15.838682+0000","last_fullsized":"2024-03-07T07:39:15.838682+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:35:48.618321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.856171+0000","last_change":"2024-03-07T07:39:13.837602+0000","last_active":"2024-03-07T07:39:16.856171+0000","last_peered":"2024-03-07T07:39:16.856171+0000","last_clean":"2024-03-07T07:39:16.856171+0000","last_became_active":"2024-03-07T07:39:13.837275+0000","last_became_peered":"2024-03-07T07:39:13.837275+0000","last_unstale":"2024-03-07T07:39:16.856171+0000","last_undegraded":"2024-03-07T07:39:16.856171+0000","last_fullsized":"2024-03-07T07:39:16.856171+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:40:43.267842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.126518+0000","last_change":"2024-03-07T07:39:14.616702+0000","last_active":"2024-03-07T07:39:16.126518+0000","last_peered":"2024-03-07T07:39:16.126518+0000","last_clean":"2024-03-07T07:39:16.126518+0000","last_became_active":"2024-03-07T07:39:14.614720+0000","last_became_peered":"2024-03-07T07:39:14.614720+0000","last_unstale":"2024-03-07T07:39:16.126518+0000","last_undegraded":"2024-03-07T07:39:16.126518+0000","last_fullsized":"2024-03-07T07:39:16.126518+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:14:45.648231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:17.863211+0000","last_change":"2024-03-07T07:39:13.836641+0000","last_active":"2024-03-07T07:39:17.863211+0000","last_peered":"2024-03-07T07:39:17.863211+0000","last_clean":"2024-03-07T07:39:17.863211+0000","last_became_active":"2024-03-07T07:39:13.834012+0000","last_became_peered":"2024-03-07T07:39:13.834012+0000","last_unstale":"2024-03-07T07:39:17.863211+0000","last_undegraded":"2024-03-07T07:39:17.863211+0000","last_fullsized":"2024-03-07T07:39:17.863211+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:40:49.596816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.126259+0000","last_change":"2024-03-07T07:39:14.611074+0000","last_active":"2024-03-07T07:39:16.126259+0000","last_peered":"2024-03-07T07:39:16.126259+0000","last_clean":"2024-03-07T07:39:16.126259+0000","last_became_active":"2024-03-07T07:39:14.610651+0000","last_became_peered":"2024-03-07T07:39:14.610651+0000","last_unstale":"2024-03-07T07:39:16.126259+0000","last_undegraded":"2024-03-07T07:39:16.126259+0000","last_fullsized":"2024-03-07T07:39:16.126259+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:18:29.684266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.125661+0000","last_change":"2024-03-07T07:39:13.839743+0000","last_active":"2024-03-07T07:39:16.125661+0000","last_peered":"2024-03-07T07:39:16.125661+0000","last_clean":"2024-03-07T07:39:16.125661+0000","last_became_active":"2024-03-07T07:39:13.839079+0000","last_became_peered":"2024-03-07T07:39:13.839079+0000","last_unstale":"2024-03-07T07:39:16.125661+0000","last_undegraded":"2024-03-07T07:39:16.125661+0000","last_fullsized":"2024-03-07T07:39:16.125661+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:11:00.757666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.125867+0000","last_change":"2024-03-07T07:39:14.615941+0000","last_active":"2024-03-07T07:39:16.125867+0000","last_peered":"2024-03-07T07:39:16.125867+0000","last_clean":"2024-03-07T07:39:16.125867+0000","last_became_active":"2024-03-07T07:39:14.615577+0000","last_became_peered":"2024-03-07T07:39:14.615577+0000","last_unstale":"2024-03-07T07:39:16.125867+0000","last_undegraded":"2024-03-07T07:39:16.125867+0000","last_fullsized":"2024-03-07T07:39:16.125867+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:06:05.457297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.603805+0000","last_change":"2024-03-07T07:39:13.842002+0000","last_active":"2024-03-07T07:39:15.603805+0000","last_peered":"2024-03-07T07:39:15.603805+0000","last_clean":"2024-03-07T07:39:15.603805+0000","last_became_active":"2024-03-07T07:39:13.841660+0000","last_became_peered":"2024-03-07T07:39:13.841660+0000","last_unstale":"2024-03-07T07:39:15.603805+0000","last_undegraded":"2024-03-07T07:39:15.603805+0000","last_fullsized":"2024-03-07T07:39:15.603805+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T19:14:58.009793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.126449+0000","last_change":"2024-03-07T07:39:14.611652+0000","last_active":"2024-03-07T07:39:16.126449+0000","last_peered":"2024-03-07T07:39:16.126449+0000","last_clean":"2024-03-07T07:39:16.126449+0000","last_became_active":"2024-03-07T07:39:14.611300+0000","last_became_peered":"2024-03-07T07:39:14.611300+0000","last_unstale":"2024-03-07T07:39:16.126449+0000","last_undegraded":"2024-03-07T07:39:16.126449+0000","last_fullsized":"2024-03-07T07:39:16.126449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:30:44.240560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.604514+0000","last_change":"2024-03-07T07:39:14.617630+0000","last_active":"2024-03-07T07:39:15.604514+0000","last_peered":"2024-03-07T07:39:15.604514+0000","last_clean":"2024-03-07T07:39:15.604514+0000","last_became_active":"2024-03-07T07:39:14.616897+0000","last_became_peered":"2024-03-07T07:39:14.616897+0000","last_unstale":"2024-03-07T07:39:15.604514+0000","last_undegraded":"2024-03-07T07:39:15.604514+0000","last_fullsized":"2024-03-07T07:39:15.604514+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T19:24:35.522835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.125448+0000","last_change":"2024-03-07T07:39:14.616670+0000","last_active":"2024-03-07T07:39:16.125448+0000","last_peered":"2024-03-07T07:39:16.125448+0000","last_clean":"2024-03-07T07:39:16.125448+0000","last_became_active":"2024-03-07T07:39:14.615720+0000","last_became_peered":"2024-03-07T07:39:14.615720+0000","last_unstale":"2024-03-07T07:39:16.125448+0000","last_undegraded":"2024-03-07T07:39:16.125448+0000","last_fullsized":"2024-03-07T07:39:16.125448+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:51:59.533172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.125591+0000","last_change":"2024-03-07T07:39:14.606794+0000","last_active":"2024-03-07T07:39:16.125591+0000","last_peered":"2024-03-07T07:39:16.125591+0000","last_clean":"2024-03-07T07:39:16.125591+0000","last_became_active":"2024-03-07T07:39:14.606395+0000","last_became_peered":"2024-03-07T07:39:14.606395+0000","last_unstale":"2024-03-07T07:39:16.125591+0000","last_undegraded":"2024-03-07T07:39:16.125591+0000","last_fullsized":"2024-03-07T07:39:16.125591+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:54:26.263361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.125250+0000","last_change":"2024-03-07T07:39:14.616627+0000","last_active":"2024-03-07T07:39:16.125250+0000","last_peered":"2024-03-07T07:39:16.125250+0000","last_clean":"2024-03-07T07:39:16.125250+0000","last_became_active":"2024-03-07T07:39:14.615865+0000","last_became_peered":"2024-03-07T07:39:14.615865+0000","last_unstale":"2024-03-07T07:39:16.125250+0000","last_undegraded":"2024-03-07T07:39:16.125250+0000","last_fullsized":"2024-03-07T07:39:16.125250+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:15:46.997786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.603625+0000","last_change":"2024-03-07T07:39:14.616493+0000","last_active":"2024-03-07T07:39:15.603625+0000","last_peered":"2024-03-07T07:39:15.603625+0000","last_clean":"2024-03-07T07:39:15.603625+0000","last_became_active":"2024-03-07T07:39:14.616185+0000","last_became_peered":"2024-03-07T07:39:14.616185+0000","last_unstale":"2024-03-07T07:39:15.603625+0000","last_undegraded":"2024-03-07T07:39:15.603625+0000","last_fullsized":"2024-03-07T07:39:15.603625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:51:57.888472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.604540+0000","last_change":"2024-03-07T07:39:14.614720+0000","last_active":"2024-03-07T07:39:15.604540+0000","last_peered":"2024-03-07T07:39:15.604540+0000","last_clean":"2024-03-07T07:39:15.604540+0000","last_became_active":"2024-03-07T07:39:14.614256+0000","last_became_peered":"2024-03-07T07:39:14.614256+0000","last_unstale":"2024-03-07T07:39:15.604540+0000","last_undegraded":"2024-03-07T07:39:15.604540+0000","last_fullsized":"2024-03-07T07:39:15.604540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:58:24.497664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.126074+0000","last_change":"2024-03-07T07:39:14.611152+0000","last_active":"2024-03-07T07:39:16.126074+0000","last_peered":"2024-03-07T07:39:16.126074+0000","last_clean":"2024-03-07T07:39:16.126074+0000","last_became_active":"2024-03-07T07:39:14.610616+0000","last_became_peered":"2024-03-07T07:39:14.610616+0000","last_unstale":"2024-03-07T07:39:16.126074+0000","last_undegraded":"2024-03-07T07:39:16.126074+0000","last_fullsized":"2024-03-07T07:39:16.126074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:59:36.283163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.838308+0000","last_change":"2024-03-07T07:39:14.610762+0000","last_active":"2024-03-07T07:39:15.838308+0000","last_peered":"2024-03-07T07:39:15.838308+0000","last_clean":"2024-03-07T07:39:15.838308+0000","last_became_active":"2024-03-07T07:39:14.610440+0000","last_became_peered":"2024-03-07T07:39:14.610440+0000","last_unstale":"2024-03-07T07:39:15.838308+0000","last_undegraded":"2024-03-07T07:39:15.838308+0000","last_fullsized":"2024-03-07T07:39:15.838308+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:45:18.046694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:16.125272+0000","last_change":"2024-03-07T07:39:14.610322+0000","last_active":"2024-03-07T07:39:16.125272+0000","last_peered":"2024-03-07T07:39:16.125272+0000","last_clean":"2024-03-07T07:39:16.125272+0000","last_became_active":"2024-03-07T07:39:14.610008+0000","last_became_peered":"2024-03-07T07:39:14.610008+0000","last_unstale":"2024-03-07T07:39:16.125272+0000","last_undegraded":"2024-03-07T07:39:16.125272+0000","last_fullsized":"2024-03-07T07:39:16.125272+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:35:46.071475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.838058+0000","last_change":"2024-03-07T07:39:14.610084+0000","last_active":"2024-03-07T07:39:15.838058+0000","last_peered":"2024-03-07T07:39:15.838058+0000","last_clean":"2024-03-07T07:39:15.838058+0000","last_became_active":"2024-03-07T07:39:14.609085+0000","last_became_peered":"2024-03-07T07:39:14.609085+0000","last_unstale":"2024-03-07T07:39:15.838058+0000","last_undegraded":"2024-03-07T07:39:15.838058+0000","last_fullsized":"2024-03-07T07:39:15.838058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:24:31.206928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-07T07:39:14.613432+0000","last_change":"2024-03-07T07:39:14.611486+0000","last_active":"2024-03-07T07:39:14.613432+0000","last_peered":"2024-03-07T07:39:14.613432+0000","last_clean":"2024-03-07T07:39:14.613432+0000","last_became_active":"2024-03-07T07:39:14.611128+0000","last_became_peered":"2024-03-07T07:39:14.611128+0000","last_unstale":"2024-03-07T07:39:14.613432+0000","last_undegraded":"2024-03-07T07:39:14.613432+0000","last_fullsized":"2024-03-07T07:39:14.613432+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:34:25.068523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.603721+0000","last_change":"2024-03-07T07:39:14.604771+0000","last_active":"2024-03-07T07:39:15.603721+0000","last_peered":"2024-03-07T07:39:15.603721+0000","last_clean":"2024-03-07T07:39:15.603721+0000","last_became_active":"2024-03-07T07:39:14.604494+0000","last_became_peered":"2024-03-07T07:39:14.604494+0000","last_unstale":"2024-03-07T07:39:15.603721+0000","last_undegraded":"2024-03-07T07:39:15.603721+0000","last_fullsized":"2024-03-07T07:39:15.603721+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:42:45.141833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-07T07:39:15.604524+0000","last_change":"2024-03-07T07:39:14.618689+0000","last_active":"2024-03-07T07:39:15.604524+0000","last_peered":"2024-03-07T07:39:15.604524+0000","last_clean":"2024-03-07T07:39:15.604524+0000","last_became_active":"2024-03-07T07:39:14.618330+0000","last_became_peered":"2024-03-07T07:39:14.618330+0000","last_unstale":"2024-03-07T07:39:15.604524+0000","last_undegraded":"2024-03-07T07:39:15.604524+0000","last_fullsized":"2024-03-07T07:39:15.604524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:23:07.431656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282597,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":24896,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":131,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":233472,"data_stored":33761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383694,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":901120,"data_stored":429951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645331,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":1032079,"data_compressed":35940,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808856,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":439339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070493,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":439259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332130,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":1030093,"data_compressed":35940,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495655,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":917504,"data_stored":438611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":431563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":917504,"data_stored":438611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":950272,"data_stored":441483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182455,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":438805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920818,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":439556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757293,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":1030365,"data_compressed":35940,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34299,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34299,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34299,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35940,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35940,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35940,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35940,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35940,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":22864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-07T07:39:21.903 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-07T07:39:21.903 INFO:tasks.cephadm:Setting up client nodes... 2024-03-07T07:39:21.904 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-07T07:39:22.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:21 smithi160 ceph-mon[31963]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:39:22.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:21 smithi160 ceph-mon[31963]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-07T07:39:22.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:21 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/1717047867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:39:22.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:21 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/24870448' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:39:22.180 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:39:22.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:21 smithi178 ceph-mon[36650]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:39:22.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:21 smithi178 ceph-mon[36650]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-07T07:39:22.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:21 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/1717047867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:39:22.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:21 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/24870448' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:39:22.356 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:21 smithi177 ceph-mon[36578]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:39:22.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:21 smithi177 ceph-mon[36578]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-07T07:39:22.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:21 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/1717047867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:39:22.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:21 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/24870448' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:39:23.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:22 smithi160 ceph-mon[31963]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-07T07:39:23.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:22 smithi160 ceph-mon[31963]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-07T07:39:23.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:22 smithi160 ceph-mon[31963]: pgmap v292: 97 pgs: 97 active+clean; 601 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 7.7 KiB/s wr, 19 op/s 2024-03-07T07:39:23.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:22 smithi178 ceph-mon[36650]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-07T07:39:23.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:22 smithi178 ceph-mon[36650]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-07T07:39:23.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:22 smithi178 ceph-mon[36650]: pgmap v292: 97 pgs: 97 active+clean; 601 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 7.7 KiB/s wr, 19 op/s 2024-03-07T07:39:23.356 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:22 smithi177 ceph-mon[36578]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-07T07:39:23.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:22 smithi177 ceph-mon[36578]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-07T07:39:23.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:22 smithi177 ceph-mon[36578]: pgmap v292: 97 pgs: 97 active+clean; 601 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 7.7 KiB/s wr, 19 op/s 2024-03-07T07:39:23.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:23.871+0000 7f8c4e866700 1 -- 172.21.15.160:0/1235990170 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c481021a0 msgr2=0x7f8c481025c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:23.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:23.871+0000 7f8c4e866700 1 --2- 172.21.15.160:0/1235990170 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c481021a0 0x7f8c481025c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8c380099c0 tx=0x7f8c38009cd0 comp rx=0 tx=0).stop 2024-03-07T07:39:23.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:23.871+0000 7f8c4e866700 1 -- 172.21.15.160:0/1235990170 shutdown_connections 2024-03-07T07:39:23.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:23.871+0000 7f8c4e866700 1 --2- 172.21.15.160:0/1235990170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c48103d40 0x7f8c4810a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:23.874 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:23.871+0000 7f8c4e866700 1 --2- 172.21.15.160:0/1235990170 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c481021a0 0x7f8c481025c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:23.874 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:23.871+0000 7f8c4e866700 1 --2- 172.21.15.160:0/1235990170 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c481033a0 0x7f8c48103800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:23.874 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:23.871+0000 7f8c4e866700 1 -- 172.21.15.160:0/1235990170 >> 172.21.15.160:0/1235990170 conn(0x7f8c480fd760 msgr2=0x7f8c480ffb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:23.874 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:23.871+0000 7f8c4e866700 1 -- 172.21.15.160:0/1235990170 shutdown_connections 2024-03-07T07:39:23.874 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:23.871+0000 7f8c4e866700 1 -- 172.21.15.160:0/1235990170 wait complete. 2024-03-07T07:39:23.874 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:23.871+0000 7f8c4e866700 1 Processor -- start 2024-03-07T07:39:23.874 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:23.871+0000 7f8c4e866700 1 -- start start 2024-03-07T07:39:23.874 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:23.872+0000 7f8c4e866700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c481021a0 0x7f8c48075ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:23.874 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:23.872+0000 7f8c4e866700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c481033a0 0x7f8c48074540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:23.874 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:23.872+0000 7f8c4e866700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c48103d40 0x7f8c48074a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:23.874 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:23.872+0000 7f8c4e866700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4810dd80 con 0x7f8c481033a0 2024-03-07T07:39:23.874 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:23.872+0000 7f8c4e866700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4810dc00 con 0x7f8c481021a0 2024-03-07T07:39:23.875 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:23.872+0000 7f8c4e866700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4810df00 con 0x7f8c48103d40 2024-03-07T07:39:23.875 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:23.872+0000 7f8c47fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c481021a0 0x7f8c48075ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:23.875 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:23.872+0000 7f8c47fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c481021a0 0x7f8c48075ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.160:47476/0 (socket says 172.21.15.160:47476) 2024-03-07T07:39:23.875 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:23.872+0000 7f8c47fff700 1 -- 172.21.15.160:0/2312175609 learned_addr learned my addr 172.21.15.160:0/2312175609 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:39:23.875 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:23.872+0000 7f8c4ce03700 1 --2- 172.21.15.160:0/2312175609 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c48103d40 0x7f8c48074a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:23.875 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:23.872+0000 7f8c477fe700 1 --2- 172.21.15.160:0/2312175609 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c481033a0 0x7f8c48074540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:23.875 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:23.872+0000 7f8c47fff700 1 -- 172.21.15.160:0/2312175609 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c48103d40 msgr2=0x7f8c48074a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:23.875 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:23.872+0000 7f8c47fff700 1 --2- 172.21.15.160:0/2312175609 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c48103d40 0x7f8c48074a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:23.875 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:23.872+0000 7f8c47fff700 1 -- 172.21.15.160:0/2312175609 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c481033a0 msgr2=0x7f8c48074540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:23.875 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:23.872+0000 7f8c47fff700 1 --2- 172.21.15.160:0/2312175609 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c481033a0 0x7f8c48074540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:23.875 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:23.872+0000 7f8c47fff700 1 -- 172.21.15.160:0/2312175609 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c48075330 con 0x7f8c481021a0 2024-03-07T07:39:23.876 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:23.872+0000 7f8c47fff700 1 --2- 172.21.15.160:0/2312175609 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c481021a0 0x7f8c48075ef0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8c3800c010 tx=0x7f8c38009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:23.876 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:23.873+0000 7f8c457fa700 1 -- 172.21.15.160:0/2312175609 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c38027070 con 0x7f8c481021a0 2024-03-07T07:39:23.876 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:23.873+0000 7f8c457fa700 1 -- 172.21.15.160:0/2312175609 <== mon.2 v2:172.21.15.177:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8c38004150 con 0x7f8c481021a0 2024-03-07T07:39:23.876 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:23.873+0000 7f8c4e866700 1 -- 172.21.15.160:0/2312175609 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c481a2410 con 0x7f8c481021a0 2024-03-07T07:39:23.876 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:23.873+0000 7f8c457fa700 1 -- 172.21.15.160:0/2312175609 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c38022d40 con 0x7f8c481021a0 2024-03-07T07:39:23.876 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:23.873+0000 7f8c4e866700 1 -- 172.21.15.160:0/2312175609 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c481a28c0 con 0x7f8c481021a0 2024-03-07T07:39:23.877 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:23.874+0000 7f8c4e866700 1 -- 172.21.15.160:0/2312175609 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c48050660 con 0x7f8c481021a0 2024-03-07T07:39:23.877 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:23.875+0000 7f8c457fa700 1 -- 172.21.15.160:0/2312175609 <== mon.2 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 17) v1 ==== 136805+0+0 (secure 0 0 0) 0x7f8c3802a070 con 0x7f8c481021a0 2024-03-07T07:39:23.877 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:23.875+0000 7f8c457fa700 1 --2- 172.21.15.160:0/2312175609 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f8c300a1f50 0x7f8c300a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:23.877 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:23.875+0000 7f8c477fe700 1 --2- 172.21.15.160:0/2312175609 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f8c300a1f50 0x7f8c300a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:23.877 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:23.875+0000 7f8c457fa700 1 -- 172.21.15.160:0/2312175609 <== mon.2 v2:172.21.15.177:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8c380ceca0 con 0x7f8c481021a0 2024-03-07T07:39:23.878 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:23.876+0000 7f8c477fe700 1 --2- 172.21.15.160:0/2312175609 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f8c300a1f50 0x7f8c300a4410 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c007c60 tx=0x7f8c3c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:23.879 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:23.879+0000 7f8c457fa700 1 -- 172.21.15.160:0/2312175609 <== mon.2 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8c38013070 con 0x7f8c481021a0 2024-03-07T07:39:24.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:24.152+0000 7f8c4e866700 1 -- 172.21.15.160:0/2312175609 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f8c4802d9f0 con 0x7f8c481021a0 2024-03-07T07:39:24.157 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:24.157+0000 7f8c457fa700 1 -- 172.21.15.160:0/2312175609 <== mon.2 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7f8c3809c0b0 con 0x7f8c481021a0 2024-03-07T07:39:24.158 INFO:teuthology.orchestra.run.smithi160.stdout:[client.0] 2024-03-07T07:39:24.158 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQAsb+ll1+8vCRAANV0ED25Hn52QnF6IGuq7ew== 2024-03-07T07:39:24.160 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:24.159+0000 7f8c4e866700 1 -- 172.21.15.160:0/2312175609 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f8c300a1f50 msgr2=0x7f8c300a4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:24.160 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:24.159+0000 7f8c4e866700 1 --2- 172.21.15.160:0/2312175609 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f8c300a1f50 0x7f8c300a4410 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c007c60 tx=0x7f8c3c0073d0 comp rx=0 tx=0).stop 2024-03-07T07:39:24.160 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:24.159+0000 7f8c4e866700 1 -- 172.21.15.160:0/2312175609 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c481021a0 msgr2=0x7f8c48075ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:24.160 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:24.159+0000 7f8c4e866700 1 --2- 172.21.15.160:0/2312175609 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c481021a0 0x7f8c48075ef0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8c3800c010 tx=0x7f8c38009510 comp rx=0 tx=0).stop 2024-03-07T07:39:24.160 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:24.160+0000 7f8c4e866700 1 -- 172.21.15.160:0/2312175609 shutdown_connections 2024-03-07T07:39:24.160 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:24.160+0000 7f8c4e866700 1 --2- 172.21.15.160:0/2312175609 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f8c300a1f50 0x7f8c300a4410 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:24.160 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:24.160+0000 7f8c4e866700 1 --2- 172.21.15.160:0/2312175609 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c48103d40 0x7f8c48074a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:24.160 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:24.160+0000 7f8c4e866700 1 --2- 172.21.15.160:0/2312175609 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c481021a0 0x7f8c48075ef0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:24.160 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:24.160+0000 7f8c4e866700 1 --2- 172.21.15.160:0/2312175609 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c481033a0 0x7f8c48074540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:24.161 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:24.160+0000 7f8c4e866700 1 -- 172.21.15.160:0/2312175609 >> 172.21.15.160:0/2312175609 conn(0x7f8c480fd760 msgr2=0x7f8c48106640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:24.161 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:24.160+0000 7f8c4e866700 1 -- 172.21.15.160:0/2312175609 shutdown_connections 2024-03-07T07:39:24.161 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:24.160+0000 7f8c4e866700 1 -- 172.21.15.160:0/2312175609 wait complete. 2024-03-07T07:39:24.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:24 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/2312175609' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-07T07:39:24.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:24 smithi160 ceph-mon[31963]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-07T07:39:24.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:24 smithi160 ceph-mon[31963]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-07T07:39:24.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:24 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/2312175609' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-07T07:39:24.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:24 smithi178 ceph-mon[36650]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-07T07:39:24.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:24 smithi178 ceph-mon[36650]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-07T07:39:24.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:24 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/2312175609' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-07T07:39:24.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:24 smithi177 ceph-mon[36578]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-07T07:39:24.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:24 smithi177 ceph-mon[36578]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-07T07:39:24.800 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-03-07T07:39:24.800 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-07T07:39:24.800 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-07T07:39:24.844 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-07T07:39:24.844 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-07T07:39:24.844 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph mgr dump --format=json 2024-03-07T07:39:25.125 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:39:25.508 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:25 smithi160 ceph-mon[31963]: pgmap v293: 97 pgs: 97 active+clean; 601 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6.5 KiB/s wr, 17 op/s 2024-03-07T07:39:25.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:25 smithi178 ceph-mon[36650]: pgmap v293: 97 pgs: 97 active+clean; 601 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6.5 KiB/s wr, 17 op/s 2024-03-07T07:39:25.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:25 smithi177 ceph-mon[36578]: pgmap v293: 97 pgs: 97 active+clean; 601 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6.5 KiB/s wr, 17 op/s 2024-03-07T07:39:26.819 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:26.810+0000 7f69ff480700 1 -- 172.21.15.160:0/3418003928 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69f806a8f0 msgr2=0x7f69f806acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:26.819 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:26.810+0000 7f69ff480700 1 --2- 172.21.15.160:0/3418003928 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69f806a8f0 0x7f69f806acd0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f69f4009990 tx=0x7f69f4009ca0 comp rx=0 tx=0).stop 2024-03-07T07:39:26.819 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:26.810+0000 7f69ff480700 1 -- 172.21.15.160:0/3418003928 shutdown_connections 2024-03-07T07:39:26.819 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:26.810+0000 7f69ff480700 1 --2- 172.21.15.160:0/3418003928 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69f8107460 0x7f69f810f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:26.819 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:26.810+0000 7f69ff480700 1 --2- 172.21.15.160:0/3418003928 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69f806a8f0 0x7f69f806acd0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:26.819 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:26.810+0000 7f69ff480700 1 --2- 172.21.15.160:0/3418003928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69f806b2a0 0x7f69f8106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:26.819 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:26.810+0000 7f69ff480700 1 -- 172.21.15.160:0/3418003928 >> 172.21.15.160:0/3418003928 conn(0x7f69f80770a0 msgr2=0x7f69f80774b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:26.819 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:26.810+0000 7f69ff480700 1 -- 172.21.15.160:0/3418003928 shutdown_connections 2024-03-07T07:39:26.819 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:26.810+0000 7f69ff480700 1 -- 172.21.15.160:0/3418003928 wait complete. 2024-03-07T07:39:26.819 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:26.810+0000 7f69ff480700 1 Processor -- start 2024-03-07T07:39:26.819 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:26.811+0000 7f69ff480700 1 -- start start 2024-03-07T07:39:26.819 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:26.811+0000 7f69ff480700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69f806a8f0 0x7f69f819e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:26.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:26.811+0000 7f69ff480700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69f806b2a0 0x7f69f819eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:26.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:26.811+0000 7f69ff480700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69f8107460 0x7f69f81986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:26.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:26.811+0000 7f69ff480700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f8112400 con 0x7f69f806a8f0 2024-03-07T07:39:26.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:26.811+0000 7f69ff480700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f8112280 con 0x7f69f8107460 2024-03-07T07:39:26.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:26.811+0000 7f69ff480700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f8112580 con 0x7f69f806b2a0 2024-03-07T07:39:26.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:26.811+0000 7f69fd21c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69f806a8f0 0x7f69f819e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:26.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:26.811+0000 7f69fd21c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69f806a8f0 0x7f69f819e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:49780/0 (socket says 172.21.15.160:49780) 2024-03-07T07:39:26.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:26.811+0000 7f69fd21c700 1 -- 172.21.15.160:0/4268981740 learned_addr learned my addr 172.21.15.160:0/4268981740 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:39:26.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:26.811+0000 7f69fca1b700 1 --2- 172.21.15.160:0/4268981740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69f806b2a0 0x7f69f819eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:26.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:26.811+0000 7f69fda1d700 1 --2- 172.21.15.160:0/4268981740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69f8107460 0x7f69f81986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:26.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:26.811+0000 7f69fd21c700 1 -- 172.21.15.160:0/4268981740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69f806b2a0 msgr2=0x7f69f819eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:26.821 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:26.811+0000 7f69fd21c700 1 --2- 172.21.15.160:0/4268981740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69f806b2a0 0x7f69f819eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:26.821 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:26.811+0000 7f69fd21c700 1 -- 172.21.15.160:0/4268981740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69f8107460 msgr2=0x7f69f81986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:26.821 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:26.811+0000 7f69fd21c700 1 --2- 172.21.15.160:0/4268981740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69f8107460 0x7f69f81986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:26.821 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:26.811+0000 7f69fd21c700 1 -- 172.21.15.160:0/4268981740 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f8198f80 con 0x7f69f806a8f0 2024-03-07T07:39:26.821 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:26.811+0000 7f69fd21c700 1 --2- 172.21.15.160:0/4268981740 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69f806a8f0 0x7f69f819e5d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f69f4000c00 tx=0x7f69f400bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:26.821 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:26.811+0000 7f69ee7fc700 1 -- 172.21.15.160:0/4268981740 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f4027070 con 0x7f69f806a8f0 2024-03-07T07:39:26.821 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:26.811+0000 7f69ff480700 1 -- 172.21.15.160:0/4268981740 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69f8199210 con 0x7f69f806a8f0 2024-03-07T07:39:26.821 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:26.811+0000 7f69ee7fc700 1 -- 172.21.15.160:0/4268981740 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f69f4003d70 con 0x7f69f806a8f0 2024-03-07T07:39:26.821 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:26.811+0000 7f69ee7fc700 1 -- 172.21.15.160:0/4268981740 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f400f460 con 0x7f69f806a8f0 2024-03-07T07:39:26.821 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:26.812+0000 7f69ff480700 1 -- 172.21.15.160:0/4268981740 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69f81a5380 con 0x7f69f806a8f0 2024-03-07T07:39:26.821 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:26.812+0000 7f69ff480700 1 -- 172.21.15.160:0/4268981740 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69f810a690 con 0x7f69f806a8f0 2024-03-07T07:39:26.821 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:26.814+0000 7f69ee7fc700 1 -- 172.21.15.160:0/4268981740 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) v1 ==== 136805+0+0 (secure 0 0 0) 0x7f69f402a070 con 0x7f69f806a8f0 2024-03-07T07:39:26.821 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:26.815+0000 7f69ee7fc700 1 --2- 172.21.15.160:0/4268981740 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f69e40a1ea0 0x7f69e40a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:26.821 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:26.815+0000 7f69fca1b700 1 --2- 172.21.15.160:0/4268981740 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f69e40a1ea0 0x7f69e40a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:26.821 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:26.815+0000 7f69ee7fc700 1 -- 172.21.15.160:0/4268981740 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f69f40ceef0 con 0x7f69f806a8f0 2024-03-07T07:39:26.822 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:26.815+0000 7f69fca1b700 1 --2- 172.21.15.160:0/4268981740 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f69e40a1ea0 0x7f69e40a4360 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f69f8199e60 tx=0x7f69e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:26.822 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:26.819+0000 7f69ee7fc700 1 -- 172.21.15.160:0/4268981740 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f69f4013070 con 0x7f69f806a8f0 2024-03-07T07:39:27.072 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:27.072+0000 7f69ff480700 1 -- 172.21.15.160:0/4268981740 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f69f8199bd0 con 0x7f69f806a8f0 2024-03-07T07:39:27.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:27.077+0000 7f69ee7fc700 1 -- 172.21.15.160:0/4268981740 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+261471 (secure 0 0 0) 0x7f69f409c400 con 0x7f69f806a8f0 2024-03-07T07:39:27.080 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:39:27.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:27.079+0000 7f69ff480700 1 -- 172.21.15.160:0/4268981740 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f69e40a1ea0 msgr2=0x7f69e40a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:27.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:27.079+0000 7f69ff480700 1 --2- 172.21.15.160:0/4268981740 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f69e40a1ea0 0x7f69e40a4360 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f69f8199e60 tx=0x7f69e8009260 comp rx=0 tx=0).stop 2024-03-07T07:39:27.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:27.079+0000 7f69ff480700 1 -- 172.21.15.160:0/4268981740 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69f806a8f0 msgr2=0x7f69f819e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:27.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:27.079+0000 7f69ff480700 1 --2- 172.21.15.160:0/4268981740 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69f806a8f0 0x7f69f819e5d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f69f4000c00 tx=0x7f69f400bab0 comp rx=0 tx=0).stop 2024-03-07T07:39:27.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:27.079+0000 7f69ff480700 1 -- 172.21.15.160:0/4268981740 shutdown_connections 2024-03-07T07:39:27.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:27.079+0000 7f69ff480700 1 --2- 172.21.15.160:0/4268981740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69f8107460 0x7f69f81986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:27.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:27.079+0000 7f69ff480700 1 --2- 172.21.15.160:0/4268981740 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69f806a8f0 0x7f69f819e5d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:27.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:27.079+0000 7f69ff480700 1 --2- 172.21.15.160:0/4268981740 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f69e40a1ea0 0x7f69e40a4360 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:27.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:27.079+0000 7f69ff480700 1 --2- 172.21.15.160:0/4268981740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69f806b2a0 0x7f69f819eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:27.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:27.079+0000 7f69ff480700 1 -- 172.21.15.160:0/4268981740 >> 172.21.15.160:0/4268981740 conn(0x7f69f80770a0 msgr2=0x7f69f8100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:27.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:27.079+0000 7f69ff480700 1 -- 172.21.15.160:0/4268981740 shutdown_connections 2024-03-07T07:39:27.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:27.079+0000 7f69ff480700 1 -- 172.21.15.160:0/4268981740 wait complete. 2024-03-07T07:39:27.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:27 smithi177 ceph-mon[36578]: pgmap v294: 97 pgs: 97 active+clean; 624 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 8.3 KiB/s wr, 18 op/s 2024-03-07T07:39:27.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:27 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/4268981740' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-07T07:39:27.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:27 smithi160 ceph-mon[31963]: pgmap v294: 97 pgs: 97 active+clean; 624 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 8.3 KiB/s wr, 18 op/s 2024-03-07T07:39:27.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:27 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/4268981740' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-07T07:39:27.791 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":419630619},{"type":"v1","addr":"172.21.15.160:6801","nonce":419630619}]},"active_addr":"172.21.15.160:6801/419630619","active_change":"2024-03-07T07:32:08.190949+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24119,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.160:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.160:0","nonce":870342466}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.160:0","nonce":1513978191}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.160:0","nonce":3297345964}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.160:0","nonce":1219726049}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.160:0","nonce":418769708}]}]} 2024-03-07T07:39:27.796 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-07T07:39:27.796 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-07T07:39:27.797 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph osd dump --format=json 2024-03-07T07:39:27.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:27 smithi178 ceph-mon[36650]: pgmap v294: 97 pgs: 97 active+clean; 624 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 8.3 KiB/s wr, 18 op/s 2024-03-07T07:39:27.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:27 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/4268981740' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-07T07:39:28.039 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:39:29.427 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:29 smithi160 ceph-mon[31963]: pgmap v295: 97 pgs: 97 active+clean; 624 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 7.4 KiB/s wr, 16 op/s 2024-03-07T07:39:29.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:29 smithi177 ceph-mon[36578]: pgmap v295: 97 pgs: 97 active+clean; 624 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 7.4 KiB/s wr, 16 op/s 2024-03-07T07:39:29.670 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.668+0000 7f5c54d8f700 1 -- 172.21.15.160:0/2490066020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c501019f0 msgr2=0x7f5c5010ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:29.671 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.668+0000 7f5c54d8f700 1 --2- 172.21.15.160:0/2490066020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c501019f0 0x7f5c5010ded0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5c40009990 tx=0x7f5c40009ca0 comp rx=0 tx=0).stop 2024-03-07T07:39:29.671 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.668+0000 7f5c54d8f700 1 -- 172.21.15.160:0/2490066020 shutdown_connections 2024-03-07T07:39:29.671 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.668+0000 7f5c54d8f700 1 --2- 172.21.15.160:0/2490066020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c501019f0 0x7f5c5010ded0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:29.671 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.668+0000 7f5c54d8f700 1 --2- 172.21.15.160:0/2490066020 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c50107090 0x7f5c50107470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:29.671 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.668+0000 7f5c54d8f700 1 --2- 172.21.15.160:0/2490066020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c50101050 0x7f5c501014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:29.671 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.668+0000 7f5c54d8f700 1 -- 172.21.15.160:0/2490066020 >> 172.21.15.160:0/2490066020 conn(0x7f5c50077f80 msgr2=0x7f5c50078390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:29.671 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.668+0000 7f5c54d8f700 1 -- 172.21.15.160:0/2490066020 shutdown_connections 2024-03-07T07:39:29.671 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.668+0000 7f5c54d8f700 1 -- 172.21.15.160:0/2490066020 wait complete. 2024-03-07T07:39:29.671 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.669+0000 7f5c54d8f700 1 Processor -- start 2024-03-07T07:39:29.671 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.669+0000 7f5c54d8f700 1 -- start start 2024-03-07T07:39:29.671 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.669+0000 7f5c54d8f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c50101050 0x7f5c5019e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:29.671 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.669+0000 7f5c54d8f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c501019f0 0x7f5c5019eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:29.671 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.669+0000 7f5c54d8f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c50107090 0x7f5c50198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:29.671 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.669+0000 7f5c54d8f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c50111310 con 0x7f5c50101050 2024-03-07T07:39:29.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.669+0000 7f5c54d8f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c50111190 con 0x7f5c50107090 2024-03-07T07:39:29.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.669+0000 7f5c54d8f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c50111490 con 0x7f5c501019f0 2024-03-07T07:39:29.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.670+0000 7f5c4e59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c50101050 0x7f5c5019e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:29.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.670+0000 7f5c4dd9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c501019f0 0x7f5c5019eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:29.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.670+0000 7f5c4e59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c50101050 0x7f5c5019e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:55116/0 (socket says 172.21.15.160:55116) 2024-03-07T07:39:29.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.670+0000 7f5c4e59c700 1 -- 172.21.15.160:0/1611749931 learned_addr learned my addr 172.21.15.160:0/1611749931 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:39:29.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.670+0000 7f5c4ed9d700 1 --2- 172.21.15.160:0/1611749931 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c50107090 0x7f5c50198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:29.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.670+0000 7f5c4e59c700 1 -- 172.21.15.160:0/1611749931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c501019f0 msgr2=0x7f5c5019eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:29.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.670+0000 7f5c4e59c700 1 --2- 172.21.15.160:0/1611749931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c501019f0 0x7f5c5019eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:29.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.670+0000 7f5c4e59c700 1 -- 172.21.15.160:0/1611749931 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c50107090 msgr2=0x7f5c50198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:29.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.670+0000 7f5c4e59c700 1 --2- 172.21.15.160:0/1611749931 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c50107090 0x7f5c50198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:29.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.670+0000 7f5c4e59c700 1 -- 172.21.15.160:0/1611749931 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c50199000 con 0x7f5c50101050 2024-03-07T07:39:29.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.670+0000 7f5c4e59c700 1 --2- 172.21.15.160:0/1611749931 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c50101050 0x7f5c5019e650 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5c4800ef10 tx=0x7f5c4800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:29.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.670+0000 7f5c3f7fe700 1 -- 172.21.15.160:0/1611749931 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4800c9c0 con 0x7f5c50101050 2024-03-07T07:39:29.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.670+0000 7f5c3f7fe700 1 -- 172.21.15.160:0/1611749931 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5c48019070 con 0x7f5c50101050 2024-03-07T07:39:29.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.670+0000 7f5c54d8f700 1 -- 172.21.15.160:0/1611749931 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c50199290 con 0x7f5c50101050 2024-03-07T07:39:29.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.670+0000 7f5c3f7fe700 1 -- 172.21.15.160:0/1611749931 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c48007e60 con 0x7f5c50101050 2024-03-07T07:39:29.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.670+0000 7f5c54d8f700 1 -- 172.21.15.160:0/1611749931 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c501a55d0 con 0x7f5c50101050 2024-03-07T07:39:29.674 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.671+0000 7f5c54d8f700 1 -- 172.21.15.160:0/1611749931 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c50102740 con 0x7f5c50101050 2024-03-07T07:39:29.674 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.672+0000 7f5c3f7fe700 1 -- 172.21.15.160:0/1611749931 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) v1 ==== 136805+0+0 (secure 0 0 0) 0x7f5c48012050 con 0x7f5c50101050 2024-03-07T07:39:29.674 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.673+0000 7f5c3f7fe700 1 --2- 172.21.15.160:0/1611749931 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f5c380a1ea0 0x7f5c380a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:29.674 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.673+0000 7f5c4dd9b700 1 --2- 172.21.15.160:0/1611749931 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f5c380a1ea0 0x7f5c380a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:29.674 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.673+0000 7f5c3f7fe700 1 -- 172.21.15.160:0/1611749931 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5c480cdc80 con 0x7f5c50101050 2024-03-07T07:39:29.674 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.673+0000 7f5c4dd9b700 1 --2- 172.21.15.160:0/1611749931 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f5c380a1ea0 0x7f5c380a4360 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5c50199ee0 tx=0x7f5c44009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:29.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.677+0000 7f5c3f7fe700 1 -- 172.21.15.160:0/1611749931 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5c48010040 con 0x7f5c50101050 2024-03-07T07:39:29.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:29 smithi178 ceph-mon[36650]: pgmap v295: 97 pgs: 97 active+clean; 624 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 7.4 KiB/s wr, 16 op/s 2024-03-07T07:39:29.874 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.874+0000 7f5c54d8f700 1 -- 172.21.15.160:0/1611749931 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c50199c50 con 0x7f5c50101050 2024-03-07T07:39:29.875 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.875+0000 7f5c3f7fe700 1 -- 172.21.15.160:0/1611749931 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22242 (secure 0 0 0) 0x7f5c4809b100 con 0x7f5c50101050 2024-03-07T07:39:29.876 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:39:29.876 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":85,"fsid":"826f6582-dc54-11ee-95c6-87774f69a715","created":"2024-03-07T07:31:14.229288+0000","modified":"2024-03-07T07:39:15.600463+0000","last_up_change":"2024-03-07T07:38:20.776225+0000","last_in_change":"2024-03-07T07:38:00.508917+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T07:34:34.529424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T07:39:12.075432+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T07:39:13.367820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bee66c67-fb6e-4717-9685-ca62587d5f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6803","nonce":1368482926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6805","nonce":1368482926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6809","nonce":1368482926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6807","nonce":1368482926}]},"public_addr":"172.21.15.160:6803/1368482926","cluster_addr":"172.21.15.160:6805/1368482926","heartbeat_back_addr":"172.21.15.160:6809/1368482926","heartbeat_front_addr":"172.21.15.160:6807/1368482926","state":["exists","up"]},{"osd":1,"uuid":"49236251-f750-424e-992b-bddf55762556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6801","nonce":2922391884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6803","nonce":2922391884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6807","nonce":2922391884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6805","nonce":2922391884}]},"public_addr":"172.21.15.177:6801/2922391884","cluster_addr":"172.21.15.177:6803/2922391884","heartbeat_back_addr":"172.21.15.177:6807/2922391884","heartbeat_front_addr":"172.21.15.177:6805/2922391884","state":["exists","up"]},{"osd":2,"uuid":"7a432b28-f7ee-43fe-9196-7aaeccfecf77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6801","nonce":1217901551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6803","nonce":1217901551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6807","nonce":1217901551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6805","nonce":1217901551}]},"public_addr":"172.21.15.178:6801/1217901551","cluster_addr":"172.21.15.178:6803/1217901551","heartbeat_back_addr":"172.21.15.178:6807/1217901551","heartbeat_front_addr":"172.21.15.178:6805/1217901551","state":["exists","up"]},{"osd":3,"uuid":"4ad22a20-f1be-4002-a48d-ecd3953e63f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6811","nonce":42543989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6813","nonce":42543989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6817","nonce":42543989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6815","nonce":42543989}]},"public_addr":"172.21.15.160:6811/42543989","cluster_addr":"172.21.15.160:6813/42543989","heartbeat_back_addr":"172.21.15.160:6817/42543989","heartbeat_front_addr":"172.21.15.160:6815/42543989","state":["exists","up"]},{"osd":4,"uuid":"2b65b104-ac82-4ce3-b6c5-11a399103045","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6809","nonce":88178553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6811","nonce":88178553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6815","nonce":88178553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6813","nonce":88178553}]},"public_addr":"172.21.15.177:6809/88178553","cluster_addr":"172.21.15.177:6811/88178553","heartbeat_back_addr":"172.21.15.177:6815/88178553","heartbeat_front_addr":"172.21.15.177:6813/88178553","state":["exists","up"]},{"osd":5,"uuid":"30a1e714-08ff-44e8-9264-9ef164604b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6809","nonce":964732891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6811","nonce":964732891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6815","nonce":964732891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6813","nonce":964732891}]},"public_addr":"172.21.15.178:6809/964732891","cluster_addr":"172.21.15.178:6811/964732891","heartbeat_back_addr":"172.21.15.178:6815/964732891","heartbeat_front_addr":"172.21.15.178:6813/964732891","state":["exists","up"]},{"osd":6,"uuid":"b1e1646e-f8a9-42b8-b62d-9d7b5d6a1ff4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6819","nonce":3251327001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6821","nonce":3251327001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6825","nonce":3251327001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6823","nonce":3251327001}]},"public_addr":"172.21.15.160:6819/3251327001","cluster_addr":"172.21.15.160:6821/3251327001","heartbeat_back_addr":"172.21.15.160:6825/3251327001","heartbeat_front_addr":"172.21.15.160:6823/3251327001","state":["exists","up"]},{"osd":7,"uuid":"e66b400b-9747-4f6d-b05b-db22468e2750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6817","nonce":1175500850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6819","nonce":1175500850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6823","nonce":1175500850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6821","nonce":1175500850}]},"public_addr":"172.21.15.177:6817/1175500850","cluster_addr":"172.21.15.177:6819/1175500850","heartbeat_back_addr":"172.21.15.177:6823/1175500850","heartbeat_front_addr":"172.21.15.177:6821/1175500850","state":["exists","up"]},{"osd":8,"uuid":"3ee0f2c4-fca4-49fc-a98b-07aa0fcba0e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6817","nonce":688015531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6819","nonce":688015531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6823","nonce":688015531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6821","nonce":688015531}]},"public_addr":"172.21.15.178:6817/688015531","cluster_addr":"172.21.15.178:6819/688015531","heartbeat_back_addr":"172.21.15.178:6823/688015531","heartbeat_front_addr":"172.21.15.178:6821/688015531","state":["exists","up"]},{"osd":9,"uuid":"856039d7-f7ee-4dbd-b019-0073e705ce0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6827","nonce":3265797851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6829","nonce":3265797851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6833","nonce":3265797851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6831","nonce":3265797851}]},"public_addr":"172.21.15.160:6827/3265797851","cluster_addr":"172.21.15.160:6829/3265797851","heartbeat_back_addr":"172.21.15.160:6833/3265797851","heartbeat_front_addr":"172.21.15.160:6831/3265797851","state":["exists","up"]},{"osd":10,"uuid":"bdc9899d-d720-4429-ad53-4c388a4dbe75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6825","nonce":2867473412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6827","nonce":2867473412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6831","nonce":2867473412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6829","nonce":2867473412}]},"public_addr":"172.21.15.177:6825/2867473412","cluster_addr":"172.21.15.177:6827/2867473412","heartbeat_back_addr":"172.21.15.177:6831/2867473412","heartbeat_front_addr":"172.21.15.177:6829/2867473412","state":["exists","up"]},{"osd":11,"uuid":"cc9adb54-ef49-47c6-90e3-fc03bc0f5cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6825","nonce":4068724731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6827","nonce":4068724731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6831","nonce":4068724731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6829","nonce":4068724731}]},"public_addr":"172.21.15.178:6825/4068724731","cluster_addr":"172.21.15.178:6827/4068724731","heartbeat_back_addr":"172.21.15.178:6831/4068724731","heartbeat_front_addr":"172.21.15.178:6829/4068724731","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:33:44.141585+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:07.685585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:31.044386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:56.308925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:35:21.582097+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:35:45.983540+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:36:11.530242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:36:36.844681+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:02.303437+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:27.629867+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:53.289495+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:38:18.536169+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/3027450612":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/874709722":"2024-03-08T07:32:08.190837+0000","172.21.15.160:6800/1130675225":"2024-03-08T07:31:41.166858+0000","172.21.15.160:0/3857193980":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/3861968297":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6800/3859174315":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/3013534743":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6801/3859174315":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/1310937667":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6801/1130675225":"2024-03-08T07:31:41.166858+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T07:39:29.878 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.877+0000 7f5c54d8f700 1 -- 172.21.15.160:0/1611749931 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f5c380a1ea0 msgr2=0x7f5c380a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:29.878 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.877+0000 7f5c54d8f700 1 --2- 172.21.15.160:0/1611749931 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f5c380a1ea0 0x7f5c380a4360 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5c50199ee0 tx=0x7f5c44009260 comp rx=0 tx=0).stop 2024-03-07T07:39:29.878 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.877+0000 7f5c54d8f700 1 -- 172.21.15.160:0/1611749931 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c50101050 msgr2=0x7f5c5019e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:29.878 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.877+0000 7f5c54d8f700 1 --2- 172.21.15.160:0/1611749931 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c50101050 0x7f5c5019e650 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5c4800ef10 tx=0x7f5c4800c3a0 comp rx=0 tx=0).stop 2024-03-07T07:39:29.878 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.877+0000 7f5c54d8f700 1 -- 172.21.15.160:0/1611749931 shutdown_connections 2024-03-07T07:39:29.878 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.877+0000 7f5c54d8f700 1 --2- 172.21.15.160:0/1611749931 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c50107090 0x7f5c50198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:29.878 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.877+0000 7f5c54d8f700 1 --2- 172.21.15.160:0/1611749931 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c50101050 0x7f5c5019e650 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:29.878 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.877+0000 7f5c54d8f700 1 --2- 172.21.15.160:0/1611749931 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f5c380a1ea0 0x7f5c380a4360 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:29.879 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.877+0000 7f5c54d8f700 1 --2- 172.21.15.160:0/1611749931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c501019f0 0x7f5c5019eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:29.879 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.877+0000 7f5c54d8f700 1 -- 172.21.15.160:0/1611749931 >> 172.21.15.160:0/1611749931 conn(0x7f5c50077f80 msgr2=0x7f5c50100410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:29.879 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.878+0000 7f5c54d8f700 1 -- 172.21.15.160:0/1611749931 shutdown_connections 2024-03-07T07:39:29.879 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:29.878+0000 7f5c54d8f700 1 -- 172.21.15.160:0/1611749931 wait complete. 2024-03-07T07:39:30.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:30 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/1611749931' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:39:30.658 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:30 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/1611749931' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:39:30.684 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-07T07:39:30.685 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph osd dump --format=json 2024-03-07T07:39:30.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:30 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/1611749931' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:39:30.928 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:39:31.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:31 smithi177 ceph-mon[36578]: pgmap v296: 97 pgs: 97 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.5 KiB/s wr, 19 op/s 2024-03-07T07:39:31.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:31 smithi160 ceph-mon[31963]: pgmap v296: 97 pgs: 97 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.5 KiB/s wr, 19 op/s 2024-03-07T07:39:31.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:31 smithi178 ceph-mon[36650]: pgmap v296: 97 pgs: 97 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.5 KiB/s wr, 19 op/s 2024-03-07T07:39:32.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.637+0000 7f6c8bacb700 1 -- 172.21.15.160:0/1004086378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c84102b60 msgr2=0x7f6c8410efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:32.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.637+0000 7f6c8bacb700 1 --2- 172.21.15.160:0/1004086378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c84102b60 0x7f6c8410efa0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6c8000b0d0 tx=0x7f6c8000b3e0 comp rx=0 tx=0).stop 2024-03-07T07:39:32.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.637+0000 7f6c8bacb700 1 -- 172.21.15.160:0/1004086378 shutdown_connections 2024-03-07T07:39:32.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.637+0000 7f6c8bacb700 1 --2- 172.21.15.160:0/1004086378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c84102b60 0x7f6c8410efa0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:32.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.637+0000 7f6c8bacb700 1 --2- 172.21.15.160:0/1004086378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c84108180 0x7f6c84108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:32.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.637+0000 7f6c8bacb700 1 --2- 172.21.15.160:0/1004086378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6c841021a0 0x7f6c84102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:32.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.637+0000 7f6c8bacb700 1 -- 172.21.15.160:0/1004086378 >> 172.21.15.160:0/1004086378 conn(0x7f6c84079280 msgr2=0x7f6c84079690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:32.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.637+0000 7f6c8bacb700 1 -- 172.21.15.160:0/1004086378 shutdown_connections 2024-03-07T07:39:32.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.638+0000 7f6c8bacb700 1 -- 172.21.15.160:0/1004086378 wait complete. 2024-03-07T07:39:32.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.638+0000 7f6c8bacb700 1 Processor -- start 2024-03-07T07:39:32.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.638+0000 7f6c8bacb700 1 -- start start 2024-03-07T07:39:32.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.639+0000 7f6c8bacb700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c841021a0 0x7f6c8419e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:32.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.639+0000 7f6c8bacb700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6c84102b60 0x7f6c8419e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:32.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.639+0000 7f6c8bacb700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c84108180 0x7f6c84198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:32.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.639+0000 7f6c8bacb700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c84112400 con 0x7f6c841021a0 2024-03-07T07:39:32.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.639+0000 7f6c8bacb700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c84112280 con 0x7f6c84102b60 2024-03-07T07:39:32.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.639+0000 7f6c8bacb700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c84112580 con 0x7f6c84108180 2024-03-07T07:39:32.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.639+0000 7f6c89867700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c841021a0 0x7f6c8419e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:32.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.639+0000 7f6c89867700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c841021a0 0x7f6c8419e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:55134/0 (socket says 172.21.15.160:55134) 2024-03-07T07:39:32.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.639+0000 7f6c89867700 1 -- 172.21.15.160:0/36369247 learned_addr learned my addr 172.21.15.160:0/36369247 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:39:32.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.639+0000 7f6c8a068700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c84108180 0x7f6c84198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:32.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.639+0000 7f6c89066700 1 --2- 172.21.15.160:0/36369247 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6c84102b60 0x7f6c8419e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:32.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.639+0000 7f6c8a068700 1 -- 172.21.15.160:0/36369247 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6c84102b60 msgr2=0x7f6c8419e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:32.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.639+0000 7f6c8a068700 1 --2- 172.21.15.160:0/36369247 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6c84102b60 0x7f6c8419e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:32.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.639+0000 7f6c8a068700 1 -- 172.21.15.160:0/36369247 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c841021a0 msgr2=0x7f6c8419e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:32.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.639+0000 7f6c8a068700 1 --2- 172.21.15.160:0/36369247 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c841021a0 0x7f6c8419e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:32.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.639+0000 7f6c8a068700 1 -- 172.21.15.160:0/36369247 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c84198da0 con 0x7f6c84108180 2024-03-07T07:39:32.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.639+0000 7f6c89867700 1 --2- 172.21.15.160:0/36369247 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c841021a0 0x7f6c8419e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:39:32.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.639+0000 7f6c8a068700 1 --2- 172.21.15.160:0/36369247 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c84108180 0x7f6c84198640 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6c8000b770 tx=0x7f6c80007a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:32.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.639+0000 7f6c7affd700 1 -- 172.21.15.160:0/36369247 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c80021020 con 0x7f6c84108180 2024-03-07T07:39:32.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.639+0000 7f6c8bacb700 1 -- 172.21.15.160:0/36369247 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c84199030 con 0x7f6c84108180 2024-03-07T07:39:32.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.639+0000 7f6c7affd700 1 -- 172.21.15.160:0/36369247 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c80004c50 con 0x7f6c84108180 2024-03-07T07:39:32.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.639+0000 7f6c7affd700 1 -- 172.21.15.160:0/36369247 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c800052b0 con 0x7f6c84108180 2024-03-07T07:39:32.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.639+0000 7f6c8bacb700 1 -- 172.21.15.160:0/36369247 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c841a5300 con 0x7f6c84108180 2024-03-07T07:39:32.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.640+0000 7f6c8bacb700 1 -- 172.21.15.160:0/36369247 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c84068e10 con 0x7f6c84108180 2024-03-07T07:39:32.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.641+0000 7f6c7affd700 1 -- 172.21.15.160:0/36369247 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) v1 ==== 136805+0+0 (secure 0 0 0) 0x7f6c80024020 con 0x7f6c84108180 2024-03-07T07:39:32.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.642+0000 7f6c7affd700 1 --2- 172.21.15.160:0/36369247 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f6c700a1ea0 0x7f6c700a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:32.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.642+0000 7f6c89867700 1 --2- 172.21.15.160:0/36369247 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f6c700a1ea0 0x7f6c700a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:32.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.642+0000 7f6c7affd700 1 -- 172.21.15.160:0/36369247 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6c8000f020 con 0x7f6c84108180 2024-03-07T07:39:32.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.642+0000 7f6c89867700 1 --2- 172.21.15.160:0/36369247 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f6c700a1ea0 0x7f6c700a4360 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c011550 tx=0x7f6c7c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:32.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.646+0000 7f6c7affd700 1 -- 172.21.15.160:0/36369247 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6c8009c180 con 0x7f6c84108180 2024-03-07T07:39:32.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.843+0000 7f6c8bacb700 1 -- 172.21.15.160:0/36369247 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c8402d9f0 con 0x7f6c84108180 2024-03-07T07:39:32.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.845+0000 7f6c7affd700 1 -- 172.21.15.160:0/36369247 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22242 (secure 0 0 0) 0x7f6c8001de20 con 0x7f6c84108180 2024-03-07T07:39:32.846 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:39:32.846 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":85,"fsid":"826f6582-dc54-11ee-95c6-87774f69a715","created":"2024-03-07T07:31:14.229288+0000","modified":"2024-03-07T07:39:15.600463+0000","last_up_change":"2024-03-07T07:38:20.776225+0000","last_in_change":"2024-03-07T07:38:00.508917+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T07:34:34.529424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T07:39:12.075432+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T07:39:13.367820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bee66c67-fb6e-4717-9685-ca62587d5f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6803","nonce":1368482926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6805","nonce":1368482926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6809","nonce":1368482926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6807","nonce":1368482926}]},"public_addr":"172.21.15.160:6803/1368482926","cluster_addr":"172.21.15.160:6805/1368482926","heartbeat_back_addr":"172.21.15.160:6809/1368482926","heartbeat_front_addr":"172.21.15.160:6807/1368482926","state":["exists","up"]},{"osd":1,"uuid":"49236251-f750-424e-992b-bddf55762556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6801","nonce":2922391884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6803","nonce":2922391884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6807","nonce":2922391884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6805","nonce":2922391884}]},"public_addr":"172.21.15.177:6801/2922391884","cluster_addr":"172.21.15.177:6803/2922391884","heartbeat_back_addr":"172.21.15.177:6807/2922391884","heartbeat_front_addr":"172.21.15.177:6805/2922391884","state":["exists","up"]},{"osd":2,"uuid":"7a432b28-f7ee-43fe-9196-7aaeccfecf77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6801","nonce":1217901551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6803","nonce":1217901551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6807","nonce":1217901551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6805","nonce":1217901551}]},"public_addr":"172.21.15.178:6801/1217901551","cluster_addr":"172.21.15.178:6803/1217901551","heartbeat_back_addr":"172.21.15.178:6807/1217901551","heartbeat_front_addr":"172.21.15.178:6805/1217901551","state":["exists","up"]},{"osd":3,"uuid":"4ad22a20-f1be-4002-a48d-ecd3953e63f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6811","nonce":42543989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6813","nonce":42543989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6817","nonce":42543989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6815","nonce":42543989}]},"public_addr":"172.21.15.160:6811/42543989","cluster_addr":"172.21.15.160:6813/42543989","heartbeat_back_addr":"172.21.15.160:6817/42543989","heartbeat_front_addr":"172.21.15.160:6815/42543989","state":["exists","up"]},{"osd":4,"uuid":"2b65b104-ac82-4ce3-b6c5-11a399103045","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6809","nonce":88178553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6811","nonce":88178553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6815","nonce":88178553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6813","nonce":88178553}]},"public_addr":"172.21.15.177:6809/88178553","cluster_addr":"172.21.15.177:6811/88178553","heartbeat_back_addr":"172.21.15.177:6815/88178553","heartbeat_front_addr":"172.21.15.177:6813/88178553","state":["exists","up"]},{"osd":5,"uuid":"30a1e714-08ff-44e8-9264-9ef164604b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6809","nonce":964732891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6811","nonce":964732891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6815","nonce":964732891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6813","nonce":964732891}]},"public_addr":"172.21.15.178:6809/964732891","cluster_addr":"172.21.15.178:6811/964732891","heartbeat_back_addr":"172.21.15.178:6815/964732891","heartbeat_front_addr":"172.21.15.178:6813/964732891","state":["exists","up"]},{"osd":6,"uuid":"b1e1646e-f8a9-42b8-b62d-9d7b5d6a1ff4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6819","nonce":3251327001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6821","nonce":3251327001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6825","nonce":3251327001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6823","nonce":3251327001}]},"public_addr":"172.21.15.160:6819/3251327001","cluster_addr":"172.21.15.160:6821/3251327001","heartbeat_back_addr":"172.21.15.160:6825/3251327001","heartbeat_front_addr":"172.21.15.160:6823/3251327001","state":["exists","up"]},{"osd":7,"uuid":"e66b400b-9747-4f6d-b05b-db22468e2750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6817","nonce":1175500850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6819","nonce":1175500850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6823","nonce":1175500850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6821","nonce":1175500850}]},"public_addr":"172.21.15.177:6817/1175500850","cluster_addr":"172.21.15.177:6819/1175500850","heartbeat_back_addr":"172.21.15.177:6823/1175500850","heartbeat_front_addr":"172.21.15.177:6821/1175500850","state":["exists","up"]},{"osd":8,"uuid":"3ee0f2c4-fca4-49fc-a98b-07aa0fcba0e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6817","nonce":688015531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6819","nonce":688015531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6823","nonce":688015531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6821","nonce":688015531}]},"public_addr":"172.21.15.178:6817/688015531","cluster_addr":"172.21.15.178:6819/688015531","heartbeat_back_addr":"172.21.15.178:6823/688015531","heartbeat_front_addr":"172.21.15.178:6821/688015531","state":["exists","up"]},{"osd":9,"uuid":"856039d7-f7ee-4dbd-b019-0073e705ce0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6827","nonce":3265797851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6829","nonce":3265797851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6833","nonce":3265797851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6831","nonce":3265797851}]},"public_addr":"172.21.15.160:6827/3265797851","cluster_addr":"172.21.15.160:6829/3265797851","heartbeat_back_addr":"172.21.15.160:6833/3265797851","heartbeat_front_addr":"172.21.15.160:6831/3265797851","state":["exists","up"]},{"osd":10,"uuid":"bdc9899d-d720-4429-ad53-4c388a4dbe75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6825","nonce":2867473412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6827","nonce":2867473412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6831","nonce":2867473412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6829","nonce":2867473412}]},"public_addr":"172.21.15.177:6825/2867473412","cluster_addr":"172.21.15.177:6827/2867473412","heartbeat_back_addr":"172.21.15.177:6831/2867473412","heartbeat_front_addr":"172.21.15.177:6829/2867473412","state":["exists","up"]},{"osd":11,"uuid":"cc9adb54-ef49-47c6-90e3-fc03bc0f5cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6825","nonce":4068724731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6827","nonce":4068724731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6831","nonce":4068724731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6829","nonce":4068724731}]},"public_addr":"172.21.15.178:6825/4068724731","cluster_addr":"172.21.15.178:6827/4068724731","heartbeat_back_addr":"172.21.15.178:6831/4068724731","heartbeat_front_addr":"172.21.15.178:6829/4068724731","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:33:44.141585+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:07.685585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:31.044386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:56.308925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:35:21.582097+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:35:45.983540+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:36:11.530242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:36:36.844681+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:02.303437+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:27.629867+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:53.289495+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:38:18.536169+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/3027450612":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/874709722":"2024-03-08T07:32:08.190837+0000","172.21.15.160:6800/1130675225":"2024-03-08T07:31:41.166858+0000","172.21.15.160:0/3857193980":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/3861968297":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6800/3859174315":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/3013534743":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6801/3859174315":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/1310937667":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6801/1130675225":"2024-03-08T07:31:41.166858+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T07:39:32.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.847+0000 7f6c8bacb700 1 -- 172.21.15.160:0/36369247 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f6c700a1ea0 msgr2=0x7f6c700a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:32.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.847+0000 7f6c8bacb700 1 --2- 172.21.15.160:0/36369247 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f6c700a1ea0 0x7f6c700a4360 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c011550 tx=0x7f6c7c009260 comp rx=0 tx=0).stop 2024-03-07T07:39:32.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.847+0000 7f6c8bacb700 1 -- 172.21.15.160:0/36369247 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c84108180 msgr2=0x7f6c84198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:32.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.847+0000 7f6c8bacb700 1 --2- 172.21.15.160:0/36369247 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c84108180 0x7f6c84198640 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6c8000b770 tx=0x7f6c80007a20 comp rx=0 tx=0).stop 2024-03-07T07:39:32.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.847+0000 7f6c8bacb700 1 -- 172.21.15.160:0/36369247 shutdown_connections 2024-03-07T07:39:32.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.847+0000 7f6c8bacb700 1 --2- 172.21.15.160:0/36369247 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f6c700a1ea0 0x7f6c700a4360 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:32.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.847+0000 7f6c8bacb700 1 --2- 172.21.15.160:0/36369247 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c84108180 0x7f6c84198640 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:32.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.847+0000 7f6c8bacb700 1 --2- 172.21.15.160:0/36369247 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c841021a0 0x7f6c8419e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:32.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.847+0000 7f6c8bacb700 1 --2- 172.21.15.160:0/36369247 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6c84102b60 0x7f6c8419e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:32.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.847+0000 7f6c8bacb700 1 -- 172.21.15.160:0/36369247 >> 172.21.15.160:0/36369247 conn(0x7f6c84079280 msgr2=0x7f6c84100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:32.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.847+0000 7f6c8bacb700 1 -- 172.21.15.160:0/36369247 shutdown_connections 2024-03-07T07:39:32.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:32.847+0000 7f6c8bacb700 1 -- 172.21.15.160:0/36369247 wait complete. 2024-03-07T07:39:33.593 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:33 smithi160 ceph-mon[31963]: pgmap v297: 97 pgs: 97 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.4 KiB/s wr, 7 op/s 2024-03-07T07:39:33.593 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:33 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/36369247' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:39:33.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:33 smithi177 ceph-mon[36578]: pgmap v297: 97 pgs: 97 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.4 KiB/s wr, 7 op/s 2024-03-07T07:39:33.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:33 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/36369247' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:39:33.624 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-07T07:39:33.625 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-07T07:39:33.625 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-07T07:39:33.625 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-07T07:39:33.626 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-07T07:39:33.626 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-07T07:39:33.627 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-07T07:39:33.627 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-07T07:39:33.628 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-03-07T07:39:33.628 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-03-07T07:39:33.628 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-03-07T07:39:33.629 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-03-07T07:39:33.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:33 smithi178 ceph-mon[36650]: pgmap v297: 97 pgs: 97 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.4 KiB/s wr, 7 op/s 2024-03-07T07:39:33.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:33 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/36369247' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:39:34.459 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:39:34.461 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:39:34.461 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:39:34.461 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:39:34.463 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:39:34.464 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:39:34.471 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:39:34.471 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:39:34.477 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:39:34.487 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:39:34.494 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:39:35.128 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:39:35.602 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:35 smithi160 ceph-mon[31963]: pgmap v298: 97 pgs: 97 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-03-07T07:39:35.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:35 smithi177 ceph-mon[36578]: pgmap v298: 97 pgs: 97 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-03-07T07:39:35.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:35 smithi178 ceph-mon[36650]: pgmap v298: 97 pgs: 97 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-03-07T07:39:37.427 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:37 smithi160 ceph-mon[31963]: pgmap v299: 97 pgs: 97 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.3 KiB/s wr, 8 op/s 2024-03-07T07:39:37.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:37 smithi177 ceph-mon[36578]: pgmap v299: 97 pgs: 97 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.3 KiB/s wr, 8 op/s 2024-03-07T07:39:37.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:37 smithi178 ceph-mon[36650]: pgmap v299: 97 pgs: 97 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.3 KiB/s wr, 8 op/s 2024-03-07T07:39:39.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:39 smithi177 ceph-mon[36578]: pgmap v300: 97 pgs: 97 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-03-07T07:39:39.647 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:39 smithi160 ceph-mon[31963]: pgmap v300: 97 pgs: 97 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-03-07T07:39:39.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:39 smithi178 ceph-mon[36650]: pgmap v300: 97 pgs: 97 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-03-07T07:39:41.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:41 smithi160 ceph-mon[31963]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-03-07T07:39:41.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:41 smithi177 ceph-mon[36578]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-03-07T07:39:41.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:41 smithi178 ceph-mon[36650]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-03-07T07:39:43.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:43 smithi177 ceph-mon[36578]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 341 B/s wr, 5 op/s 2024-03-07T07:39:43.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:43 smithi160 ceph-mon[31963]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 341 B/s wr, 5 op/s 2024-03-07T07:39:43.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:43 smithi178 ceph-mon[36650]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 341 B/s wr, 5 op/s 2024-03-07T07:39:44.756 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.752+0000 7f34b0890700 1 -- 172.21.15.160:0/1283518810 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34ac106330 msgr2=0x7f34ac10df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:44.756 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.752+0000 7f34b0890700 1 --2- 172.21.15.160:0/1283518810 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34ac106330 0x7f34ac10df00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f34a400b0d0 tx=0x7f34a400b3e0 comp rx=0 tx=0).stop 2024-03-07T07:39:44.757 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.752+0000 7f34b0890700 1 -- 172.21.15.160:0/1283518810 shutdown_connections 2024-03-07T07:39:44.757 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.752+0000 7f34b0890700 1 --2- 172.21.15.160:0/1283518810 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34ac106330 0x7f34ac10df00 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:44.757 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.752+0000 7f34b0890700 1 --2- 172.21.15.160:0/1283518810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34ac1012e0 0x7f34ac1016c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:44.757 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.752+0000 7f34b0890700 1 --2- 172.21.15.160:0/1283518810 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34ac101c90 0x7f34ac105c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:44.757 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.752+0000 7f34b0890700 1 -- 172.21.15.160:0/1283518810 >> 172.21.15.160:0/1283518810 conn(0x7f34ac0770a0 msgr2=0x7f34ac0774b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:44.757 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.752+0000 7f34b0890700 1 -- 172.21.15.160:0/1283518810 shutdown_connections 2024-03-07T07:39:44.757 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.752+0000 7f34b0890700 1 -- 172.21.15.160:0/1283518810 wait complete. 2024-03-07T07:39:44.757 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.753+0000 7f34b0890700 1 Processor -- start 2024-03-07T07:39:44.757 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.753+0000 7f34b0890700 1 -- start start 2024-03-07T07:39:44.757 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.753+0000 7f34b0890700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34ac1012e0 0x7f34ac19d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:44.757 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.753+0000 7f34b0890700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34ac101c90 0x7f34ac19d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:44.757 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.753+0000 7f34b0890700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34ac106330 0x7f34ac1975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:44.758 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.753+0000 7f34b0890700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ac111320 con 0x7f34ac101c90 2024-03-07T07:39:44.758 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.753+0000 7f34b0890700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ac1111a0 con 0x7f34ac1012e0 2024-03-07T07:39:44.758 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.753+0000 7f34b0890700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ac1114a0 con 0x7f34ac106330 2024-03-07T07:39:44.758 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.753+0000 7f34a9d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34ac101c90 0x7f34ac19d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:44.758 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.753+0000 7f34a9d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34ac101c90 0x7f34ac19d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:53810/0 (socket says 172.21.15.160:53810) 2024-03-07T07:39:44.758 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.753+0000 7f34a9d9b700 1 -- 172.21.15.160:0/2899269959 learned_addr learned my addr 172.21.15.160:0/2899269959 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:39:44.758 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.753+0000 7f34aa59c700 1 --2- 172.21.15.160:0/2899269959 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34ac1012e0 0x7f34ac19d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:44.758 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.754+0000 7f34aad9d700 1 --2- 172.21.15.160:0/2899269959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34ac106330 0x7f34ac1975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:44.758 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.754+0000 7f34a9d9b700 1 -- 172.21.15.160:0/2899269959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34ac106330 msgr2=0x7f34ac1975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:44.758 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.754+0000 7f34a9d9b700 1 --2- 172.21.15.160:0/2899269959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34ac106330 0x7f34ac1975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:44.758 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.754+0000 7f34a9d9b700 1 -- 172.21.15.160:0/2899269959 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34ac1012e0 msgr2=0x7f34ac19d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:44.759 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.754+0000 7f34a9d9b700 1 --2- 172.21.15.160:0/2899269959 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34ac1012e0 0x7f34ac19d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:44.759 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.754+0000 7f34a9d9b700 1 -- 172.21.15.160:0/2899269959 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34ac197d20 con 0x7f34ac101c90 2024-03-07T07:39:44.759 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.754+0000 7f34aa59c700 1 --2- 172.21.15.160:0/2899269959 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34ac1012e0 0x7f34ac19d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:39:44.759 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.754+0000 7f34a9d9b700 1 --2- 172.21.15.160:0/2899269959 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34ac101c90 0x7f34ac19d940 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f34a000ea30 tx=0x7f34a000ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:44.759 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.754+0000 7f349b7fe700 1 -- 172.21.15.160:0/2899269959 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a000cbd0 con 0x7f34ac101c90 2024-03-07T07:39:44.759 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.754+0000 7f34b0890700 1 -- 172.21.15.160:0/2899269959 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34ac198010 con 0x7f34ac101c90 2024-03-07T07:39:44.759 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.754+0000 7f349b7fe700 1 -- 172.21.15.160:0/2899269959 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f34a0004500 con 0x7f34ac101c90 2024-03-07T07:39:44.759 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.754+0000 7f349b7fe700 1 -- 172.21.15.160:0/2899269959 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a0003e30 con 0x7f34ac101c90 2024-03-07T07:39:44.759 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.754+0000 7f34b0890700 1 -- 172.21.15.160:0/2899269959 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34ac1a42d0 con 0x7f34ac101c90 2024-03-07T07:39:44.759 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.755+0000 7f34b0890700 1 -- 172.21.15.160:0/2899269959 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f34ac0628e0 con 0x7f34ac101c90 2024-03-07T07:39:44.759 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.756+0000 7f349b7fe700 1 -- 172.21.15.160:0/2899269959 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) v1 ==== 136805+0+0 (secure 0 0 0) 0x7f34a0004000 con 0x7f34ac101c90 2024-03-07T07:39:44.761 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.757+0000 7f349b7fe700 1 --2- 172.21.15.160:0/2899269959 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f34940a1f20 0x7f34940a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:44.761 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.757+0000 7f34aa59c700 1 --2- 172.21.15.160:0/2899269959 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f34940a1f20 0x7f34940a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:44.761 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.757+0000 7f349b7fe700 1 -- 172.21.15.160:0/2899269959 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f34a00cdf40 con 0x7f34ac101c90 2024-03-07T07:39:44.761 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.757+0000 7f349b7fe700 1 --2- 172.21.15.160:0/2899269959 >> [v2:172.21.15.178:6800/1217901551,v1:172.21.15.178:6801/1217901551] conn(0x7f34940ab700 0x7f34940adb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:44.761 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.757+0000 7f349b7fe700 1 -- 172.21.15.160:0/2899269959 --> [v2:172.21.15.178:6800/1217901551,v1:172.21.15.178:6801/1217901551] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f34940ae1d0 con 0x7f34940ab700 2024-03-07T07:39:44.761 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.757+0000 7f349b7fe700 1 -- 172.21.15.160:0/2899269959 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34a00ce420 con 0x7f34ac101c90 2024-03-07T07:39:44.761 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.757+0000 7f34aa59c700 1 --2- 172.21.15.160:0/2899269959 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f34940a1f20 0x7f34940a43e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f349c009680 tx=0x7f349c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:44.761 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.757+0000 7f34aad9d700 1 --2- 172.21.15.160:0/2899269959 >> [v2:172.21.15.178:6800/1217901551,v1:172.21.15.178:6801/1217901551] conn(0x7f34940ab700 0x7f34940adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:44.761 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.758+0000 7f34aad9d700 1 --2- 172.21.15.160:0/2899269959 >> [v2:172.21.15.178:6800/1217901551,v1:172.21.15.178:6801/1217901551] conn(0x7f34940ab700 0x7f34940adb20 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:44.761 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.758+0000 7f349b7fe700 1 -- 172.21.15.160:0/2899269959 <== osd.2 v2:172.21.15.178:6800/1217901551 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f34940ae1d0 con 0x7f34940ab700 2024-03-07T07:39:44.775 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.774+0000 7f34b0890700 1 -- 172.21.15.160:0/2899269959 --> [v2:172.21.15.178:6800/1217901551,v1:172.21.15.178:6801/1217901551] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f34ac062ce0 con 0x7f34940ab700 2024-03-07T07:39:44.775 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.775+0000 7f349b7fe700 1 -- 172.21.15.160:0/2899269959 <== osd.2 v2:172.21.15.178:6800/1217901551 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f34ac062ce0 con 0x7f34940ab700 2024-03-07T07:39:44.777 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.775+0000 7f34b0890700 1 -- 172.21.15.160:0/2899269959 >> [v2:172.21.15.178:6800/1217901551,v1:172.21.15.178:6801/1217901551] conn(0x7f34940ab700 msgr2=0x7f34940adb20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:44.777 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.775+0000 7f34b0890700 1 --2- 172.21.15.160:0/2899269959 >> [v2:172.21.15.178:6800/1217901551,v1:172.21.15.178:6801/1217901551] conn(0x7f34940ab700 0x7f34940adb20 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:44.777 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.775+0000 7f34b0890700 1 -- 172.21.15.160:0/2899269959 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f34940a1f20 msgr2=0x7f34940a43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:44.777 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.775+0000 7f34b0890700 1 --2- 172.21.15.160:0/2899269959 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f34940a1f20 0x7f34940a43e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f349c009680 tx=0x7f349c009260 comp rx=0 tx=0).stop 2024-03-07T07:39:44.778 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.775+0000 7f34b0890700 1 -- 172.21.15.160:0/2899269959 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34ac101c90 msgr2=0x7f34ac19d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:44.778 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.775+0000 7f34b0890700 1 --2- 172.21.15.160:0/2899269959 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34ac101c90 0x7f34ac19d940 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f34a000ea30 tx=0x7f34a000ed40 comp rx=0 tx=0).stop 2024-03-07T07:39:44.778 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.775+0000 7f34aad9d700 1 -- 172.21.15.160:0/2899269959 reap_dead start 2024-03-07T07:39:44.778 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.775+0000 7f34b0890700 1 -- 172.21.15.160:0/2899269959 shutdown_connections 2024-03-07T07:39:44.778 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.775+0000 7f34b0890700 1 -- 172.21.15.160:0/2899269959 >> 172.21.15.160:0/2899269959 conn(0x7f34ac0770a0 msgr2=0x7f34ac0ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:44.778 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.775+0000 7f34b0890700 1 -- 172.21.15.160:0/2899269959 shutdown_connections 2024-03-07T07:39:44.778 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.775+0000 7f34b0890700 1 -- 172.21.15.160:0/2899269959 wait complete. 2024-03-07T07:39:44.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.952+0000 7f40996d4700 1 -- 172.21.15.160:0/2412022241 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4094108180 msgr2=0x7f4094108560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:44.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.952+0000 7f40996d4700 1 --2- 172.21.15.160:0/2412022241 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4094108180 0x7f4094108560 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4084009fc0 tx=0x7f408400b330 comp rx=0 tx=0).stop 2024-03-07T07:39:44.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.952+0000 7f40996d4700 1 -- 172.21.15.160:0/2412022241 shutdown_connections 2024-03-07T07:39:44.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.952+0000 7f40996d4700 1 --2- 172.21.15.160:0/2412022241 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f409406b260 0x7f409410efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:44.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.952+0000 7f40996d4700 1 --2- 172.21.15.160:0/2412022241 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4094108180 0x7f4094108560 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:44.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.952+0000 7f40996d4700 1 --2- 172.21.15.160:0/2412022241 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f409406a8a0 0x7f409406ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:44.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.952+0000 7f40996d4700 1 -- 172.21.15.160:0/2412022241 >> 172.21.15.160:0/2412022241 conn(0x7f4094078060 msgr2=0x7f4094078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:44.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.952+0000 7f40996d4700 1 -- 172.21.15.160:0/2412022241 shutdown_connections 2024-03-07T07:39:44.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.952+0000 7f40996d4700 1 -- 172.21.15.160:0/2412022241 wait complete. 2024-03-07T07:39:44.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.952+0000 7f40996d4700 1 Processor -- start 2024-03-07T07:39:44.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.953+0000 7f40996d4700 1 -- start start 2024-03-07T07:39:44.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.953+0000 7f40996d4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f409406a8a0 0x7f4094101100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:44.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.953+0000 7f40996d4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f409406b260 0x7f4094101640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:44.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.953+0000 7f40996d4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4094108180 0x7f4094101b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:44.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.953+0000 7f40996d4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4094112400 con 0x7f4094108180 2024-03-07T07:39:44.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.953+0000 7f40996d4700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4094112280 con 0x7f409406a8a0 2024-03-07T07:39:44.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.953+0000 7f40996d4700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4094112580 con 0x7f409406b260 2024-03-07T07:39:44.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.953+0000 7f40937fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4094108180 0x7f4094101b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:44.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.953+0000 7f40927fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f409406b260 0x7f4094101640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:44.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.953+0000 7f40937fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4094108180 0x7f4094101b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:53828/0 (socket says 172.21.15.160:53828) 2024-03-07T07:39:44.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.953+0000 7f40937fe700 1 -- 172.21.15.160:0/2235917974 learned_addr learned my addr 172.21.15.160:0/2235917974 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:39:44.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.953+0000 7f4092ffd700 1 --2- 172.21.15.160:0/2235917974 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f409406a8a0 0x7f4094101100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:44.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.953+0000 7f4092ffd700 1 -- 172.21.15.160:0/2235917974 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f409406b260 msgr2=0x7f4094101640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:44.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.953+0000 7f4092ffd700 1 --2- 172.21.15.160:0/2235917974 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f409406b260 0x7f4094101640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:44.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.953+0000 7f4092ffd700 1 -- 172.21.15.160:0/2235917974 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4094108180 msgr2=0x7f4094101b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:44.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.953+0000 7f4092ffd700 1 --2- 172.21.15.160:0/2235917974 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4094108180 0x7f4094101b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:44.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.953+0000 7f4092ffd700 1 -- 172.21.15.160:0/2235917974 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40941a6df0 con 0x7f409406a8a0 2024-03-07T07:39:44.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.953+0000 7f40937fe700 1 --2- 172.21.15.160:0/2235917974 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4094108180 0x7f4094101b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:39:44.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.953+0000 7f4092ffd700 1 --2- 172.21.15.160:0/2235917974 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f409406a8a0 0x7f4094101100 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f408400b6c0 tx=0x7f408400cfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:44.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.954+0000 7f407bfff700 1 -- 172.21.15.160:0/2235917974 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4084021070 con 0x7f409406a8a0 2024-03-07T07:39:44.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.954+0000 7f40996d4700 1 -- 172.21.15.160:0/2235917974 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40941a7080 con 0x7f409406a8a0 2024-03-07T07:39:44.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.954+0000 7f407bfff700 1 -- 172.21.15.160:0/2235917974 <== mon.2 v2:172.21.15.177:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4084007ac0 con 0x7f409406a8a0 2024-03-07T07:39:44.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.954+0000 7f407bfff700 1 -- 172.21.15.160:0/2235917974 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4084005040 con 0x7f409406a8a0 2024-03-07T07:39:44.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.954+0000 7f40996d4700 1 -- 172.21.15.160:0/2235917974 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40941a7580 con 0x7f409406a8a0 2024-03-07T07:39:44.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.954+0000 7f40996d4700 1 -- 172.21.15.160:0/2235917974 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4094062860 con 0x7f409406a8a0 2024-03-07T07:39:44.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.956+0000 7f407bfff700 1 -- 172.21.15.160:0/2235917974 <== mon.2 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 17) v1 ==== 136805+0+0 (secure 0 0 0) 0x7f4084024070 con 0x7f409406a8a0 2024-03-07T07:39:44.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.956+0000 7f407bfff700 1 --2- 172.21.15.160:0/2235917974 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f407c0a1ea0 0x7f407c0a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:44.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.956+0000 7f40927fc700 1 --2- 172.21.15.160:0/2235917974 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f407c0a1ea0 0x7f407c0a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:44.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.956+0000 7f407bfff700 1 -- 172.21.15.160:0/2235917974 <== mon.2 v2:172.21.15.177:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f408400f020 con 0x7f409406a8a0 2024-03-07T07:39:44.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.956+0000 7f407bfff700 1 --2- 172.21.15.160:0/2235917974 >> [v2:172.21.15.178:6816/688015531,v1:172.21.15.178:6817/688015531] conn(0x7f407c0ab680 0x7f407c0adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:44.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.956+0000 7f407bfff700 1 -- 172.21.15.160:0/2235917974 --> [v2:172.21.15.178:6816/688015531,v1:172.21.15.178:6817/688015531] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f408400aea0 con 0x7f407c0ab680 2024-03-07T07:39:44.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.956+0000 7f407bfff700 1 -- 172.21.15.160:0/2235917974 <== mon.2 v2:172.21.15.177:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40840cf4c0 con 0x7f409406a8a0 2024-03-07T07:39:44.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.957+0000 7f40927fc700 1 --2- 172.21.15.160:0/2235917974 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f407c0a1ea0 0x7f407c0a4360 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4088009680 tx=0x7f4088009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:44.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.957+0000 7f40937fe700 1 --2- 172.21.15.160:0/2235917974 >> [v2:172.21.15.178:6816/688015531,v1:172.21.15.178:6817/688015531] conn(0x7f407c0ab680 0x7f407c0adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:44.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.957+0000 7f40937fe700 1 --2- 172.21.15.160:0/2235917974 >> [v2:172.21.15.178:6816/688015531,v1:172.21.15.178:6817/688015531] conn(0x7f407c0ab680 0x7f407c0adaa0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:44.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.958+0000 7f407bfff700 1 -- 172.21.15.160:0/2235917974 <== osd.8 v2:172.21.15.178:6816/688015531 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f408400aea0 con 0x7f407c0ab680 2024-03-07T07:39:44.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.974+0000 7f40996d4700 1 -- 172.21.15.160:0/2235917974 --> [v2:172.21.15.178:6816/688015531,v1:172.21.15.178:6817/688015531] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4094062ce0 con 0x7f407c0ab680 2024-03-07T07:39:44.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.974+0000 7f407bfff700 1 -- 172.21.15.160:0/2235917974 <== osd.8 v2:172.21.15.178:6816/688015531 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f4094062ce0 con 0x7f407c0ab680 2024-03-07T07:39:44.975 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.974+0000 7f40996d4700 1 -- 172.21.15.160:0/2235917974 >> [v2:172.21.15.178:6816/688015531,v1:172.21.15.178:6817/688015531] conn(0x7f407c0ab680 msgr2=0x7f407c0adaa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:44.975 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.974+0000 7f40996d4700 1 --2- 172.21.15.160:0/2235917974 >> [v2:172.21.15.178:6816/688015531,v1:172.21.15.178:6817/688015531] conn(0x7f407c0ab680 0x7f407c0adaa0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:44.975 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.974+0000 7f40996d4700 1 -- 172.21.15.160:0/2235917974 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f407c0a1ea0 msgr2=0x7f407c0a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:44.976 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.974+0000 7f40996d4700 1 --2- 172.21.15.160:0/2235917974 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f407c0a1ea0 0x7f407c0a4360 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4088009680 tx=0x7f4088009260 comp rx=0 tx=0).stop 2024-03-07T07:39:44.976 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.974+0000 7f40996d4700 1 -- 172.21.15.160:0/2235917974 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f409406a8a0 msgr2=0x7f4094101100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:44.976 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.974+0000 7f40996d4700 1 --2- 172.21.15.160:0/2235917974 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f409406a8a0 0x7f4094101100 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f408400b6c0 tx=0x7f408400cfa0 comp rx=0 tx=0).stop 2024-03-07T07:39:44.976 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.974+0000 7f40937fe700 1 -- 172.21.15.160:0/2235917974 reap_dead start 2024-03-07T07:39:44.976 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.975+0000 7f40996d4700 1 -- 172.21.15.160:0/2235917974 shutdown_connections 2024-03-07T07:39:44.976 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.975+0000 7f40996d4700 1 -- 172.21.15.160:0/2235917974 >> 172.21.15.160:0/2235917974 conn(0x7f4094078060 msgr2=0x7f40940ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:44.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.975+0000 7f40996d4700 1 -- 172.21.15.160:0/2235917974 shutdown_connections 2024-03-07T07:39:44.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:44.975+0000 7f40996d4700 1 -- 172.21.15.160:0/2235917974 wait complete. 2024-03-07T07:39:45.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:45 smithi177 ceph-mon[36578]: pgmap v303: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 B/s wr, 6 op/s 2024-03-07T07:39:45.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:45 smithi160 ceph-mon[31963]: pgmap v303: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 B/s wr, 6 op/s 2024-03-07T07:39:45.765 INFO:teuthology.orchestra.run.smithi160.stdout:77309411393 2024-03-07T07:39:45.766 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-07T07:39:45.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:45 smithi178 ceph-mon[36650]: pgmap v303: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 B/s wr, 6 op/s 2024-03-07T07:39:47.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:47 smithi177 ceph-mon[36578]: pgmap v304: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-03-07T07:39:47.610 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:47 smithi160 ceph-mon[31963]: pgmap v304: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-03-07T07:39:47.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:47 smithi178 ceph-mon[36650]: pgmap v304: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-03-07T07:39:48.249 INFO:teuthology.orchestra.run.smithi160.stdout:253403070499 2024-03-07T07:39:48.250 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-03-07T07:39:49.427 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:49 smithi160 ceph-mon[31963]: pgmap v305: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-03-07T07:39:49.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:49 smithi177 ceph-mon[36578]: pgmap v305: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-03-07T07:39:49.607 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:49 smithi178 ceph-mon[36650]: pgmap v305: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-03-07T07:39:50.307 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.166+0000 7f7ad8fe4700 1 -- 172.21.15.160:0/1699529674 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ad4106330 msgr2=0x7f7ad410df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:50.308 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.166+0000 7f7ad8fe4700 1 --2- 172.21.15.160:0/1699529674 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ad4106330 0x7f7ad410df00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7acc00b0d0 tx=0x7f7acc00b3e0 comp rx=0 tx=0).stop 2024-03-07T07:39:50.308 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.166+0000 7f7ad8fe4700 1 -- 172.21.15.160:0/1699529674 shutdown_connections 2024-03-07T07:39:50.308 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.166+0000 7f7ad8fe4700 1 --2- 172.21.15.160:0/1699529674 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ad4106330 0x7f7ad410df00 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:50.308 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.166+0000 7f7ad8fe4700 1 --2- 172.21.15.160:0/1699529674 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ad41012e0 0x7f7ad41016c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:50.308 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.166+0000 7f7ad8fe4700 1 --2- 172.21.15.160:0/1699529674 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ad4101c90 0x7f7ad4105c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:50.308 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.166+0000 7f7ad8fe4700 1 -- 172.21.15.160:0/1699529674 >> 172.21.15.160:0/1699529674 conn(0x7f7ad40770a0 msgr2=0x7f7ad40774b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:50.308 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.166+0000 7f7ad8fe4700 1 -- 172.21.15.160:0/1699529674 shutdown_connections 2024-03-07T07:39:50.308 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.166+0000 7f7ad8fe4700 1 -- 172.21.15.160:0/1699529674 wait complete. 2024-03-07T07:39:50.308 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.167+0000 7f7ad8fe4700 1 Processor -- start 2024-03-07T07:39:50.308 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.167+0000 7f7ad8fe4700 1 -- start start 2024-03-07T07:39:50.308 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.167+0000 7f7ad8fe4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ad41012e0 0x7f7ad419f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:50.308 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.167+0000 7f7ad8fe4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ad4101c90 0x7f7ad419f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:50.309 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.168+0000 7f7ad8fe4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ad4106330 0x7f7ad41a3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:50.309 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.168+0000 7f7ad8fe4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad4111320 con 0x7f7ad4101c90 2024-03-07T07:39:50.309 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.168+0000 7f7ad8fe4700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad41111a0 con 0x7f7ad41012e0 2024-03-07T07:39:50.309 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.168+0000 7f7ad8fe4700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad41114a0 con 0x7f7ad4106330 2024-03-07T07:39:50.309 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.168+0000 7f7ad1d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ad4101c90 0x7f7ad419f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:50.309 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.168+0000 7f7ad259c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ad41012e0 0x7f7ad419f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:50.309 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.168+0000 7f7ad259c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ad41012e0 0x7f7ad419f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.160:43156/0 (socket says 172.21.15.160:43156) 2024-03-07T07:39:50.309 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.168+0000 7f7ad1d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ad4101c90 0x7f7ad419f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:38720/0 (socket says 172.21.15.160:38720) 2024-03-07T07:39:50.309 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.168+0000 7f7ad259c700 1 -- 172.21.15.160:0/3416928322 learned_addr learned my addr 172.21.15.160:0/3416928322 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:39:50.309 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.168+0000 7f7ad2d9d700 1 --2- 172.21.15.160:0/3416928322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ad4106330 0x7f7ad41a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:50.309 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.168+0000 7f7ad259c700 1 -- 172.21.15.160:0/3416928322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ad4106330 msgr2=0x7f7ad41a3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:50.309 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.168+0000 7f7ad259c700 1 --2- 172.21.15.160:0/3416928322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ad4106330 0x7f7ad41a3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:50.309 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.168+0000 7f7ad259c700 1 -- 172.21.15.160:0/3416928322 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ad4101c90 msgr2=0x7f7ad419f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:50.310 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.168+0000 7f7ad259c700 1 --2- 172.21.15.160:0/3416928322 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ad4101c90 0x7f7ad419f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:50.310 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.168+0000 7f7ad259c700 1 -- 172.21.15.160:0/3416928322 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad41a3e70 con 0x7f7ad41012e0 2024-03-07T07:39:50.310 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.168+0000 7f7ad259c700 1 --2- 172.21.15.160:0/3416928322 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ad41012e0 0x7f7ad419f1d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7ac400e9a0 tx=0x7f7ac400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:50.310 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.168+0000 7f7ac37fe700 1 -- 172.21.15.160:0/3416928322 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac400ca10 con 0x7f7ad41012e0 2024-03-07T07:39:50.310 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.168+0000 7f7ac37fe700 1 -- 172.21.15.160:0/3416928322 <== mon.2 v2:172.21.15.177:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7ac400cbb0 con 0x7f7ad41012e0 2024-03-07T07:39:50.310 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.168+0000 7f7ad8fe4700 1 -- 172.21.15.160:0/3416928322 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ad41a4160 con 0x7f7ad41012e0 2024-03-07T07:39:50.310 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.168+0000 7f7ac37fe700 1 -- 172.21.15.160:0/3416928322 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac4018680 con 0x7f7ad41012e0 2024-03-07T07:39:50.310 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.168+0000 7f7ad8fe4700 1 -- 172.21.15.160:0/3416928322 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ad41ab9e0 con 0x7f7ad41012e0 2024-03-07T07:39:50.310 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.169+0000 7f7ad8fe4700 1 -- 172.21.15.160:0/3416928322 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7ad40628e0 con 0x7f7ad41012e0 2024-03-07T07:39:50.310 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.171+0000 7f7ac37fe700 1 -- 172.21.15.160:0/3416928322 <== mon.2 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 17) v1 ==== 136805+0+0 (secure 0 0 0) 0x7f7ac4010430 con 0x7f7ad41012e0 2024-03-07T07:39:50.310 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.171+0000 7f7ac37fe700 1 --2- 172.21.15.160:0/3416928322 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f7abc0a1ea0 0x7f7abc0a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:50.310 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.171+0000 7f7ad1d9b700 1 --2- 172.21.15.160:0/3416928322 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f7abc0a1ea0 0x7f7abc0a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:50.310 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.171+0000 7f7ac37fe700 1 -- 172.21.15.160:0/3416928322 <== mon.2 v2:172.21.15.177:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f7ac40ce8d0 con 0x7f7ad41012e0 2024-03-07T07:39:50.310 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.171+0000 7f7ac37fe700 1 --2- 172.21.15.160:0/3416928322 >> [v2:172.21.15.177:6808/88178553,v1:172.21.15.177:6809/88178553] conn(0x7f7abc0ab680 0x7f7abc0adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:50.310 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.171+0000 7f7ac37fe700 1 -- 172.21.15.160:0/3416928322 --> [v2:172.21.15.177:6808/88178553,v1:172.21.15.177:6809/88178553] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7ac401bec0 con 0x7f7abc0ab680 2024-03-07T07:39:50.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.171+0000 7f7ac37fe700 1 -- 172.21.15.160:0/3416928322 <== mon.2 v2:172.21.15.177:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ac40cec80 con 0x7f7ad41012e0 2024-03-07T07:39:50.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.171+0000 7f7ad1d9b700 1 --2- 172.21.15.160:0/3416928322 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f7abc0a1ea0 0x7f7abc0a4360 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8009680 tx=0x7f7ac8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:50.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.172+0000 7f7ad2d9d700 1 --2- 172.21.15.160:0/3416928322 >> [v2:172.21.15.177:6808/88178553,v1:172.21.15.177:6809/88178553] conn(0x7f7abc0ab680 0x7f7abc0adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:50.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.172+0000 7f7ad2d9d700 1 --2- 172.21.15.160:0/3416928322 >> [v2:172.21.15.177:6808/88178553,v1:172.21.15.177:6809/88178553] conn(0x7f7abc0ab680 0x7f7abc0adaa0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:50.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.173+0000 7f7ac37fe700 1 -- 172.21.15.160:0/3416928322 <== osd.4 v2:172.21.15.177:6808/88178553 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f7ac401bec0 con 0x7f7abc0ab680 2024-03-07T07:39:50.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.188+0000 7f7ad8fe4700 1 -- 172.21.15.160:0/3416928322 --> [v2:172.21.15.177:6808/88178553,v1:172.21.15.177:6809/88178553] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7ad4062d60 con 0x7f7abc0ab680 2024-03-07T07:39:50.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.189+0000 7f7ac37fe700 1 -- 172.21.15.160:0/3416928322 <== osd.4 v2:172.21.15.177:6808/88178553 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f7ad4062d60 con 0x7f7abc0ab680 2024-03-07T07:39:50.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.189+0000 7f7ad8fe4700 1 -- 172.21.15.160:0/3416928322 >> [v2:172.21.15.177:6808/88178553,v1:172.21.15.177:6809/88178553] conn(0x7f7abc0ab680 msgr2=0x7f7abc0adaa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:50.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.189+0000 7f7ad8fe4700 1 --2- 172.21.15.160:0/3416928322 >> [v2:172.21.15.177:6808/88178553,v1:172.21.15.177:6809/88178553] conn(0x7f7abc0ab680 0x7f7abc0adaa0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:50.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.189+0000 7f7ad8fe4700 1 -- 172.21.15.160:0/3416928322 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f7abc0a1ea0 msgr2=0x7f7abc0a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:50.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.189+0000 7f7ad8fe4700 1 --2- 172.21.15.160:0/3416928322 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f7abc0a1ea0 0x7f7abc0a4360 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8009680 tx=0x7f7ac8009260 comp rx=0 tx=0).stop 2024-03-07T07:39:50.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.189+0000 7f7ad8fe4700 1 -- 172.21.15.160:0/3416928322 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ad41012e0 msgr2=0x7f7ad419f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:50.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.189+0000 7f7ad8fe4700 1 --2- 172.21.15.160:0/3416928322 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ad41012e0 0x7f7ad419f1d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7ac400e9a0 tx=0x7f7ac400ecb0 comp rx=0 tx=0).stop 2024-03-07T07:39:50.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.189+0000 7f7ad2d9d700 1 -- 172.21.15.160:0/3416928322 reap_dead start 2024-03-07T07:39:50.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.189+0000 7f7ad8fe4700 1 -- 172.21.15.160:0/3416928322 shutdown_connections 2024-03-07T07:39:50.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.189+0000 7f7ad8fe4700 1 -- 172.21.15.160:0/3416928322 >> 172.21.15.160:0/3416928322 conn(0x7f7ad40770a0 msgr2=0x7f7ad410cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:50.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.189+0000 7f7ad8fe4700 1 -- 172.21.15.160:0/3416928322 shutdown_connections 2024-03-07T07:39:50.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:50.189+0000 7f7ad8fe4700 1 -- 172.21.15.160:0/3416928322 wait complete. 2024-03-07T07:39:51.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:51 smithi177 ceph-mon[36578]: pgmap v306: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 B/s wr, 7 op/s 2024-03-07T07:39:51.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:51 smithi160 ceph-mon[31963]: pgmap v306: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 B/s wr, 7 op/s 2024-03-07T07:39:51.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:51 smithi178 ceph-mon[36650]: pgmap v306: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 B/s wr, 7 op/s 2024-03-07T07:39:51.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.828+0000 7f9c925a0700 1 -- 172.21.15.160:0/3190238039 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c8c101190 msgr2=0x7f9c8c101570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:51.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.828+0000 7f9c925a0700 1 --2- 172.21.15.160:0/3190238039 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c8c101190 0x7f9c8c101570 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9c880099c0 tx=0x7f9c88009cd0 comp rx=0 tx=0).stop 2024-03-07T07:39:51.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.828+0000 7f9c925a0700 1 -- 172.21.15.160:0/3190238039 shutdown_connections 2024-03-07T07:39:51.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.828+0000 7f9c925a0700 1 --2- 172.21.15.160:0/3190238039 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c8c106270 0x7f9c8c10de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:51.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.828+0000 7f9c925a0700 1 --2- 172.21.15.160:0/3190238039 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c8c101190 0x7f9c8c101570 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:51.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.828+0000 7f9c925a0700 1 --2- 172.21.15.160:0/3190238039 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c8c101b40 0x7f9c8c105b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:51.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.828+0000 7f9c925a0700 1 -- 172.21.15.160:0/3190238039 >> 172.21.15.160:0/3190238039 conn(0x7f9c8c0fca20 msgr2=0x7f9c8c0fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:51.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.829+0000 7f9c925a0700 1 -- 172.21.15.160:0/3190238039 shutdown_connections 2024-03-07T07:39:51.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.829+0000 7f9c925a0700 1 -- 172.21.15.160:0/3190238039 wait complete. 2024-03-07T07:39:51.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.829+0000 7f9c925a0700 1 Processor -- start 2024-03-07T07:39:51.839 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.829+0000 7f9c925a0700 1 -- start start 2024-03-07T07:39:51.839 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.830+0000 7f9c925a0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c8c101190 0x7f9c8c197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:51.839 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.830+0000 7f9c925a0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c8c101b40 0x7f9c8c197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:51.839 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.830+0000 7f9c925a0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c8c106270 0x7f9c8c19bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:51.839 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.830+0000 7f9c925a0700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8c111280 con 0x7f9c8c101190 2024-03-07T07:39:51.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.830+0000 7f9c925a0700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8c111100 con 0x7f9c8c101b40 2024-03-07T07:39:51.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.830+0000 7f9c925a0700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8c111400 con 0x7f9c8c106270 2024-03-07T07:39:51.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.830+0000 7f9c9159e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c8c101190 0x7f9c8c197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:51.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.830+0000 7f9c9159e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c8c101190 0x7f9c8c197710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:38746/0 (socket says 172.21.15.160:38746) 2024-03-07T07:39:51.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.830+0000 7f9c9159e700 1 -- 172.21.15.160:0/103527450 learned_addr learned my addr 172.21.15.160:0/103527450 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:39:51.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.830+0000 7f9c91d9f700 1 --2- 172.21.15.160:0/103527450 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c8c106270 0x7f9c8c19bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:51.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.830+0000 7f9c90d9d700 1 --2- 172.21.15.160:0/103527450 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c8c101b40 0x7f9c8c197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:51.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.830+0000 7f9c9159e700 1 -- 172.21.15.160:0/103527450 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c8c106270 msgr2=0x7f9c8c19bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:51.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.830+0000 7f9c9159e700 1 --2- 172.21.15.160:0/103527450 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c8c106270 0x7f9c8c19bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:51.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.830+0000 7f9c9159e700 1 -- 172.21.15.160:0/103527450 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c8c101b40 msgr2=0x7f9c8c197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:51.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.830+0000 7f9c9159e700 1 --2- 172.21.15.160:0/103527450 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c8c101b40 0x7f9c8c197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:51.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.830+0000 7f9c9159e700 1 -- 172.21.15.160:0/103527450 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c8c19c6d0 con 0x7f9c8c101190 2024-03-07T07:39:51.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.830+0000 7f9c9159e700 1 --2- 172.21.15.160:0/103527450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c8c101190 0x7f9c8c197710 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9c88006010 tx=0x7f9c8800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:51.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.830+0000 7f9c827fc700 1 -- 172.21.15.160:0/103527450 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c88003680 con 0x7f9c8c101190 2024-03-07T07:39:51.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.830+0000 7f9c827fc700 1 -- 172.21.15.160:0/103527450 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c880050d0 con 0x7f9c8c101190 2024-03-07T07:39:51.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.830+0000 7f9c925a0700 1 -- 172.21.15.160:0/103527450 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c8c19c960 con 0x7f9c8c101190 2024-03-07T07:39:51.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.830+0000 7f9c827fc700 1 -- 172.21.15.160:0/103527450 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c88003db0 con 0x7f9c8c101190 2024-03-07T07:39:51.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.830+0000 7f9c925a0700 1 -- 172.21.15.160:0/103527450 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c8c1a4240 con 0x7f9c8c101190 2024-03-07T07:39:51.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.831+0000 7f9c925a0700 1 -- 172.21.15.160:0/103527450 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9c8c0628a0 con 0x7f9c8c101190 2024-03-07T07:39:51.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.832+0000 7f9c827fc700 1 -- 172.21.15.160:0/103527450 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) v1 ==== 136805+0+0 (secure 0 0 0) 0x7f9c88005280 con 0x7f9c8c101190 2024-03-07T07:39:51.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.833+0000 7f9c827fc700 1 --2- 172.21.15.160:0/103527450 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f9c780a2040 0x7f9c780a4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:51.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.833+0000 7f9c90d9d700 1 --2- 172.21.15.160:0/103527450 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f9c780a2040 0x7f9c780a4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:51.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.833+0000 7f9c827fc700 1 -- 172.21.15.160:0/103527450 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9c880cee20 con 0x7f9c8c101190 2024-03-07T07:39:51.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.833+0000 7f9c827fc700 1 --2- 172.21.15.160:0/103527450 >> [v2:172.21.15.177:6800/2922391884,v1:172.21.15.177:6801/2922391884] conn(0x7f9c780ab820 0x7f9c780adc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:51.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.833+0000 7f9c827fc700 1 -- 172.21.15.160:0/103527450 --> [v2:172.21.15.177:6800/2922391884,v1:172.21.15.177:6801/2922391884] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9c88003680 con 0x7f9c780ab820 2024-03-07T07:39:51.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.833+0000 7f9c827fc700 1 -- 172.21.15.160:0/103527450 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c880cf300 con 0x7f9c8c101190 2024-03-07T07:39:51.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.833+0000 7f9c90d9d700 1 --2- 172.21.15.160:0/103527450 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f9c780a2040 0x7f9c780a4500 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c107820 tx=0x7f9c7c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:51.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.833+0000 7f9c91d9f700 1 --2- 172.21.15.160:0/103527450 >> [v2:172.21.15.177:6800/2922391884,v1:172.21.15.177:6801/2922391884] conn(0x7f9c780ab820 0x7f9c780adc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:51.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.834+0000 7f9c91d9f700 1 --2- 172.21.15.160:0/103527450 >> [v2:172.21.15.177:6800/2922391884,v1:172.21.15.177:6801/2922391884] conn(0x7f9c780ab820 0x7f9c780adc40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:51.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.834+0000 7f9c827fc700 1 -- 172.21.15.160:0/103527450 <== osd.1 v2:172.21.15.177:6800/2922391884 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f9c88003680 con 0x7f9c780ab820 2024-03-07T07:39:51.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.851+0000 7f9c925a0700 1 -- 172.21.15.160:0/103527450 --> [v2:172.21.15.177:6800/2922391884,v1:172.21.15.177:6801/2922391884] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9c8c062d20 con 0x7f9c780ab820 2024-03-07T07:39:51.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.852+0000 7f9c827fc700 1 -- 172.21.15.160:0/103527450 <== osd.1 v2:172.21.15.177:6800/2922391884 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9c8c062d20 con 0x7f9c780ab820 2024-03-07T07:39:51.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.852+0000 7f9c925a0700 1 -- 172.21.15.160:0/103527450 >> [v2:172.21.15.177:6800/2922391884,v1:172.21.15.177:6801/2922391884] conn(0x7f9c780ab820 msgr2=0x7f9c780adc40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:51.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.852+0000 7f9c925a0700 1 --2- 172.21.15.160:0/103527450 >> [v2:172.21.15.177:6800/2922391884,v1:172.21.15.177:6801/2922391884] conn(0x7f9c780ab820 0x7f9c780adc40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:51.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.852+0000 7f9c925a0700 1 -- 172.21.15.160:0/103527450 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f9c780a2040 msgr2=0x7f9c780a4500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:51.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.852+0000 7f9c925a0700 1 --2- 172.21.15.160:0/103527450 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f9c780a2040 0x7f9c780a4500 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c107820 tx=0x7f9c7c009260 comp rx=0 tx=0).stop 2024-03-07T07:39:51.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.852+0000 7f9c925a0700 1 -- 172.21.15.160:0/103527450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c8c101190 msgr2=0x7f9c8c197710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:51.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.852+0000 7f9c925a0700 1 --2- 172.21.15.160:0/103527450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c8c101190 0x7f9c8c197710 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9c88006010 tx=0x7f9c8800f690 comp rx=0 tx=0).stop 2024-03-07T07:39:51.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.852+0000 7f9c91d9f700 1 -- 172.21.15.160:0/103527450 reap_dead start 2024-03-07T07:39:51.855 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.853+0000 7f9c925a0700 1 -- 172.21.15.160:0/103527450 shutdown_connections 2024-03-07T07:39:51.855 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.853+0000 7f9c925a0700 1 -- 172.21.15.160:0/103527450 >> 172.21.15.160:0/103527450 conn(0x7f9c8c0fca20 msgr2=0x7f9c8c0fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:51.855 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.853+0000 7f9c925a0700 1 -- 172.21.15.160:0/103527450 shutdown_connections 2024-03-07T07:39:51.855 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:51.853+0000 7f9c925a0700 1 -- 172.21.15.160:0/103527450 wait complete. 2024-03-07T07:39:52.310 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.310+0000 7f21908d6700 1 -- 172.21.15.160:0/1826501077 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f218c107090 msgr2=0x7f218c107470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:52.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.310+0000 7f21908d6700 1 --2- 172.21.15.160:0/1826501077 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f218c107090 0x7f218c107470 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f217c009990 tx=0x7f217c009ca0 comp rx=0 tx=0).stop 2024-03-07T07:39:52.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.310+0000 7f21908d6700 1 -- 172.21.15.160:0/1826501077 shutdown_connections 2024-03-07T07:39:52.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.310+0000 7f21908d6700 1 --2- 172.21.15.160:0/1826501077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f218c1019f0 0x7f218c10ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:52.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.310+0000 7f21908d6700 1 --2- 172.21.15.160:0/1826501077 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f218c107090 0x7f218c107470 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:52.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.310+0000 7f21908d6700 1 --2- 172.21.15.160:0/1826501077 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f218c101050 0x7f218c1014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:52.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.310+0000 7f21908d6700 1 -- 172.21.15.160:0/1826501077 >> 172.21.15.160:0/1826501077 conn(0x7f218c077f80 msgr2=0x7f218c078390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:52.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.310+0000 7f21908d6700 1 -- 172.21.15.160:0/1826501077 shutdown_connections 2024-03-07T07:39:52.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.310+0000 7f21908d6700 1 -- 172.21.15.160:0/1826501077 wait complete. 2024-03-07T07:39:52.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.311+0000 7f21908d6700 1 Processor -- start 2024-03-07T07:39:52.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.311+0000 7f21908d6700 1 -- start start 2024-03-07T07:39:52.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.311+0000 7f21908d6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f218c101050 0x7f218c103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:52.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.311+0000 7f21908d6700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f218c1019f0 0x7f218c1035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:52.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.311+0000 7f21908d6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f218c107090 0x7f218c103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:52.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.311+0000 7f21908d6700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f218c1112d0 con 0x7f218c107090 2024-03-07T07:39:52.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.311+0000 7f21908d6700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f218c111150 con 0x7f218c1019f0 2024-03-07T07:39:52.314 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.311+0000 7f21908d6700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f218c111450 con 0x7f218c101050 2024-03-07T07:39:52.314 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.311+0000 7f218a59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f218c101050 0x7f218c103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:52.314 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.311+0000 7f218a59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f218c101050 0x7f218c103070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.160:51656/0 (socket says 172.21.15.160:51656) 2024-03-07T07:39:52.314 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.311+0000 7f218a59c700 1 -- 172.21.15.160:0/2620159772 learned_addr learned my addr 172.21.15.160:0/2620159772 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:39:52.314 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.311+0000 7f218ad9d700 1 --2- 172.21.15.160:0/2620159772 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f218c107090 0x7f218c103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:52.314 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.311+0000 7f2189d9b700 1 --2- 172.21.15.160:0/2620159772 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f218c1019f0 0x7f218c1035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:52.314 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.311+0000 7f218a59c700 1 -- 172.21.15.160:0/2620159772 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f218c1019f0 msgr2=0x7f218c1035b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:52.314 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.311+0000 7f218a59c700 1 --2- 172.21.15.160:0/2620159772 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f218c1019f0 0x7f218c1035b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:52.314 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.311+0000 7f218a59c700 1 -- 172.21.15.160:0/2620159772 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f218c107090 msgr2=0x7f218c103af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:52.314 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.311+0000 7f218a59c700 1 --2- 172.21.15.160:0/2620159772 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f218c107090 0x7f218c103af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:52.314 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.311+0000 7f218a59c700 1 -- 172.21.15.160:0/2620159772 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f218c1a6df0 con 0x7f218c101050 2024-03-07T07:39:52.314 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.312+0000 7f218a59c700 1 --2- 172.21.15.160:0/2620159772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f218c101050 0x7f218c103070 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f217c00b2e0 tx=0x7f217c00b610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:52.314 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.312+0000 7f217b7fe700 1 -- 172.21.15.160:0/2620159772 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217c026070 con 0x7f218c101050 2024-03-07T07:39:52.314 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.312+0000 7f217b7fe700 1 -- 172.21.15.160:0/2620159772 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f217c021410 con 0x7f218c101050 2024-03-07T07:39:52.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.312+0000 7f21908d6700 1 -- 172.21.15.160:0/2620159772 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f218c1a7080 con 0x7f218c101050 2024-03-07T07:39:52.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.312+0000 7f217b7fe700 1 -- 172.21.15.160:0/2620159772 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217c017650 con 0x7f218c101050 2024-03-07T07:39:52.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.312+0000 7f21908d6700 1 -- 172.21.15.160:0/2620159772 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f218c1a74d0 con 0x7f218c101050 2024-03-07T07:39:52.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.313+0000 7f21908d6700 1 -- 172.21.15.160:0/2620159772 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f218c06a0d0 con 0x7f218c101050 2024-03-07T07:39:52.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.314+0000 7f217b7fe700 1 -- 172.21.15.160:0/2620159772 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) v1 ==== 136805+0+0 (secure 0 0 0) 0x7f217c029070 con 0x7f218c101050 2024-03-07T07:39:52.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.314+0000 7f217b7fe700 1 --2- 172.21.15.160:0/2620159772 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f21740a1ea0 0x7f21740a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:52.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.315+0000 7f2189d9b700 1 --2- 172.21.15.160:0/2620159772 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f21740a1ea0 0x7f21740a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:52.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.315+0000 7f217b7fe700 1 -- 172.21.15.160:0/2620159772 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f217c0cf150 con 0x7f218c101050 2024-03-07T07:39:52.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.315+0000 7f217b7fe700 1 --2- 172.21.15.160:0/2620159772 >> [v2:172.21.15.160:6802/1368482926,v1:172.21.15.160:6803/1368482926] conn(0x7f21740ab680 0x7f21740adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:52.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.315+0000 7f217b7fe700 1 -- 172.21.15.160:0/2620159772 --> [v2:172.21.15.160:6802/1368482926,v1:172.21.15.160:6803/1368482926] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f217c027e50 con 0x7f21740ab680 2024-03-07T07:39:52.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.315+0000 7f217b7fe700 1 -- 172.21.15.160:0/2620159772 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f217c0cf5d0 con 0x7f218c101050 2024-03-07T07:39:52.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.315+0000 7f2189d9b700 1 --2- 172.21.15.160:0/2620159772 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f21740a1ea0 0x7f21740a4360 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f218c06aee0 tx=0x7f2180009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:52.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.315+0000 7f218ad9d700 1 --2- 172.21.15.160:0/2620159772 >> [v2:172.21.15.160:6802/1368482926,v1:172.21.15.160:6803/1368482926] conn(0x7f21740ab680 0x7f21740adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:52.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.315+0000 7f218ad9d700 1 --2- 172.21.15.160:0/2620159772 >> [v2:172.21.15.160:6802/1368482926,v1:172.21.15.160:6803/1368482926] conn(0x7f21740ab680 0x7f21740adaa0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:52.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.316+0000 7f217b7fe700 1 -- 172.21.15.160:0/2620159772 <== osd.0 v2:172.21.15.160:6802/1368482926 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f217c027e50 con 0x7f21740ab680 2024-03-07T07:39:52.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.332+0000 7f21908d6700 1 -- 172.21.15.160:0/2620159772 --> [v2:172.21.15.160:6802/1368482926,v1:172.21.15.160:6803/1368482926] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f218c102740 con 0x7f21740ab680 2024-03-07T07:39:52.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.332+0000 7f217b7fe700 1 -- 172.21.15.160:0/2620159772 <== osd.0 v2:172.21.15.160:6802/1368482926 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f218c102740 con 0x7f21740ab680 2024-03-07T07:39:52.333 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.332+0000 7f21908d6700 1 -- 172.21.15.160:0/2620159772 >> [v2:172.21.15.160:6802/1368482926,v1:172.21.15.160:6803/1368482926] conn(0x7f21740ab680 msgr2=0x7f21740adaa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:52.333 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.332+0000 7f21908d6700 1 --2- 172.21.15.160:0/2620159772 >> [v2:172.21.15.160:6802/1368482926,v1:172.21.15.160:6803/1368482926] conn(0x7f21740ab680 0x7f21740adaa0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:52.333 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.333+0000 7f21908d6700 1 -- 172.21.15.160:0/2620159772 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f21740a1ea0 msgr2=0x7f21740a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:52.333 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.333+0000 7f21908d6700 1 --2- 172.21.15.160:0/2620159772 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f21740a1ea0 0x7f21740a4360 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f218c06aee0 tx=0x7f2180009260 comp rx=0 tx=0).stop 2024-03-07T07:39:52.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.333+0000 7f21908d6700 1 -- 172.21.15.160:0/2620159772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f218c101050 msgr2=0x7f218c103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:52.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.333+0000 7f21908d6700 1 --2- 172.21.15.160:0/2620159772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f218c101050 0x7f218c103070 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f217c00b2e0 tx=0x7f217c00b610 comp rx=0 tx=0).stop 2024-03-07T07:39:52.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.333+0000 7f218ad9d700 1 -- 172.21.15.160:0/2620159772 reap_dead start 2024-03-07T07:39:52.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.333+0000 7f21908d6700 1 -- 172.21.15.160:0/2620159772 shutdown_connections 2024-03-07T07:39:52.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.333+0000 7f21908d6700 1 -- 172.21.15.160:0/2620159772 >> 172.21.15.160:0/2620159772 conn(0x7f218c077f80 msgr2=0x7f218c105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:52.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.333+0000 7f21908d6700 1 -- 172.21.15.160:0/2620159772 shutdown_connections 2024-03-07T07:39:52.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.333+0000 7f21908d6700 1 -- 172.21.15.160:0/2620159772 wait complete. 2024-03-07T07:39:53.427 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:53 smithi160 ceph-mon[31963]: pgmap v307: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-07T07:39:53.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:53 smithi178 ceph-mon[36650]: pgmap v307: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-07T07:39:53.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:53 smithi177 ceph-mon[36578]: pgmap v307: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-07T07:39:54.361 INFO:teuthology.orchestra.run.smithi160.stdout:34359738445 2024-03-07T07:39:54.361 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-07T07:39:54.589 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:54 smithi160 ceph-mon[31963]: pgmap v308: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-03-07T07:39:54.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:54 smithi178 ceph-mon[36650]: pgmap v308: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-03-07T07:39:54.813 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.810+0000 7f83ddd83700 1 -- 172.21.15.160:0/1331540233 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83d8102b60 msgr2=0x7f83d810f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:54.814 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.810+0000 7f83ddd83700 1 --2- 172.21.15.160:0/1331540233 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83d8102b60 0x7f83d810f000 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f83d000b0a0 tx=0x7f83d000b3b0 comp rx=0 tx=0).stop 2024-03-07T07:39:54.814 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.810+0000 7f83ddd83700 1 -- 172.21.15.160:0/1331540233 shutdown_connections 2024-03-07T07:39:54.814 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.810+0000 7f83ddd83700 1 --2- 172.21.15.160:0/1331540233 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83d8102b60 0x7f83d810f000 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:54.814 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.810+0000 7f83ddd83700 1 --2- 172.21.15.160:0/1331540233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83d8108250 0x7f83d8108630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:54.814 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.810+0000 7f83ddd83700 1 --2- 172.21.15.160:0/1331540233 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f83d81021a0 0x7f83d8102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:54.814 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.810+0000 7f83ddd83700 1 -- 172.21.15.160:0/1331540233 >> 172.21.15.160:0/1331540233 conn(0x7f83d8079280 msgr2=0x7f83d8079690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:54.814 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.810+0000 7f83ddd83700 1 -- 172.21.15.160:0/1331540233 shutdown_connections 2024-03-07T07:39:54.814 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.810+0000 7f83ddd83700 1 -- 172.21.15.160:0/1331540233 wait complete. 2024-03-07T07:39:54.814 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.811+0000 7f83ddd83700 1 Processor -- start 2024-03-07T07:39:54.814 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.811+0000 7f83ddd83700 1 -- start start 2024-03-07T07:39:54.814 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.811+0000 7f83ddd83700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f83d81021a0 0x7f83d8074450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:54.814 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.811+0000 7f83ddd83700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83d8102b60 0x7f83d8074990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:54.814 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.811+0000 7f83ddd83700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83d8108250 0x7f83d8074ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:54.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.811+0000 7f83ddd83700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d8112480 con 0x7f83d8102b60 2024-03-07T07:39:54.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.811+0000 7f83ddd83700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d8112300 con 0x7f83d81021a0 2024-03-07T07:39:54.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.811+0000 7f83ddd83700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d8112600 con 0x7f83d8108250 2024-03-07T07:39:54.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.811+0000 7f83d6ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83d8102b60 0x7f83d8074990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:54.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.811+0000 7f83d6ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83d8102b60 0x7f83d8074990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:38804/0 (socket says 172.21.15.160:38804) 2024-03-07T07:39:54.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.811+0000 7f83d6ffd700 1 -- 172.21.15.160:0/559213277 learned_addr learned my addr 172.21.15.160:0/559213277 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:39:54.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.811+0000 7f83d77fe700 1 --2- 172.21.15.160:0/559213277 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f83d81021a0 0x7f83d8074450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:54.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.811+0000 7f83d7fff700 1 --2- 172.21.15.160:0/559213277 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83d8108250 0x7f83d8074ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:54.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.812+0000 7f83d6ffd700 1 -- 172.21.15.160:0/559213277 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83d8108250 msgr2=0x7f83d8074ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:54.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.812+0000 7f83d6ffd700 1 --2- 172.21.15.160:0/559213277 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83d8108250 0x7f83d8074ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:54.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.812+0000 7f83d6ffd700 1 -- 172.21.15.160:0/559213277 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f83d81021a0 msgr2=0x7f83d8074450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:54.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.812+0000 7f83d6ffd700 1 --2- 172.21.15.160:0/559213277 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f83d81021a0 0x7f83d8074450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:54.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.812+0000 7f83d6ffd700 1 -- 172.21.15.160:0/559213277 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d81a6d80 con 0x7f83d8102b60 2024-03-07T07:39:54.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.812+0000 7f83d77fe700 1 --2- 172.21.15.160:0/559213277 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f83d81021a0 0x7f83d8074450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:39:54.816 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.812+0000 7f83d7fff700 1 --2- 172.21.15.160:0/559213277 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83d8108250 0x7f83d8074ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:39:54.816 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.812+0000 7f83d6ffd700 1 --2- 172.21.15.160:0/559213277 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83d8102b60 0x7f83d8074990 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f83cc00e9a0 tx=0x7f83cc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:54.816 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.812+0000 7f83d4ff9700 1 -- 172.21.15.160:0/559213277 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83cc00ca10 con 0x7f83d8102b60 2024-03-07T07:39:54.816 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.812+0000 7f83ddd83700 1 -- 172.21.15.160:0/559213277 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83d81a7070 con 0x7f83d8102b60 2024-03-07T07:39:54.816 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.812+0000 7f83d4ff9700 1 -- 172.21.15.160:0/559213277 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f83cc00cbb0 con 0x7f83d8102b60 2024-03-07T07:39:54.816 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.812+0000 7f83d4ff9700 1 -- 172.21.15.160:0/559213277 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83cc018680 con 0x7f83d8102b60 2024-03-07T07:39:54.816 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.812+0000 7f83ddd83700 1 -- 172.21.15.160:0/559213277 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83d81a75b0 con 0x7f83d8102b60 2024-03-07T07:39:54.816 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.813+0000 7f83ddd83700 1 -- 172.21.15.160:0/559213277 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f83d8062860 con 0x7f83d8102b60 2024-03-07T07:39:54.817 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.814+0000 7f83d4ff9700 1 -- 172.21.15.160:0/559213277 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) v1 ==== 136805+0+0 (secure 0 0 0) 0x7f83cc010430 con 0x7f83d8102b60 2024-03-07T07:39:54.817 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.815+0000 7f83d4ff9700 1 --2- 172.21.15.160:0/559213277 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f83c00a1ea0 0x7f83c00a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:54.817 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.815+0000 7f83d77fe700 1 --2- 172.21.15.160:0/559213277 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f83c00a1ea0 0x7f83c00a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:54.817 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.815+0000 7f83d4ff9700 1 -- 172.21.15.160:0/559213277 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f83cc0ce7d0 con 0x7f83d8102b60 2024-03-07T07:39:54.818 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.815+0000 7f83d4ff9700 1 --2- 172.21.15.160:0/559213277 >> [v2:172.21.15.178:6808/964732891,v1:172.21.15.178:6809/964732891] conn(0x7f83c00ab680 0x7f83c00adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:54.818 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.815+0000 7f83d4ff9700 1 -- 172.21.15.160:0/559213277 --> [v2:172.21.15.178:6808/964732891,v1:172.21.15.178:6809/964732891] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f83cc01bec0 con 0x7f83c00ab680 2024-03-07T07:39:54.818 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.815+0000 7f83d4ff9700 1 -- 172.21.15.160:0/559213277 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83cc0cecb0 con 0x7f83d8102b60 2024-03-07T07:39:54.818 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.815+0000 7f83d77fe700 1 --2- 172.21.15.160:0/559213277 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f83c00a1ea0 0x7f83c00a4360 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f83c8009fd0 tx=0x7f83c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:54.818 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.815+0000 7f83d7fff700 1 --2- 172.21.15.160:0/559213277 >> [v2:172.21.15.178:6808/964732891,v1:172.21.15.178:6809/964732891] conn(0x7f83c00ab680 0x7f83c00adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:54.818 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.816+0000 7f83d7fff700 1 --2- 172.21.15.160:0/559213277 >> [v2:172.21.15.178:6808/964732891,v1:172.21.15.178:6809/964732891] conn(0x7f83c00ab680 0x7f83c00adaa0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:54.818 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.816+0000 7f83d4ff9700 1 -- 172.21.15.160:0/559213277 <== osd.5 v2:172.21.15.178:6808/964732891 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f83cc01bec0 con 0x7f83c00ab680 2024-03-07T07:39:54.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.832+0000 7f83ddd83700 1 -- 172.21.15.160:0/559213277 --> [v2:172.21.15.178:6808/964732891,v1:172.21.15.178:6809/964732891] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f83d8062ce0 con 0x7f83c00ab680 2024-03-07T07:39:54.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.833+0000 7f83d4ff9700 1 -- 172.21.15.160:0/559213277 <== osd.5 v2:172.21.15.178:6808/964732891 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f83d8062ce0 con 0x7f83c00ab680 2024-03-07T07:39:54.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.833+0000 7f83ddd83700 1 -- 172.21.15.160:0/559213277 >> [v2:172.21.15.178:6808/964732891,v1:172.21.15.178:6809/964732891] conn(0x7f83c00ab680 msgr2=0x7f83c00adaa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:54.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.833+0000 7f83ddd83700 1 --2- 172.21.15.160:0/559213277 >> [v2:172.21.15.178:6808/964732891,v1:172.21.15.178:6809/964732891] conn(0x7f83c00ab680 0x7f83c00adaa0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:54.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.833+0000 7f83ddd83700 1 -- 172.21.15.160:0/559213277 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f83c00a1ea0 msgr2=0x7f83c00a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:54.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.833+0000 7f83ddd83700 1 --2- 172.21.15.160:0/559213277 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f83c00a1ea0 0x7f83c00a4360 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f83c8009fd0 tx=0x7f83c8009260 comp rx=0 tx=0).stop 2024-03-07T07:39:54.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.833+0000 7f83ddd83700 1 -- 172.21.15.160:0/559213277 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83d8102b60 msgr2=0x7f83d8074990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:54.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.833+0000 7f83ddd83700 1 --2- 172.21.15.160:0/559213277 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83d8102b60 0x7f83d8074990 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f83cc00e9a0 tx=0x7f83cc00ecb0 comp rx=0 tx=0).stop 2024-03-07T07:39:54.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.833+0000 7f83d7fff700 1 -- 172.21.15.160:0/559213277 reap_dead start 2024-03-07T07:39:54.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.833+0000 7f83ddd83700 1 -- 172.21.15.160:0/559213277 shutdown_connections 2024-03-07T07:39:54.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.833+0000 7f83ddd83700 1 -- 172.21.15.160:0/559213277 >> 172.21.15.160:0/559213277 conn(0x7f83d8079280 msgr2=0x7f83d8100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:54.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.833+0000 7f83ddd83700 1 -- 172.21.15.160:0/559213277 shutdown_connections 2024-03-07T07:39:54.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:54.833+0000 7f83ddd83700 1 -- 172.21.15.160:0/559213277 wait complete. 2024-03-07T07:39:54.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:54 smithi177 ceph-mon[36578]: pgmap v308: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-03-07T07:39:55.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.398+0000 7f371b7b7700 1 -- 172.21.15.160:0/507635568 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3714102430 msgr2=0x7f3714102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:55.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.398+0000 7f371b7b7700 1 --2- 172.21.15.160:0/507635568 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3714102430 0x7f3714102810 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f37100099c0 tx=0x7f3710009cd0 comp rx=0 tx=0).stop 2024-03-07T07:39:55.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.398+0000 7f371b7b7700 1 -- 172.21.15.160:0/507635568 shutdown_connections 2024-03-07T07:39:55.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.398+0000 7f371b7b7700 1 --2- 172.21.15.160:0/507635568 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37141074e0 0x7f371410f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:55.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.398+0000 7f371b7b7700 1 --2- 172.21.15.160:0/507635568 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3714102430 0x7f3714102810 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:55.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.398+0000 7f371b7b7700 1 --2- 172.21.15.160:0/507635568 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3714102de0 0x7f3714106db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:55.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.398+0000 7f371b7b7700 1 -- 172.21.15.160:0/507635568 >> 172.21.15.160:0/507635568 conn(0x7f3714079270 msgr2=0x7f3714079680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:55.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.399+0000 7f371b7b7700 1 -- 172.21.15.160:0/507635568 shutdown_connections 2024-03-07T07:39:55.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.399+0000 7f371b7b7700 1 -- 172.21.15.160:0/507635568 wait complete. 2024-03-07T07:39:55.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.399+0000 7f371b7b7700 1 Processor -- start 2024-03-07T07:39:55.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.399+0000 7f371b7b7700 1 -- start start 2024-03-07T07:39:55.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.399+0000 7f371b7b7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3714102430 0x7f371419e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:55.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.399+0000 7f371b7b7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3714102de0 0x7f371419eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:55.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.399+0000 7f371b7b7700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37141074e0 0x7f3714198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:55.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.399+0000 7f371b7b7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37141124a0 con 0x7f3714102430 2024-03-07T07:39:55.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.399+0000 7f371b7b7700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3714112320 con 0x7f37141074e0 2024-03-07T07:39:55.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.399+0000 7f371b7b7700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3714112620 con 0x7f3714102de0 2024-03-07T07:39:55.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.400+0000 7f3718d52700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3714102de0 0x7f371419eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:55.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.400+0000 7f3719553700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3714102430 0x7f371419e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:55.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.400+0000 7f3718d52700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3714102de0 0x7f371419eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.160:51714/0 (socket says 172.21.15.160:51714) 2024-03-07T07:39:55.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.400+0000 7f3718d52700 1 -- 172.21.15.160:0/4275385565 learned_addr learned my addr 172.21.15.160:0/4275385565 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:39:55.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.400+0000 7f3719d54700 1 --2- 172.21.15.160:0/4275385565 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37141074e0 0x7f3714198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:55.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.400+0000 7f3718d52700 1 -- 172.21.15.160:0/4275385565 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37141074e0 msgr2=0x7f3714198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:55.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.400+0000 7f3718d52700 1 --2- 172.21.15.160:0/4275385565 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37141074e0 0x7f3714198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:55.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.400+0000 7f3718d52700 1 -- 172.21.15.160:0/4275385565 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3714102430 msgr2=0x7f371419e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:55.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.400+0000 7f3718d52700 1 --2- 172.21.15.160:0/4275385565 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3714102430 0x7f371419e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:55.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.400+0000 7f3718d52700 1 -- 172.21.15.160:0/4275385565 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3714198fe0 con 0x7f3714102de0 2024-03-07T07:39:55.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.400+0000 7f3719553700 1 --2- 172.21.15.160:0/4275385565 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3714102430 0x7f371419e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:39:55.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.400+0000 7f3719d54700 1 --2- 172.21.15.160:0/4275385565 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37141074e0 0x7f3714198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:39:55.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.400+0000 7f3718d52700 1 --2- 172.21.15.160:0/4275385565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3714102de0 0x7f371419eb70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f370400e940 tx=0x7f370400ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:55.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.400+0000 7f370a7fc700 1 -- 172.21.15.160:0/4275385565 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f370400c9e0 con 0x7f3714102de0 2024-03-07T07:39:55.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.400+0000 7f371b7b7700 1 -- 172.21.15.160:0/4275385565 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37141992d0 con 0x7f3714102de0 2024-03-07T07:39:55.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.400+0000 7f370a7fc700 1 -- 172.21.15.160:0/4275385565 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f370400cb80 con 0x7f3714102de0 2024-03-07T07:39:55.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.400+0000 7f371b7b7700 1 -- 172.21.15.160:0/4275385565 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37141a55b0 con 0x7f3714102de0 2024-03-07T07:39:55.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.400+0000 7f370a7fc700 1 -- 172.21.15.160:0/4275385565 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3704018620 con 0x7f3714102de0 2024-03-07T07:39:55.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.401+0000 7f371b7b7700 1 -- 172.21.15.160:0/4275385565 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3714062860 con 0x7f3714102de0 2024-03-07T07:39:55.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.403+0000 7f370a7fc700 1 -- 172.21.15.160:0/4275385565 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) v1 ==== 136805+0+0 (secure 0 0 0) 0x7f3704010430 con 0x7f3714102de0 2024-03-07T07:39:55.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.403+0000 7f370a7fc700 1 --2- 172.21.15.160:0/4275385565 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f37000a1ea0 0x7f37000a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:55.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.403+0000 7f3719553700 1 --2- 172.21.15.160:0/4275385565 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f37000a1ea0 0x7f37000a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:55.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.403+0000 7f370a7fc700 1 -- 172.21.15.160:0/4275385565 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f37040ce2f0 con 0x7f3714102de0 2024-03-07T07:39:55.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.403+0000 7f370a7fc700 1 --2- 172.21.15.160:0/4275385565 >> [v2:172.21.15.178:6824/4068724731,v1:172.21.15.178:6825/4068724731] conn(0x7f37000ab680 0x7f37000adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:55.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.403+0000 7f370a7fc700 1 -- 172.21.15.160:0/4275385565 --> [v2:172.21.15.178:6824/4068724731,v1:172.21.15.178:6825/4068724731] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f370401bec0 con 0x7f37000ab680 2024-03-07T07:39:55.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.403+0000 7f370a7fc700 1 -- 172.21.15.160:0/4275385565 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37040ce7d0 con 0x7f3714102de0 2024-03-07T07:39:55.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.403+0000 7f3719d54700 1 --2- 172.21.15.160:0/4275385565 >> [v2:172.21.15.178:6824/4068724731,v1:172.21.15.178:6825/4068724731] conn(0x7f37000ab680 0x7f37000adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:55.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.403+0000 7f3719553700 1 --2- 172.21.15.160:0/4275385565 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f37000a1ea0 0x7f37000a4360 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3710000c00 tx=0x7f3710005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:55.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.404+0000 7f3719d54700 1 --2- 172.21.15.160:0/4275385565 >> [v2:172.21.15.178:6824/4068724731,v1:172.21.15.178:6825/4068724731] conn(0x7f37000ab680 0x7f37000adaa0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:55.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.405+0000 7f370a7fc700 1 -- 172.21.15.160:0/4275385565 <== osd.11 v2:172.21.15.178:6824/4068724731 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f370401bec0 con 0x7f37000ab680 2024-03-07T07:39:55.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.421+0000 7f371b7b7700 1 -- 172.21.15.160:0/4275385565 --> [v2:172.21.15.178:6824/4068724731,v1:172.21.15.178:6825/4068724731] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3714062c90 con 0x7f37000ab680 2024-03-07T07:39:55.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.421+0000 7f370a7fc700 1 -- 172.21.15.160:0/4275385565 <== osd.11 v2:172.21.15.178:6824/4068724731 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f3714062c90 con 0x7f37000ab680 2024-03-07T07:39:55.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.421+0000 7f371b7b7700 1 -- 172.21.15.160:0/4275385565 >> [v2:172.21.15.178:6824/4068724731,v1:172.21.15.178:6825/4068724731] conn(0x7f37000ab680 msgr2=0x7f37000adaa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:55.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.421+0000 7f371b7b7700 1 --2- 172.21.15.160:0/4275385565 >> [v2:172.21.15.178:6824/4068724731,v1:172.21.15.178:6825/4068724731] conn(0x7f37000ab680 0x7f37000adaa0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:55.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.421+0000 7f371b7b7700 1 -- 172.21.15.160:0/4275385565 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f37000a1ea0 msgr2=0x7f37000a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:55.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.421+0000 7f371b7b7700 1 --2- 172.21.15.160:0/4275385565 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f37000a1ea0 0x7f37000a4360 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3710000c00 tx=0x7f3710005fb0 comp rx=0 tx=0).stop 2024-03-07T07:39:55.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.421+0000 7f371b7b7700 1 -- 172.21.15.160:0/4275385565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3714102de0 msgr2=0x7f371419eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:55.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.421+0000 7f371b7b7700 1 --2- 172.21.15.160:0/4275385565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3714102de0 0x7f371419eb70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f370400e940 tx=0x7f370400ec50 comp rx=0 tx=0).stop 2024-03-07T07:39:55.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.421+0000 7f3719d54700 1 -- 172.21.15.160:0/4275385565 reap_dead start 2024-03-07T07:39:55.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.421+0000 7f371b7b7700 1 -- 172.21.15.160:0/4275385565 shutdown_connections 2024-03-07T07:39:55.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.421+0000 7f371b7b7700 1 -- 172.21.15.160:0/4275385565 >> 172.21.15.160:0/4275385565 conn(0x7f3714079270 msgr2=0x7f3714101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:55.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.421+0000 7f371b7b7700 1 -- 172.21.15.160:0/4275385565 shutdown_connections 2024-03-07T07:39:55.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.421+0000 7f371b7b7700 1 -- 172.21.15.160:0/4275385565 wait complete. 2024-03-07T07:39:55.916 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.915+0000 7f980e265700 1 -- 172.21.15.160:0/1067267267 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9808101330 msgr2=0x7f98081017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:55.916 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.915+0000 7f980e265700 1 --2- 172.21.15.160:0/1067267267 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9808101330 0x7f98081017b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f97fc009920 tx=0x7f97fc009c30 comp rx=0 tx=0).stop 2024-03-07T07:39:55.916 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.915+0000 7f980e265700 1 -- 172.21.15.160:0/1067267267 shutdown_connections 2024-03-07T07:39:55.916 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.915+0000 7f980e265700 1 --2- 172.21.15.160:0/1067267267 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9808101cf0 0x7f980810df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:55.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.915+0000 7f980e265700 1 --2- 172.21.15.160:0/1067267267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9808107180 0x7f9808107560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:55.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.915+0000 7f980e265700 1 --2- 172.21.15.160:0/1067267267 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9808101330 0x7f98081017b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:55.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.915+0000 7f980e265700 1 -- 172.21.15.160:0/1067267267 >> 172.21.15.160:0/1067267267 conn(0x7f9808077f10 msgr2=0x7f9808078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:55.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.915+0000 7f980e265700 1 -- 172.21.15.160:0/1067267267 shutdown_connections 2024-03-07T07:39:55.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.915+0000 7f980e265700 1 -- 172.21.15.160:0/1067267267 wait complete. 2024-03-07T07:39:55.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.916+0000 7f980e265700 1 Processor -- start 2024-03-07T07:39:55.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.916+0000 7f980e265700 1 -- start start 2024-03-07T07:39:55.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.916+0000 7f980e265700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9808101330 0x7f980819d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:55.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.916+0000 7f980e265700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9808101cf0 0x7f980819dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:55.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.916+0000 7f980e265700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9808107180 0x7f9808197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:55.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.916+0000 7f980e265700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9808111350 con 0x7f9808101cf0 2024-03-07T07:39:55.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.916+0000 7f980e265700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98081111d0 con 0x7f9808107180 2024-03-07T07:39:55.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.916+0000 7f980e265700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98081114d0 con 0x7f9808101330 2024-03-07T07:39:55.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.916+0000 7f98077fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9808101cf0 0x7f980819dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:55.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.916+0000 7f9807fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9808101330 0x7f980819d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:55.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.916+0000 7f98077fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9808101cf0 0x7f980819dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:38842/0 (socket says 172.21.15.160:38842) 2024-03-07T07:39:55.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.916+0000 7f98077fe700 1 -- 172.21.15.160:0/2945748295 learned_addr learned my addr 172.21.15.160:0/2945748295 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:39:55.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.916+0000 7f980c802700 1 --2- 172.21.15.160:0/2945748295 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9808107180 0x7f9808197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:55.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.917+0000 7f98077fe700 1 -- 172.21.15.160:0/2945748295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9808101330 msgr2=0x7f980819d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:55.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.917+0000 7f98077fe700 1 --2- 172.21.15.160:0/2945748295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9808101330 0x7f980819d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:55.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.917+0000 7f98077fe700 1 -- 172.21.15.160:0/2945748295 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9808107180 msgr2=0x7f9808197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:55.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.917+0000 7f98077fe700 1 --2- 172.21.15.160:0/2945748295 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9808107180 0x7f9808197650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:55.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.917+0000 7f98077fe700 1 -- 172.21.15.160:0/2945748295 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9808197f30 con 0x7f9808101cf0 2024-03-07T07:39:55.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.917+0000 7f9807fff700 1 --2- 172.21.15.160:0/2945748295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9808101330 0x7f980819d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:39:55.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.917+0000 7f980c802700 1 --2- 172.21.15.160:0/2945748295 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9808107180 0x7f9808197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:39:55.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.917+0000 7f98077fe700 1 --2- 172.21.15.160:0/2945748295 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9808101cf0 0x7f980819dac0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f97fc000c00 tx=0x7f97fc00fb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:55.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.917+0000 7f98057fa700 1 -- 172.21.15.160:0/2945748295 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97fc005270 con 0x7f9808101cf0 2024-03-07T07:39:55.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.917+0000 7f980e265700 1 -- 172.21.15.160:0/2945748295 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98081981c0 con 0x7f9808101cf0 2024-03-07T07:39:55.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.917+0000 7f98057fa700 1 -- 172.21.15.160:0/2945748295 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f97fc00ba40 con 0x7f9808101cf0 2024-03-07T07:39:55.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.917+0000 7f98057fa700 1 -- 172.21.15.160:0/2945748295 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97fc003b90 con 0x7f9808101cf0 2024-03-07T07:39:55.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.917+0000 7f980e265700 1 -- 172.21.15.160:0/2945748295 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98081a4330 con 0x7f9808101cf0 2024-03-07T07:39:55.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.918+0000 7f980e265700 1 -- 172.21.15.160:0/2945748295 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f98080628a0 con 0x7f9808101cf0 2024-03-07T07:39:55.922 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.919+0000 7f98057fa700 1 -- 172.21.15.160:0/2945748295 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) v1 ==== 136805+0+0 (secure 0 0 0) 0x7f97fc0054a0 con 0x7f9808101cf0 2024-03-07T07:39:55.922 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.920+0000 7f98057fa700 1 --2- 172.21.15.160:0/2945748295 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f97f00a1ea0 0x7f97f00a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:55.922 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.920+0000 7f9807fff700 1 --2- 172.21.15.160:0/2945748295 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f97f00a1ea0 0x7f97f00a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:55.922 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.920+0000 7f98057fa700 1 -- 172.21.15.160:0/2945748295 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f97fc0cf140 con 0x7f9808101cf0 2024-03-07T07:39:55.922 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.920+0000 7f98057fa700 1 --2- 172.21.15.160:0/2945748295 >> [v2:172.21.15.177:6816/1175500850,v1:172.21.15.177:6817/1175500850] conn(0x7f97f00ab550 0x7f97f00ad970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:55.922 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.920+0000 7f98057fa700 1 -- 172.21.15.160:0/2945748295 --> [v2:172.21.15.177:6816/1175500850,v1:172.21.15.177:6817/1175500850] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f97fc00b0b0 con 0x7f97f00ab550 2024-03-07T07:39:55.922 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.920+0000 7f98057fa700 1 -- 172.21.15.160:0/2945748295 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97fc0cf620 con 0x7f9808101cf0 2024-03-07T07:39:55.922 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.920+0000 7f980c802700 1 --2- 172.21.15.160:0/2945748295 >> [v2:172.21.15.177:6816/1175500850,v1:172.21.15.177:6817/1175500850] conn(0x7f97f00ab550 0x7f97f00ad970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:55.923 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.920+0000 7f9807fff700 1 --2- 172.21.15.160:0/2945748295 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f97f00a1ea0 0x7f97f00a4360 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f97f8005fd0 tx=0x7f97f8009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:55.923 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.921+0000 7f980c802700 1 --2- 172.21.15.160:0/2945748295 >> [v2:172.21.15.177:6816/1175500850,v1:172.21.15.177:6817/1175500850] conn(0x7f97f00ab550 0x7f97f00ad970 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:55.923 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.921+0000 7f98057fa700 1 -- 172.21.15.160:0/2945748295 <== osd.7 v2:172.21.15.177:6816/1175500850 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f97fc00b0b0 con 0x7f97f00ab550 2024-03-07T07:39:55.938 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.937+0000 7f980e265700 1 -- 172.21.15.160:0/2945748295 --> [v2:172.21.15.177:6816/1175500850,v1:172.21.15.177:6817/1175500850] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9808062ca0 con 0x7f97f00ab550 2024-03-07T07:39:55.938 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.938+0000 7f98057fa700 1 -- 172.21.15.160:0/2945748295 <== osd.7 v2:172.21.15.177:6816/1175500850 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9808062ca0 con 0x7f97f00ab550 2024-03-07T07:39:55.939 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.938+0000 7f980e265700 1 -- 172.21.15.160:0/2945748295 >> [v2:172.21.15.177:6816/1175500850,v1:172.21.15.177:6817/1175500850] conn(0x7f97f00ab550 msgr2=0x7f97f00ad970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:55.939 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.938+0000 7f980e265700 1 --2- 172.21.15.160:0/2945748295 >> [v2:172.21.15.177:6816/1175500850,v1:172.21.15.177:6817/1175500850] conn(0x7f97f00ab550 0x7f97f00ad970 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:55.939 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.938+0000 7f980e265700 1 -- 172.21.15.160:0/2945748295 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f97f00a1ea0 msgr2=0x7f97f00a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:55.939 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.938+0000 7f980e265700 1 --2- 172.21.15.160:0/2945748295 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f97f00a1ea0 0x7f97f00a4360 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f97f8005fd0 tx=0x7f97f8009310 comp rx=0 tx=0).stop 2024-03-07T07:39:55.939 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.938+0000 7f980e265700 1 -- 172.21.15.160:0/2945748295 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9808101cf0 msgr2=0x7f980819dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:55.940 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.938+0000 7f980e265700 1 --2- 172.21.15.160:0/2945748295 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9808101cf0 0x7f980819dac0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f97fc000c00 tx=0x7f97fc00fb40 comp rx=0 tx=0).stop 2024-03-07T07:39:55.940 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.938+0000 7f980c802700 1 -- 172.21.15.160:0/2945748295 reap_dead start 2024-03-07T07:39:55.940 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.938+0000 7f980e265700 1 -- 172.21.15.160:0/2945748295 shutdown_connections 2024-03-07T07:39:55.940 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.938+0000 7f980e265700 1 -- 172.21.15.160:0/2945748295 >> 172.21.15.160:0/2945748295 conn(0x7f9808077f10 msgr2=0x7f9808100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:55.940 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.938+0000 7f980e265700 1 -- 172.21.15.160:0/2945748295 shutdown_connections 2024-03-07T07:39:55.940 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:55.938+0000 7f980e265700 1 -- 172.21.15.160:0/2945748295 wait complete. 2024-03-07T07:39:56.442 INFO:teuthology.orchestra.run.smithi160.stdout:163208757301 2024-03-07T07:39:56.443 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-07T07:39:56.610 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.608+0000 7f1bf39e6700 1 -- 172.21.15.160:0/3743401055 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1bec102de0 msgr2=0x7f1bec106d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:56.610 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.608+0000 7f1bf39e6700 1 --2- 172.21.15.160:0/3743401055 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1bec102de0 0x7f1bec106d80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc009990 tx=0x7f1bdc009ca0 comp rx=0 tx=0).stop 2024-03-07T07:39:56.610 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.608+0000 7f1bf39e6700 1 -- 172.21.15.160:0/3743401055 shutdown_connections 2024-03-07T07:39:56.610 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.608+0000 7f1bf39e6700 1 --2- 172.21.15.160:0/3743401055 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1bec107460 0x7f1bec10f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:56.610 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.608+0000 7f1bf39e6700 1 --2- 172.21.15.160:0/3743401055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1bec102430 0x7f1bec102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:56.611 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.608+0000 7f1bf39e6700 1 --2- 172.21.15.160:0/3743401055 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1bec102de0 0x7f1bec106d80 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:56.611 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.608+0000 7f1bf39e6700 1 -- 172.21.15.160:0/3743401055 >> 172.21.15.160:0/3743401055 conn(0x7f1bec0791e0 msgr2=0x7f1bec0795f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:56.611 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.608+0000 7f1bf39e6700 1 -- 172.21.15.160:0/3743401055 shutdown_connections 2024-03-07T07:39:56.611 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.608+0000 7f1bf39e6700 1 -- 172.21.15.160:0/3743401055 wait complete. 2024-03-07T07:39:56.611 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.609+0000 7f1bf39e6700 1 Processor -- start 2024-03-07T07:39:56.611 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.609+0000 7f1bf39e6700 1 -- start start 2024-03-07T07:39:56.611 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.609+0000 7f1bf39e6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1bec102430 0x7f1bec19e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:56.611 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.609+0000 7f1bf39e6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1bec102de0 0x7f1bec19eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:56.611 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.609+0000 7f1bf39e6700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1bec107460 0x7f1bec198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:56.611 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.609+0000 7f1bf39e6700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec112420 con 0x7f1bec102430 2024-03-07T07:39:56.611 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.609+0000 7f1bf39e6700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec1122a0 con 0x7f1bec107460 2024-03-07T07:39:56.611 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.609+0000 7f1bf39e6700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec1125a0 con 0x7f1bec102de0 2024-03-07T07:39:56.611 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.609+0000 7f1bf1782700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1bec102430 0x7f1bec19e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:56.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.609+0000 7f1bf0f81700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1bec102de0 0x7f1bec19eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:56.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.609+0000 7f1bf1782700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1bec102430 0x7f1bec19e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:38868/0 (socket says 172.21.15.160:38868) 2024-03-07T07:39:56.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.609+0000 7f1bf0f81700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1bec102de0 0x7f1bec19eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.160:51758/0 (socket says 172.21.15.160:51758) 2024-03-07T07:39:56.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.609+0000 7f1bf1782700 1 -- 172.21.15.160:0/4294640922 learned_addr learned my addr 172.21.15.160:0/4294640922 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:39:56.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.609+0000 7f1bf1f83700 1 --2- 172.21.15.160:0/4294640922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1bec107460 0x7f1bec198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:56.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.609+0000 7f1bf1782700 1 -- 172.21.15.160:0/4294640922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1bec102de0 msgr2=0x7f1bec19eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:56.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.609+0000 7f1bf1782700 1 --2- 172.21.15.160:0/4294640922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1bec102de0 0x7f1bec19eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:56.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.609+0000 7f1bf1782700 1 -- 172.21.15.160:0/4294640922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1bec107460 msgr2=0x7f1bec198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:56.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.609+0000 7f1bf1782700 1 --2- 172.21.15.160:0/4294640922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1bec107460 0x7f1bec198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:56.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.609+0000 7f1bf1782700 1 -- 172.21.15.160:0/4294640922 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bec199020 con 0x7f1bec102430 2024-03-07T07:39:56.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.609+0000 7f1bf0f81700 1 --2- 172.21.15.160:0/4294640922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1bec102de0 0x7f1bec19eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:39:56.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.609+0000 7f1bf1f83700 1 --2- 172.21.15.160:0/4294640922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1bec107460 0x7f1bec198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:39:56.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.610+0000 7f1bf1782700 1 --2- 172.21.15.160:0/4294640922 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1bec102430 0x7f1bec19e620 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1be800e9c0 tx=0x7f1be800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:56.613 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.610+0000 7f1be27fc700 1 -- 172.21.15.160:0/4294640922 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be800cbd0 con 0x7f1bec102430 2024-03-07T07:39:56.613 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.610+0000 7f1bf39e6700 1 -- 172.21.15.160:0/4294640922 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bec1992b0 con 0x7f1bec102430 2024-03-07T07:39:56.613 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.610+0000 7f1be27fc700 1 -- 172.21.15.160:0/4294640922 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1be800ee60 con 0x7f1bec102430 2024-03-07T07:39:56.613 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.610+0000 7f1be27fc700 1 -- 172.21.15.160:0/4294640922 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be8005630 con 0x7f1bec102430 2024-03-07T07:39:56.613 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.610+0000 7f1bf39e6700 1 -- 172.21.15.160:0/4294640922 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bec074550 con 0x7f1bec102430 2024-03-07T07:39:56.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.611+0000 7f1bf39e6700 1 -- 172.21.15.160:0/4294640922 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1bec1995f0 con 0x7f1bec102430 2024-03-07T07:39:56.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.612+0000 7f1be27fc700 1 -- 172.21.15.160:0/4294640922 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) v1 ==== 136805+0+0 (secure 0 0 0) 0x7f1be8004750 con 0x7f1bec102430 2024-03-07T07:39:56.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.613+0000 7f1be27fc700 1 --2- 172.21.15.160:0/4294640922 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f1bd80a1ea0 0x7f1bd80a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:56.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.613+0000 7f1bf0f81700 1 --2- 172.21.15.160:0/4294640922 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f1bd80a1ea0 0x7f1bd80a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:56.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.613+0000 7f1be27fc700 1 -- 172.21.15.160:0/4294640922 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1be80ce620 con 0x7f1bec102430 2024-03-07T07:39:56.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.613+0000 7f1be27fc700 1 --2- 172.21.15.160:0/4294640922 >> [v2:172.21.15.160:6818/3251327001,v1:172.21.15.160:6819/3251327001] conn(0x7f1bd80ab680 0x7f1bd80adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:56.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.613+0000 7f1be27fc700 1 -- 172.21.15.160:0/4294640922 --> [v2:172.21.15.160:6818/3251327001,v1:172.21.15.160:6819/3251327001] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1be801ee50 con 0x7f1bd80ab680 2024-03-07T07:39:56.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.613+0000 7f1be27fc700 1 -- 172.21.15.160:0/4294640922 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1be80ceb00 con 0x7f1bec102430 2024-03-07T07:39:56.615 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.613+0000 7f1bf1f83700 1 --2- 172.21.15.160:0/4294640922 >> [v2:172.21.15.160:6818/3251327001,v1:172.21.15.160:6819/3251327001] conn(0x7f1bd80ab680 0x7f1bd80adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:56.615 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.613+0000 7f1bf0f81700 1 --2- 172.21.15.160:0/4294640922 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f1bd80a1ea0 0x7f1bd80a4360 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc006010 tx=0x7f1bdc005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:56.615 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.613+0000 7f1bf1f83700 1 --2- 172.21.15.160:0/4294640922 >> [v2:172.21.15.160:6818/3251327001,v1:172.21.15.160:6819/3251327001] conn(0x7f1bd80ab680 0x7f1bd80adaa0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:56.615 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.614+0000 7f1be27fc700 1 -- 172.21.15.160:0/4294640922 <== osd.6 v2:172.21.15.160:6818/3251327001 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f1be801ee50 con 0x7f1bd80ab680 2024-03-07T07:39:56.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.633+0000 7f1bf39e6700 1 -- 172.21.15.160:0/4294640922 --> [v2:172.21.15.160:6818/3251327001,v1:172.21.15.160:6819/3251327001] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1bec062af0 con 0x7f1bd80ab680 2024-03-07T07:39:56.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.634+0000 7f1be27fc700 1 -- 172.21.15.160:0/4294640922 <== osd.6 v2:172.21.15.160:6818/3251327001 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f1bec062af0 con 0x7f1bd80ab680 2024-03-07T07:39:56.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.634+0000 7f1bf39e6700 1 -- 172.21.15.160:0/4294640922 >> [v2:172.21.15.160:6818/3251327001,v1:172.21.15.160:6819/3251327001] conn(0x7f1bd80ab680 msgr2=0x7f1bd80adaa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:56.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.634+0000 7f1bf39e6700 1 --2- 172.21.15.160:0/4294640922 >> [v2:172.21.15.160:6818/3251327001,v1:172.21.15.160:6819/3251327001] conn(0x7f1bd80ab680 0x7f1bd80adaa0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:56.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.634+0000 7f1bf39e6700 1 -- 172.21.15.160:0/4294640922 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f1bd80a1ea0 msgr2=0x7f1bd80a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:56.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.634+0000 7f1bf39e6700 1 --2- 172.21.15.160:0/4294640922 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f1bd80a1ea0 0x7f1bd80a4360 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc006010 tx=0x7f1bdc005960 comp rx=0 tx=0).stop 2024-03-07T07:39:56.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.634+0000 7f1bf39e6700 1 -- 172.21.15.160:0/4294640922 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1bec102430 msgr2=0x7f1bec19e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:56.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.634+0000 7f1bf39e6700 1 --2- 172.21.15.160:0/4294640922 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1bec102430 0x7f1bec19e620 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1be800e9c0 tx=0x7f1be800ecd0 comp rx=0 tx=0).stop 2024-03-07T07:39:56.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.634+0000 7f1bf1f83700 1 -- 172.21.15.160:0/4294640922 reap_dead start 2024-03-07T07:39:56.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.635+0000 7f1bf39e6700 1 -- 172.21.15.160:0/4294640922 shutdown_connections 2024-03-07T07:39:56.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.635+0000 7f1bf39e6700 1 -- 172.21.15.160:0/4294640922 >> 172.21.15.160:0/4294640922 conn(0x7f1bec0791e0 msgr2=0x7f1bec1085e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:56.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.635+0000 7f1bf39e6700 1 -- 172.21.15.160:0/4294640922 shutdown_connections 2024-03-07T07:39:56.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.635+0000 7f1bf39e6700 1 -- 172.21.15.160:0/4294640922 wait complete. 2024-03-07T07:39:56.822 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.821+0000 7ffb23953700 1 -- 172.21.15.160:0/3325406328 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb1c107460 msgr2=0x7ffb1c10f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:56.822 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.821+0000 7ffb23953700 1 --2- 172.21.15.160:0/3325406328 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb1c107460 0x7ffb1c10f030 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ffb1400b0a0 tx=0x7ffb1400b3b0 comp rx=0 tx=0).stop 2024-03-07T07:39:56.822 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.821+0000 7ffb23953700 1 -- 172.21.15.160:0/3325406328 shutdown_connections 2024-03-07T07:39:56.822 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.821+0000 7ffb23953700 1 --2- 172.21.15.160:0/3325406328 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb1c107460 0x7ffb1c10f030 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:56.822 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.821+0000 7ffb23953700 1 --2- 172.21.15.160:0/3325406328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffb1c102430 0x7ffb1c102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:56.822 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.821+0000 7ffb23953700 1 --2- 172.21.15.160:0/3325406328 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffb1c102de0 0x7ffb1c106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:56.822 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.821+0000 7ffb23953700 1 -- 172.21.15.160:0/3325406328 >> 172.21.15.160:0/3325406328 conn(0x7ffb1c0791e0 msgr2=0x7ffb1c0795f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:56.822 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.821+0000 7ffb23953700 1 -- 172.21.15.160:0/3325406328 shutdown_connections 2024-03-07T07:39:56.823 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.821+0000 7ffb23953700 1 -- 172.21.15.160:0/3325406328 wait complete. 2024-03-07T07:39:56.823 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.821+0000 7ffb23953700 1 Processor -- start 2024-03-07T07:39:56.823 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.821+0000 7ffb23953700 1 -- start start 2024-03-07T07:39:56.823 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.822+0000 7ffb23953700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffb1c102430 0x7ffb1c19e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:56.823 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.822+0000 7ffb23953700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffb1c102de0 0x7ffb1c19eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:56.823 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.822+0000 7ffb23953700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb1c107460 0x7ffb1c198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:56.823 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.822+0000 7ffb23953700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1c112440 con 0x7ffb1c107460 2024-03-07T07:39:56.823 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.822+0000 7ffb23953700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1c1122c0 con 0x7ffb1c102430 2024-03-07T07:39:56.823 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.822+0000 7ffb23953700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1c1125c0 con 0x7ffb1c102de0 2024-03-07T07:39:56.823 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.822+0000 7ffb216ef700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffb1c102430 0x7ffb1c19e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:56.823 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.822+0000 7ffb216ef700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffb1c102430 0x7ffb1c19e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.160:43324/0 (socket says 172.21.15.160:43324) 2024-03-07T07:39:56.823 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.822+0000 7ffb21ef0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb1c107460 0x7ffb1c198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:56.823 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.822+0000 7ffb20eee700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffb1c102de0 0x7ffb1c19eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:56.823 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.822+0000 7ffb216ef700 1 -- 172.21.15.160:0/782135903 learned_addr learned my addr 172.21.15.160:0/782135903 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:39:56.825 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.822+0000 7ffb216ef700 1 -- 172.21.15.160:0/782135903 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffb1c102de0 msgr2=0x7ffb1c19eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:56.825 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.822+0000 7ffb216ef700 1 --2- 172.21.15.160:0/782135903 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffb1c102de0 0x7ffb1c19eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:56.825 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.822+0000 7ffb216ef700 1 -- 172.21.15.160:0/782135903 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb1c107460 msgr2=0x7ffb1c198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:56.825 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.822+0000 7ffb216ef700 1 --2- 172.21.15.160:0/782135903 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb1c107460 0x7ffb1c198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:56.826 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.822+0000 7ffb216ef700 1 -- 172.21.15.160:0/782135903 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb1c198f60 con 0x7ffb1c102430 2024-03-07T07:39:56.826 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.822+0000 7ffb20eee700 1 --2- 172.21.15.160:0/782135903 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffb1c102de0 0x7ffb1c19eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:39:56.826 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.822+0000 7ffb216ef700 1 --2- 172.21.15.160:0/782135903 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffb1c102430 0x7ffb1c19e5b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ffb1800e9c0 tx=0x7ffb1800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:56.826 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.822+0000 7ffb127fc700 1 -- 172.21.15.160:0/782135903 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1800cbd0 con 0x7ffb1c102430 2024-03-07T07:39:56.826 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.822+0000 7ffb127fc700 1 -- 172.21.15.160:0/782135903 <== mon.2 v2:172.21.15.177:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb1800ee60 con 0x7ffb1c102430 2024-03-07T07:39:56.826 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.822+0000 7ffb23953700 1 -- 172.21.15.160:0/782135903 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb1c199250 con 0x7ffb1c102430 2024-03-07T07:39:56.826 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.822+0000 7ffb127fc700 1 -- 172.21.15.160:0/782135903 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb18005630 con 0x7ffb1c102430 2024-03-07T07:39:56.826 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.823+0000 7ffb23953700 1 -- 172.21.15.160:0/782135903 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb1c1a5530 con 0x7ffb1c102430 2024-03-07T07:39:56.826 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.823+0000 7ffb23953700 1 -- 172.21.15.160:0/782135903 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ffb1c0628a0 con 0x7ffb1c102430 2024-03-07T07:39:56.826 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.824+0000 7ffb127fc700 1 -- 172.21.15.160:0/782135903 <== mon.2 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 17) v1 ==== 136805+0+0 (secure 0 0 0) 0x7ffb18004750 con 0x7ffb1c102430 2024-03-07T07:39:56.826 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.825+0000 7ffb127fc700 1 --2- 172.21.15.160:0/782135903 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7ffb080a1ea0 0x7ffb080a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:56.826 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.825+0000 7ffb20eee700 1 --2- 172.21.15.160:0/782135903 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7ffb080a1ea0 0x7ffb080a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:56.826 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.825+0000 7ffb127fc700 1 -- 172.21.15.160:0/782135903 <== mon.2 v2:172.21.15.177:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ffb180ce2a0 con 0x7ffb1c102430 2024-03-07T07:39:56.826 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.825+0000 7ffb127fc700 1 --2- 172.21.15.160:0/782135903 >> [v2:172.21.15.177:6824/2867473412,v1:172.21.15.177:6825/2867473412] conn(0x7ffb080ab680 0x7ffb080adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:56.827 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.825+0000 7ffb127fc700 1 -- 172.21.15.160:0/782135903 --> [v2:172.21.15.177:6824/2867473412,v1:172.21.15.177:6825/2867473412] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffb080ae150 con 0x7ffb080ab680 2024-03-07T07:39:56.827 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.825+0000 7ffb127fc700 1 -- 172.21.15.160:0/782135903 <== mon.2 v2:172.21.15.177:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb180ce650 con 0x7ffb1c102430 2024-03-07T07:39:56.827 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.825+0000 7ffb21ef0700 1 --2- 172.21.15.160:0/782135903 >> [v2:172.21.15.177:6824/2867473412,v1:172.21.15.177:6825/2867473412] conn(0x7ffb080ab680 0x7ffb080adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:56.827 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.825+0000 7ffb20eee700 1 --2- 172.21.15.160:0/782135903 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7ffb080a1ea0 0x7ffb080a4360 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c009680 tx=0x7ffb0c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:56.827 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.826+0000 7ffb21ef0700 1 --2- 172.21.15.160:0/782135903 >> [v2:172.21.15.177:6824/2867473412,v1:172.21.15.177:6825/2867473412] conn(0x7ffb080ab680 0x7ffb080adaa0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:56.827 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.826+0000 7ffb127fc700 1 -- 172.21.15.160:0/782135903 <== osd.10 v2:172.21.15.177:6824/2867473412 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ffb080ae150 con 0x7ffb080ab680 2024-03-07T07:39:56.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.842+0000 7ffb23953700 1 -- 172.21.15.160:0/782135903 --> [v2:172.21.15.177:6824/2867473412,v1:172.21.15.177:6825/2867473412] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ffb1c062ca0 con 0x7ffb080ab680 2024-03-07T07:39:56.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.842+0000 7ffb127fc700 1 -- 172.21.15.160:0/782135903 <== osd.10 v2:172.21.15.177:6824/2867473412 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ffb1c062ca0 con 0x7ffb080ab680 2024-03-07T07:39:56.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.843+0000 7ffb23953700 1 -- 172.21.15.160:0/782135903 >> [v2:172.21.15.177:6824/2867473412,v1:172.21.15.177:6825/2867473412] conn(0x7ffb080ab680 msgr2=0x7ffb080adaa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:56.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.843+0000 7ffb23953700 1 --2- 172.21.15.160:0/782135903 >> [v2:172.21.15.177:6824/2867473412,v1:172.21.15.177:6825/2867473412] conn(0x7ffb080ab680 0x7ffb080adaa0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:56.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.843+0000 7ffb23953700 1 -- 172.21.15.160:0/782135903 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7ffb080a1ea0 msgr2=0x7ffb080a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:56.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.843+0000 7ffb23953700 1 --2- 172.21.15.160:0/782135903 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7ffb080a1ea0 0x7ffb080a4360 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c009680 tx=0x7ffb0c009260 comp rx=0 tx=0).stop 2024-03-07T07:39:56.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.843+0000 7ffb23953700 1 -- 172.21.15.160:0/782135903 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffb1c102430 msgr2=0x7ffb1c19e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:56.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.843+0000 7ffb23953700 1 --2- 172.21.15.160:0/782135903 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffb1c102430 0x7ffb1c19e5b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ffb1800e9c0 tx=0x7ffb1800ecd0 comp rx=0 tx=0).stop 2024-03-07T07:39:56.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.843+0000 7ffb21ef0700 1 -- 172.21.15.160:0/782135903 reap_dead start 2024-03-07T07:39:56.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.843+0000 7ffb23953700 1 -- 172.21.15.160:0/782135903 shutdown_connections 2024-03-07T07:39:56.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.843+0000 7ffb23953700 1 -- 172.21.15.160:0/782135903 >> 172.21.15.160:0/782135903 conn(0x7ffb1c0791e0 msgr2=0x7ffb1c1001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:56.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.843+0000 7ffb23953700 1 -- 172.21.15.160:0/782135903 shutdown_connections 2024-03-07T07:39:56.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:56.843+0000 7ffb23953700 1 -- 172.21.15.160:0/782135903 wait complete. 2024-03-07T07:39:57.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:57 smithi160 ceph-mon[31963]: pgmap v309: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-03-07T07:39:57.518 INFO:teuthology.orchestra.run.smithi160.stdout:141733920825 2024-03-07T07:39:57.519 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-07T07:39:57.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:57 smithi177 ceph-mon[36578]: pgmap v309: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-03-07T07:39:57.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:57 smithi178 ceph-mon[36650]: pgmap v309: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-03-07T07:39:57.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.956+0000 7fa4b6e0f700 1 -- 172.21.15.160:0/1679402714 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4b0101050 msgr2=0x7fa4b01014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:57.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.956+0000 7fa4b6e0f700 1 --2- 172.21.15.160:0/1679402714 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4b0101050 0x7fa4b01014b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa49c009990 tx=0x7fa49c009ca0 comp rx=0 tx=0).stop 2024-03-07T07:39:57.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.956+0000 7fa4b6e0f700 1 -- 172.21.15.160:0/1679402714 shutdown_connections 2024-03-07T07:39:57.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.956+0000 7fa4b6e0f700 1 --2- 172.21.15.160:0/1679402714 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4b01019f0 0x7fa4b010ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:57.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.956+0000 7fa4b6e0f700 1 --2- 172.21.15.160:0/1679402714 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4b0107090 0x7fa4b0107470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:57.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.956+0000 7fa4b6e0f700 1 --2- 172.21.15.160:0/1679402714 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4b0101050 0x7fa4b01014b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:57.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.956+0000 7fa4b6e0f700 1 -- 172.21.15.160:0/1679402714 >> 172.21.15.160:0/1679402714 conn(0x7fa4b0077f80 msgr2=0x7fa4b0078390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:57.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.956+0000 7fa4b6e0f700 1 -- 172.21.15.160:0/1679402714 shutdown_connections 2024-03-07T07:39:57.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.956+0000 7fa4b6e0f700 1 -- 172.21.15.160:0/1679402714 wait complete. 2024-03-07T07:39:57.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.956+0000 7fa4b6e0f700 1 Processor -- start 2024-03-07T07:39:57.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.956+0000 7fa4b6e0f700 1 -- start start 2024-03-07T07:39:57.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.956+0000 7fa4b6e0f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4b0101050 0x7fa4b019d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:57.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.956+0000 7fa4b6e0f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4b01019f0 0x7fa4b019d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:57.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.956+0000 7fa4b6e0f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4b0107090 0x7fa4b01975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:57.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.956+0000 7fa4b6e0f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b01112f0 con 0x7fa4b0101050 2024-03-07T07:39:57.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.956+0000 7fa4b6e0f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b0111170 con 0x7fa4b01019f0 2024-03-07T07:39:57.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.956+0000 7fa4b6e0f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b0111470 con 0x7fa4b0107090 2024-03-07T07:39:57.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.957+0000 7fa4b4bab700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4b0101050 0x7fa4b019d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:57.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.957+0000 7fa4b4bab700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4b0101050 0x7fa4b019d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:38892/0 (socket says 172.21.15.160:38892) 2024-03-07T07:39:57.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.957+0000 7fa4b4bab700 1 -- 172.21.15.160:0/3211614841 learned_addr learned my addr 172.21.15.160:0/3211614841 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:39:57.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.957+0000 7fa4affff700 1 --2- 172.21.15.160:0/3211614841 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4b01019f0 0x7fa4b019d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:57.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.957+0000 7fa4b53ac700 1 --2- 172.21.15.160:0/3211614841 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4b0107090 0x7fa4b01975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:57.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.957+0000 7fa4b4bab700 1 -- 172.21.15.160:0/3211614841 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4b0107090 msgr2=0x7fa4b01975d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:57.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.957+0000 7fa4b4bab700 1 --2- 172.21.15.160:0/3211614841 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4b0107090 0x7fa4b01975d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:57.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.957+0000 7fa4b4bab700 1 -- 172.21.15.160:0/3211614841 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4b01019f0 msgr2=0x7fa4b019d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:57.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.957+0000 7fa4b4bab700 1 --2- 172.21.15.160:0/3211614841 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4b01019f0 0x7fa4b019d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:57.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.957+0000 7fa4b4bab700 1 -- 172.21.15.160:0/3211614841 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4b0197d30 con 0x7fa4b0101050 2024-03-07T07:39:57.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.957+0000 7fa4affff700 1 --2- 172.21.15.160:0/3211614841 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4b01019f0 0x7fa4b019d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:39:57.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.957+0000 7fa4b4bab700 1 --2- 172.21.15.160:0/3211614841 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4b0101050 0x7fa4b019d410 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa4a400e9a0 tx=0x7fa4a400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:57.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.957+0000 7fa4adffb700 1 -- 172.21.15.160:0/3211614841 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a400ca10 con 0x7fa4b0101050 2024-03-07T07:39:57.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.957+0000 7fa4adffb700 1 -- 172.21.15.160:0/3211614841 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa4a400cbb0 con 0x7fa4b0101050 2024-03-07T07:39:57.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.957+0000 7fa4b6e0f700 1 -- 172.21.15.160:0/3211614841 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4b0198020 con 0x7fa4b0101050 2024-03-07T07:39:57.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.957+0000 7fa4adffb700 1 -- 172.21.15.160:0/3211614841 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a4018680 con 0x7fa4b0101050 2024-03-07T07:39:57.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.957+0000 7fa4b6e0f700 1 -- 172.21.15.160:0/3211614841 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4b0069e00 con 0x7fa4b0101050 2024-03-07T07:39:57.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.958+0000 7fa4b6e0f700 1 -- 172.21.15.160:0/3211614841 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa4b006a3b0 con 0x7fa4b0101050 2024-03-07T07:39:57.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.959+0000 7fa4adffb700 1 -- 172.21.15.160:0/3211614841 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) v1 ==== 136805+0+0 (secure 0 0 0) 0x7fa4a4010430 con 0x7fa4b0101050 2024-03-07T07:39:57.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.960+0000 7fa4adffb700 1 --2- 172.21.15.160:0/3211614841 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa4a00a1ea0 0x7fa4a00a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:57.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.960+0000 7fa4affff700 1 --2- 172.21.15.160:0/3211614841 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa4a00a1ea0 0x7fa4a00a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:57.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.960+0000 7fa4adffb700 1 -- 172.21.15.160:0/3211614841 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa4a40ce770 con 0x7fa4b0101050 2024-03-07T07:39:57.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.960+0000 7fa4adffb700 1 --2- 172.21.15.160:0/3211614841 >> [v2:172.21.15.160:6826/3265797851,v1:172.21.15.160:6827/3265797851] conn(0x7fa4a00ab680 0x7fa4a00adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:57.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.960+0000 7fa4adffb700 1 -- 172.21.15.160:0/3211614841 --> [v2:172.21.15.160:6826/3265797851,v1:172.21.15.160:6827/3265797851] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa4a4019ec0 con 0x7fa4a00ab680 2024-03-07T07:39:57.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.960+0000 7fa4adffb700 1 -- 172.21.15.160:0/3211614841 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4a40cec50 con 0x7fa4b0101050 2024-03-07T07:39:57.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.960+0000 7fa4affff700 1 --2- 172.21.15.160:0/3211614841 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa4a00a1ea0 0x7fa4a00a4360 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa49c00b2e0 tx=0x7fa49c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:57.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.960+0000 7fa4b53ac700 1 --2- 172.21.15.160:0/3211614841 >> [v2:172.21.15.160:6826/3265797851,v1:172.21.15.160:6827/3265797851] conn(0x7fa4a00ab680 0x7fa4a00adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:57.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.960+0000 7fa4b53ac700 1 --2- 172.21.15.160:0/3211614841 >> [v2:172.21.15.160:6826/3265797851,v1:172.21.15.160:6827/3265797851] conn(0x7fa4a00ab680 0x7fa4a00adaa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:57.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.961+0000 7fa4adffb700 1 -- 172.21.15.160:0/3211614841 <== osd.9 v2:172.21.15.160:6826/3265797851 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fa4a4019ec0 con 0x7fa4a00ab680 2024-03-07T07:39:57.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.978+0000 7fa4b6e0f700 1 -- 172.21.15.160:0/3211614841 --> [v2:172.21.15.160:6826/3265797851,v1:172.21.15.160:6827/3265797851] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa4b006a830 con 0x7fa4a00ab680 2024-03-07T07:39:57.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.978+0000 7fa4adffb700 1 -- 172.21.15.160:0/3211614841 <== osd.9 v2:172.21.15.160:6826/3265797851 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa4b006a830 con 0x7fa4a00ab680 2024-03-07T07:39:57.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.979+0000 7fa4b6e0f700 1 -- 172.21.15.160:0/3211614841 >> [v2:172.21.15.160:6826/3265797851,v1:172.21.15.160:6827/3265797851] conn(0x7fa4a00ab680 msgr2=0x7fa4a00adaa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:57.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.979+0000 7fa4b6e0f700 1 --2- 172.21.15.160:0/3211614841 >> [v2:172.21.15.160:6826/3265797851,v1:172.21.15.160:6827/3265797851] conn(0x7fa4a00ab680 0x7fa4a00adaa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:57.980 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.979+0000 7fa4b6e0f700 1 -- 172.21.15.160:0/3211614841 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa4a00a1ea0 msgr2=0x7fa4a00a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:57.980 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.979+0000 7fa4b6e0f700 1 --2- 172.21.15.160:0/3211614841 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa4a00a1ea0 0x7fa4a00a4360 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa49c00b2e0 tx=0x7fa49c005960 comp rx=0 tx=0).stop 2024-03-07T07:39:57.980 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.979+0000 7fa4b6e0f700 1 -- 172.21.15.160:0/3211614841 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4b0101050 msgr2=0x7fa4b019d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:57.980 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.979+0000 7fa4b6e0f700 1 --2- 172.21.15.160:0/3211614841 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4b0101050 0x7fa4b019d410 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa4a400e9a0 tx=0x7fa4a400ecb0 comp rx=0 tx=0).stop 2024-03-07T07:39:57.980 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.979+0000 7fa4b53ac700 1 -- 172.21.15.160:0/3211614841 reap_dead start 2024-03-07T07:39:57.980 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.979+0000 7fa4b6e0f700 1 -- 172.21.15.160:0/3211614841 shutdown_connections 2024-03-07T07:39:57.980 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.979+0000 7fa4b6e0f700 1 -- 172.21.15.160:0/3211614841 >> 172.21.15.160:0/3211614841 conn(0x7fa4b0077f80 msgr2=0x7fa4b00ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:57.980 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.979+0000 7fa4b6e0f700 1 -- 172.21.15.160:0/3211614841 shutdown_connections 2024-03-07T07:39:57.980 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:57.979+0000 7fa4b6e0f700 1 -- 172.21.15.160:0/3211614841 wait complete. 2024-03-07T07:39:59.316 INFO:teuthology.orchestra.run.smithi160.stdout:309237645339 2024-03-07T07:39:59.317 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-03-07T07:39:59.332 INFO:teuthology.orchestra.run.smithi160.stdout:219043332138 2024-03-07T07:39:59.332 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-07T07:39:59.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:39:59 smithi177 ceph-mon[36578]: pgmap v310: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-07T07:39:59.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.497+0000 7fbc83b6f700 1 -- 172.21.15.160:0/3985614032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbc7c103050 msgr2=0x7fbc7c103430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:59.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.497+0000 7fbc83b6f700 1 --2- 172.21.15.160:0/3985614032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbc7c103050 0x7fbc7c103430 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c009990 tx=0x7fbc6c009ca0 comp rx=0 tx=0).stop 2024-03-07T07:39:59.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.497+0000 7fbc83b6f700 1 -- 172.21.15.160:0/3985614032 shutdown_connections 2024-03-07T07:39:59.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.497+0000 7fbc83b6f700 1 --2- 172.21.15.160:0/3985614032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc7c101ab0 0x7fbc7c10e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:59.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.497+0000 7fbc83b6f700 1 --2- 172.21.15.160:0/3985614032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbc7c103050 0x7fbc7c103430 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:59.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.497+0000 7fbc83b6f700 1 --2- 172.21.15.160:0/3985614032 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc7c1010f0 0x7fbc7c101570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:59.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.497+0000 7fbc83b6f700 1 -- 172.21.15.160:0/3985614032 >> 172.21.15.160:0/3985614032 conn(0x7fbc7c0770f0 msgr2=0x7fbc7c077500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:59.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.497+0000 7fbc83b6f700 1 -- 172.21.15.160:0/3985614032 shutdown_connections 2024-03-07T07:39:59.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.497+0000 7fbc83b6f700 1 -- 172.21.15.160:0/3985614032 wait complete. 2024-03-07T07:39:59.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.498+0000 7fbc83b6f700 1 Processor -- start 2024-03-07T07:39:59.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.498+0000 7fbc83b6f700 1 -- start start 2024-03-07T07:39:59.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.498+0000 7fbc83b6f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbc7c1010f0 0x7fbc7c19e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:59.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.498+0000 7fbc83b6f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc7c101ab0 0x7fbc7c19eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:59.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.498+0000 7fbc83b6f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc7c103050 0x7fbc7c1987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:59.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.498+0000 7fbc83b6f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7c111470 con 0x7fbc7c1010f0 2024-03-07T07:39:59.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.498+0000 7fbc83b6f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7c1112f0 con 0x7fbc7c103050 2024-03-07T07:39:59.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.498+0000 7fbc83b6f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7c1115f0 con 0x7fbc7c101ab0 2024-03-07T07:39:59.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.498+0000 7fbc8190b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbc7c1010f0 0x7fbc7c19e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:59.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.498+0000 7fbc8190b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbc7c1010f0 0x7fbc7c19e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:38782/0 (socket says 172.21.15.160:38782) 2024-03-07T07:39:59.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.498+0000 7fbc8190b700 1 -- 172.21.15.160:0/3232324137 learned_addr learned my addr 172.21.15.160:0/3232324137 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:39:59.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.498+0000 7fbc8110a700 1 --2- 172.21.15.160:0/3232324137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc7c101ab0 0x7fbc7c19eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:59.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.498+0000 7fbc8210c700 1 --2- 172.21.15.160:0/3232324137 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc7c103050 0x7fbc7c1987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:59.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.498+0000 7fbc8190b700 1 -- 172.21.15.160:0/3232324137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc7c101ab0 msgr2=0x7fbc7c19eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:59.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.499+0000 7fbc8190b700 1 --2- 172.21.15.160:0/3232324137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc7c101ab0 0x7fbc7c19eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:59.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.499+0000 7fbc8190b700 1 -- 172.21.15.160:0/3232324137 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc7c103050 msgr2=0x7fbc7c1987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:59.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.499+0000 7fbc8190b700 1 --2- 172.21.15.160:0/3232324137 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc7c103050 0x7fbc7c1987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:59.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.499+0000 7fbc8190b700 1 -- 172.21.15.160:0/3232324137 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc7c198fc0 con 0x7fbc7c1010f0 2024-03-07T07:39:59.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.499+0000 7fbc8210c700 1 --2- 172.21.15.160:0/3232324137 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc7c103050 0x7fbc7c1987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:39:59.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.499+0000 7fbc8190b700 1 --2- 172.21.15.160:0/3232324137 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbc7c1010f0 0x7fbc7c19e660 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c00bae0 tx=0x7fbc6c00bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:59.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.499+0000 7fbc72ffd700 1 -- 172.21.15.160:0/3232324137 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c026070 con 0x7fbc7c1010f0 2024-03-07T07:39:59.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.499+0000 7fbc72ffd700 1 -- 172.21.15.160:0/3232324137 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbc6c003d70 con 0x7fbc7c1010f0 2024-03-07T07:39:59.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.499+0000 7fbc83b6f700 1 -- 172.21.15.160:0/3232324137 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc7c199250 con 0x7fbc7c1010f0 2024-03-07T07:39:59.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.499+0000 7fbc72ffd700 1 -- 172.21.15.160:0/3232324137 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c021410 con 0x7fbc7c1010f0 2024-03-07T07:39:59.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.499+0000 7fbc83b6f700 1 -- 172.21.15.160:0/3232324137 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc7c1a54c0 con 0x7fbc7c1010f0 2024-03-07T07:39:59.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.500+0000 7fbc83b6f700 1 -- 172.21.15.160:0/3232324137 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbc7c1072a0 con 0x7fbc7c1010f0 2024-03-07T07:39:59.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.501+0000 7fbc72ffd700 1 -- 172.21.15.160:0/3232324137 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) v1 ==== 136805+0+0 (secure 0 0 0) 0x7fbc6c029070 con 0x7fbc7c1010f0 2024-03-07T07:39:59.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.501+0000 7fbc72ffd700 1 --2- 172.21.15.160:0/3232324137 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fbc680a1ea0 0x7fbc680a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:59.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.501+0000 7fbc8110a700 1 --2- 172.21.15.160:0/3232324137 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fbc680a1ea0 0x7fbc680a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:59.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.501+0000 7fbc72ffd700 1 -- 172.21.15.160:0/3232324137 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fbc6c0cf2e0 con 0x7fbc7c1010f0 2024-03-07T07:39:59.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.502+0000 7fbc72ffd700 1 --2- 172.21.15.160:0/3232324137 >> [v2:172.21.15.160:6810/42543989,v1:172.21.15.160:6811/42543989] conn(0x7fbc680ab680 0x7fbc680adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:39:59.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.502+0000 7fbc72ffd700 1 -- 172.21.15.160:0/3232324137 --> [v2:172.21.15.160:6810/42543989,v1:172.21.15.160:6811/42543989] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbc6c02be50 con 0x7fbc680ab680 2024-03-07T07:39:59.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.502+0000 7fbc72ffd700 1 -- 172.21.15.160:0/3232324137 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc6c0cf7c0 con 0x7fbc7c1010f0 2024-03-07T07:39:59.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.502+0000 7fbc8210c700 1 --2- 172.21.15.160:0/3232324137 >> [v2:172.21.15.160:6810/42543989,v1:172.21.15.160:6811/42543989] conn(0x7fbc680ab680 0x7fbc680adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:39:59.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.502+0000 7fbc8110a700 1 --2- 172.21.15.160:0/3232324137 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fbc680a1ea0 0x7fbc680a4360 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbc78009680 tx=0x7fbc78009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:59.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.502+0000 7fbc8210c700 1 --2- 172.21.15.160:0/3232324137 >> [v2:172.21.15.160:6810/42543989,v1:172.21.15.160:6811/42543989] conn(0x7fbc680ab680 0x7fbc680adaa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:39:59.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.503+0000 7fbc72ffd700 1 -- 172.21.15.160:0/3232324137 <== osd.3 v2:172.21.15.160:6810/42543989 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fbc6c02be50 con 0x7fbc680ab680 2024-03-07T07:39:59.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.519+0000 7fbc83b6f700 1 -- 172.21.15.160:0/3232324137 --> [v2:172.21.15.160:6810/42543989,v1:172.21.15.160:6811/42543989] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbc7c1027b0 con 0x7fbc680ab680 2024-03-07T07:39:59.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.519+0000 7fbc72ffd700 1 -- 172.21.15.160:0/3232324137 <== osd.3 v2:172.21.15.160:6810/42543989 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fbc7c1027b0 con 0x7fbc680ab680 2024-03-07T07:39:59.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.520+0000 7fbc83b6f700 1 -- 172.21.15.160:0/3232324137 >> [v2:172.21.15.160:6810/42543989,v1:172.21.15.160:6811/42543989] conn(0x7fbc680ab680 msgr2=0x7fbc680adaa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:59.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.520+0000 7fbc83b6f700 1 --2- 172.21.15.160:0/3232324137 >> [v2:172.21.15.160:6810/42543989,v1:172.21.15.160:6811/42543989] conn(0x7fbc680ab680 0x7fbc680adaa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:39:59.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.520+0000 7fbc83b6f700 1 -- 172.21.15.160:0/3232324137 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fbc680a1ea0 msgr2=0x7fbc680a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:59.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.520+0000 7fbc83b6f700 1 --2- 172.21.15.160:0/3232324137 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fbc680a1ea0 0x7fbc680a4360 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbc78009680 tx=0x7fbc78009260 comp rx=0 tx=0).stop 2024-03-07T07:39:59.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.520+0000 7fbc83b6f700 1 -- 172.21.15.160:0/3232324137 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbc7c1010f0 msgr2=0x7fbc7c19e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:39:59.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.520+0000 7fbc83b6f700 1 --2- 172.21.15.160:0/3232324137 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbc7c1010f0 0x7fbc7c19e660 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c00bae0 tx=0x7fbc6c00bb10 comp rx=0 tx=0).stop 2024-03-07T07:39:59.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.520+0000 7fbc8210c700 1 -- 172.21.15.160:0/3232324137 reap_dead start 2024-03-07T07:39:59.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.520+0000 7fbc83b6f700 1 -- 172.21.15.160:0/3232324137 shutdown_connections 2024-03-07T07:39:59.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.520+0000 7fbc83b6f700 1 -- 172.21.15.160:0/3232324137 >> 172.21.15.160:0/3232324137 conn(0x7fbc7c0770f0 msgr2=0x7fbc7c06aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:39:59.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.520+0000 7fbc83b6f700 1 -- 172.21.15.160:0/3232324137 shutdown_connections 2024-03-07T07:39:59.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:39:52.520+0000 7fbc83b6f700 1 -- 172.21.15.160:0/3232324137 wait complete. 2024-03-07T07:39:59.638 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:39:59 smithi160 ceph-mon[31963]: pgmap v310: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-07T07:39:59.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:39:59 smithi178 ceph-mon[36650]: pgmap v310: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-07T07:40:00.224 INFO:teuthology.orchestra.run.smithi160.stdout:107374182462 2024-03-07T07:40:00.224 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-07T07:40:00.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:00 smithi177 ceph-mon[36578]: overall HEALTH_OK 2024-03-07T07:40:00.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:00 smithi160 ceph-mon[31963]: overall HEALTH_OK 2024-03-07T07:40:00.719 INFO:teuthology.orchestra.run.smithi160.stdout:343597383702 2024-03-07T07:40:00.720 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-03-07T07:40:00.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:00 smithi178 ceph-mon[36650]: overall HEALTH_OK 2024-03-07T07:40:01.159 INFO:teuthology.orchestra.run.smithi160.stdout:197568495663 2024-03-07T07:40:01.159 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-07T07:40:01.280 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:40:01.397 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:01 smithi160 ceph-mon[31963]: pgmap v311: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-07T07:40:01.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:01 smithi177 ceph-mon[36578]: pgmap v311: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-07T07:40:01.734 INFO:teuthology.orchestra.run.smithi160.stdout:287762808864 2024-03-07T07:40:01.735 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-03-07T07:40:01.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:01 smithi178 ceph-mon[36650]: pgmap v311: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-07T07:40:02.211 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:40:02.216 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:40:02.219 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:40:02.264 INFO:teuthology.orchestra.run.smithi160.stdout:55834574920 2024-03-07T07:40:02.264 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-07T07:40:03.595 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:03 smithi160 ceph-mon[31963]: pgmap v312: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-07T07:40:03.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:03 smithi177 ceph-mon[36578]: pgmap v312: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-07T07:40:03.613 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:40:03.614 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:40:03.614 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:40:03.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:03 smithi178 ceph-mon[36650]: pgmap v312: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-07T07:40:04.036 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:40:04.049 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:40:05.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:05 smithi177 ceph-mon[36578]: pgmap v313: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-07T07:40:05.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:05 smithi160 ceph-mon[31963]: pgmap v313: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-07T07:40:05.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:05 smithi178 ceph-mon[36650]: pgmap v313: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-07T07:40:07.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:07 smithi177 ceph-mon[36578]: pgmap v314: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-07T07:40:07.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:07 smithi160 ceph-mon[31963]: pgmap v314: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-07T07:40:07.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:07 smithi178 ceph-mon[36650]: pgmap v314: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-07T07:40:08.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:08 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-03-07T07:40:08.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:08 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-03-07T07:40:08.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:08 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-03-07T07:40:08.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:08 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-03-07T07:40:08.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:08 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]: dispatch 2024-03-07T07:40:08.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:08 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]: dispatch 2024-03-07T07:40:08.429 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:08 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-03-07T07:40:08.429 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:08 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-03-07T07:40:08.429 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:08 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-03-07T07:40:08.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:08 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-03-07T07:40:08.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:08 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-03-07T07:40:08.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:08 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-03-07T07:40:08.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:08 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-03-07T07:40:08.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:08 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]: dispatch 2024-03-07T07:40:08.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:08 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]: dispatch 2024-03-07T07:40:08.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:08 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-03-07T07:40:08.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:08 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-03-07T07:40:08.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:08 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-03-07T07:40:08.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:08 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-03-07T07:40:08.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:08 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-03-07T07:40:08.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:08 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-03-07T07:40:08.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:08 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-03-07T07:40:08.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:08 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]: dispatch 2024-03-07T07:40:08.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:08 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]: dispatch 2024-03-07T07:40:08.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:08 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-03-07T07:40:08.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:08 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-03-07T07:40:08.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:08 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-03-07T07:40:09.477 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:40:09.645 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:09 smithi160 ceph-mon[31963]: pgmap v315: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-07T07:40:09.646 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:09 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-03-07T07:40:09.646 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:09 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-03-07T07:40:09.646 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:09 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-03-07T07:40:09.646 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:09 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-03-07T07:40:09.646 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:09 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]': finished 2024-03-07T07:40:09.646 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:09 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2024-03-07T07:40:09.646 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:09 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-03-07T07:40:09.646 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:09 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-03-07T07:40:09.646 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:09 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-03-07T07:40:09.646 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:09 smithi160 ceph-mon[31963]: osdmap e86: 12 total, 12 up, 12 in 2024-03-07T07:40:09.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:09 smithi178 ceph-mon[36650]: pgmap v315: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-07T07:40:09.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:09 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-03-07T07:40:09.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:09 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-03-07T07:40:09.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:09 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-03-07T07:40:09.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:09 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-03-07T07:40:09.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:09 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]': finished 2024-03-07T07:40:09.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:09 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2024-03-07T07:40:09.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:09 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-03-07T07:40:09.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:09 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-03-07T07:40:09.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:09 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-03-07T07:40:09.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:09 smithi178 ceph-mon[36650]: osdmap e86: 12 total, 12 up, 12 in 2024-03-07T07:40:09.843 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:40:09.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:09 smithi177 ceph-mon[36578]: pgmap v315: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-07T07:40:09.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:09 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-03-07T07:40:09.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:09 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-03-07T07:40:09.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:09 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-03-07T07:40:09.858 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:09 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-03-07T07:40:09.858 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:09 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]': finished 2024-03-07T07:40:09.858 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:09 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2024-03-07T07:40:09.858 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:09 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-03-07T07:40:09.858 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:09 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-03-07T07:40:09.858 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:09 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-03-07T07:40:09.858 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:09 smithi177 ceph-mon[36578]: osdmap e86: 12 total, 12 up, 12 in 2024-03-07T07:40:10.636 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:10 smithi160 ceph-mon[31963]: osdmap e87: 12 total, 12 up, 12 in 2024-03-07T07:40:10.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:10 smithi178 ceph-mon[36650]: osdmap e87: 12 total, 12 up, 12 in 2024-03-07T07:40:10.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:10 smithi177 ceph-mon[36578]: osdmap e87: 12 total, 12 up, 12 in 2024-03-07T07:40:10.864 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:10.863+0000 7f534eda2700 1 -- 172.21.15.160:0/780660668 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5348106370 msgr2=0x7f534810df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:10.864 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:10.863+0000 7f534eda2700 1 --2- 172.21.15.160:0/780660668 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5348106370 0x7f534810df40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f534000b0a0 tx=0x7f534000b3b0 comp rx=0 tx=0).stop 2024-03-07T07:40:10.864 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:10.863+0000 7f534eda2700 1 -- 172.21.15.160:0/780660668 shutdown_connections 2024-03-07T07:40:10.864 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:10.863+0000 7f534eda2700 1 --2- 172.21.15.160:0/780660668 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5348106370 0x7f534810df40 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:10.865 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:10.863+0000 7f534eda2700 1 --2- 172.21.15.160:0/780660668 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53481012e0 0x7f53481016c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:10.865 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:10.863+0000 7f534eda2700 1 --2- 172.21.15.160:0/780660668 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5348101c90 0x7f5348105c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:10.865 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:10.863+0000 7f534eda2700 1 -- 172.21.15.160:0/780660668 >> 172.21.15.160:0/780660668 conn(0x7f5348077070 msgr2=0x7f5348077480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:10.865 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:10.864+0000 7f534eda2700 1 -- 172.21.15.160:0/780660668 shutdown_connections 2024-03-07T07:40:10.865 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:10.864+0000 7f534eda2700 1 -- 172.21.15.160:0/780660668 wait complete. 2024-03-07T07:40:10.865 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:10.864+0000 7f534eda2700 1 Processor -- start 2024-03-07T07:40:10.865 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:10.864+0000 7f534eda2700 1 -- start start 2024-03-07T07:40:10.866 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:10.864+0000 7f534eda2700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53481012e0 0x7f5348102d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:10.866 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:10.864+0000 7f534eda2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5348101c90 0x7f5348103270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:10.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:10.864+0000 7f534eda2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5348106370 0x7f53481037b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:10.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:10.864+0000 7f534eda2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5348111350 con 0x7f5348101c90 2024-03-07T07:40:10.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:10.864+0000 7f534eda2700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53481111d0 con 0x7f53481012e0 2024-03-07T07:40:10.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:10.864+0000 7f534eda2700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53481114d0 con 0x7f5348106370 2024-03-07T07:40:10.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:10.864+0000 7f5347fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5348101c90 0x7f5348103270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:10.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:10.865+0000 7f5347fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5348101c90 0x7f5348103270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41230/0 (socket says 172.21.15.160:41230) 2024-03-07T07:40:10.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:10.865+0000 7f5347fff700 1 -- 172.21.15.160:0/3846364957 learned_addr learned my addr 172.21.15.160:0/3846364957 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:40:10.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:10.865+0000 7f534d33f700 1 --2- 172.21.15.160:0/3846364957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5348106370 0x7f53481037b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:10.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:10.865+0000 7f534cb3e700 1 --2- 172.21.15.160:0/3846364957 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53481012e0 0x7f5348102d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:10.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:10.865+0000 7f5347fff700 1 -- 172.21.15.160:0/3846364957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5348106370 msgr2=0x7f53481037b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:10.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:10.865+0000 7f5347fff700 1 --2- 172.21.15.160:0/3846364957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5348106370 0x7f53481037b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:10.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:10.865+0000 7f5347fff700 1 -- 172.21.15.160:0/3846364957 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53481012e0 msgr2=0x7f5348102d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:10.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:10.865+0000 7f5347fff700 1 --2- 172.21.15.160:0/3846364957 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53481012e0 0x7f5348102d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:10.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:10.865+0000 7f5347fff700 1 -- 172.21.15.160:0/3846364957 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53481a6e30 con 0x7f5348101c90 2024-03-07T07:40:10.868 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:10.865+0000 7f5347fff700 1 --2- 172.21.15.160:0/3846364957 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5348101c90 0x7f5348103270 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f533400e9f0 tx=0x7f533400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:10.868 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:10.865+0000 7f5345ffb700 1 -- 172.21.15.160:0/3846364957 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5334004a30 con 0x7f5348101c90 2024-03-07T07:40:10.868 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:10.865+0000 7f5345ffb700 1 -- 172.21.15.160:0/3846364957 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f533400cca0 con 0x7f5348101c90 2024-03-07T07:40:10.868 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:10.865+0000 7f534eda2700 1 -- 172.21.15.160:0/3846364957 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53481a7120 con 0x7f5348101c90 2024-03-07T07:40:10.868 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:10.865+0000 7f5345ffb700 1 -- 172.21.15.160:0/3846364957 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53340056c0 con 0x7f5348101c90 2024-03-07T07:40:10.868 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:10.865+0000 7f534eda2700 1 -- 172.21.15.160:0/3846364957 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53481a7610 con 0x7f5348101c90 2024-03-07T07:40:10.868 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:10.866+0000 7f534eda2700 1 -- 172.21.15.160:0/3846364957 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5348102770 con 0x7f5348101c90 2024-03-07T07:40:10.869 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:10.867+0000 7f5345ffb700 1 -- 172.21.15.160:0/3846364957 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) v1 ==== 136805+0+0 (secure 0 0 0) 0x7f533400c7c0 con 0x7f5348101c90 2024-03-07T07:40:10.869 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:10.868+0000 7f5345ffb700 1 --2- 172.21.15.160:0/3846364957 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f53380a1ea0 0x7f53380a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:10.869 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:10.868+0000 7f534cb3e700 1 --2- 172.21.15.160:0/3846364957 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f53380a1ea0 0x7f53380a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:10.869 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:10.868+0000 7f5345ffb700 1 -- 172.21.15.160:0/3846364957 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f53340cdf90 con 0x7f5348101c90 2024-03-07T07:40:10.869 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:10.869+0000 7f534cb3e700 1 --2- 172.21.15.160:0/3846364957 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f53380a1ea0 0x7f53380a4360 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f533c009c60 tx=0x7f533c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:10.873 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:10.873+0000 7f5345ffb700 1 -- 172.21.15.160:0/3846364957 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f533409b320 con 0x7f5348101c90 2024-03-07T07:40:11.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:11.081+0000 7f534eda2700 1 -- 172.21.15.160:0/3846364957 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f53481a78d0 con 0x7f5348101c90 2024-03-07T07:40:11.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:11.082+0000 7f5345ffb700 1 -- 172.21.15.160:0/3846364957 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5334085070 con 0x7f5348101c90 2024-03-07T07:40:11.083 INFO:teuthology.orchestra.run.smithi160.stdout:253403070504 2024-03-07T07:40:11.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:11.084+0000 7f534eda2700 1 -- 172.21.15.160:0/3846364957 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f53380a1ea0 msgr2=0x7f53380a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:11.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:11.084+0000 7f534eda2700 1 --2- 172.21.15.160:0/3846364957 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f53380a1ea0 0x7f53380a4360 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f533c009c60 tx=0x7f533c009260 comp rx=0 tx=0).stop 2024-03-07T07:40:11.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:11.084+0000 7f534eda2700 1 -- 172.21.15.160:0/3846364957 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5348101c90 msgr2=0x7f5348103270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:11.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:11.084+0000 7f534eda2700 1 --2- 172.21.15.160:0/3846364957 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5348101c90 0x7f5348103270 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f533400e9f0 tx=0x7f533400ed00 comp rx=0 tx=0).stop 2024-03-07T07:40:11.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:11.084+0000 7f534eda2700 1 -- 172.21.15.160:0/3846364957 shutdown_connections 2024-03-07T07:40:11.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:11.084+0000 7f534eda2700 1 --2- 172.21.15.160:0/3846364957 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f53380a1ea0 0x7f53380a4360 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:11.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:11.084+0000 7f534eda2700 1 --2- 172.21.15.160:0/3846364957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5348106370 0x7f53481037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:11.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:11.084+0000 7f534eda2700 1 --2- 172.21.15.160:0/3846364957 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53481012e0 0x7f5348102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:11.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:11.084+0000 7f534eda2700 1 --2- 172.21.15.160:0/3846364957 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5348101c90 0x7f5348103270 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:11.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:11.084+0000 7f534eda2700 1 -- 172.21.15.160:0/3846364957 >> 172.21.15.160:0/3846364957 conn(0x7f5348077070 msgr2=0x7f5348105050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:11.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:11.084+0000 7f534eda2700 1 -- 172.21.15.160:0/3846364957 shutdown_connections 2024-03-07T07:40:11.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:11.084+0000 7f534eda2700 1 -- 172.21.15.160:0/3846364957 wait complete. 2024-03-07T07:40:11.611 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:11 smithi160 ceph-mon[31963]: pgmap v318: 97 pgs: 1 peering, 96 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 7.2 KiB/s, 0 objects/s recovering 2024-03-07T07:40:11.611 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:11 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/3846364957' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-07T07:40:11.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:11 smithi178 ceph-mon[36650]: pgmap v318: 97 pgs: 1 peering, 96 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 7.2 KiB/s, 0 objects/s recovering 2024-03-07T07:40:11.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:11 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/3846364957' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-07T07:40:11.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:11 smithi177 ceph-mon[36578]: pgmap v318: 97 pgs: 1 peering, 96 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 7.2 KiB/s, 0 objects/s recovering 2024-03-07T07:40:11.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:11 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/3846364957' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-07T07:40:12.034 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.033+0000 7fc7a6e52700 1 -- 172.21.15.160:0/3014820382 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7a0102de0 msgr2=0x7fc7a0106d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:12.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.033+0000 7fc7a6e52700 1 --2- 172.21.15.160:0/3014820382 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7a0102de0 0x7fc7a0106d80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc78c009990 tx=0x7fc78c009ca0 comp rx=0 tx=0).stop 2024-03-07T07:40:12.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.033+0000 7fc7a6e52700 1 -- 172.21.15.160:0/3014820382 shutdown_connections 2024-03-07T07:40:12.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.033+0000 7fc7a6e52700 1 --2- 172.21.15.160:0/3014820382 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7a0107460 0x7fc7a010f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:12.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.033+0000 7fc7a6e52700 1 --2- 172.21.15.160:0/3014820382 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7a0102430 0x7fc7a0102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:12.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.033+0000 7fc7a6e52700 1 --2- 172.21.15.160:0/3014820382 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7a0102de0 0x7fc7a0106d80 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:12.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.033+0000 7fc7a6e52700 1 -- 172.21.15.160:0/3014820382 >> 172.21.15.160:0/3014820382 conn(0x7fc7a00791e0 msgr2=0x7fc7a00795f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:12.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.033+0000 7fc7a6e52700 1 -- 172.21.15.160:0/3014820382 shutdown_connections 2024-03-07T07:40:12.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.033+0000 7fc7a6e52700 1 -- 172.21.15.160:0/3014820382 wait complete. 2024-03-07T07:40:12.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.034+0000 7fc7a6e52700 1 Processor -- start 2024-03-07T07:40:12.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.034+0000 7fc7a6e52700 1 -- start start 2024-03-07T07:40:12.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.034+0000 7fc7a6e52700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7a0102430 0x7fc7a0074450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:12.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.034+0000 7fc7a6e52700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7a0102de0 0x7fc7a0074990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:12.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.034+0000 7fc7a6e52700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7a0107460 0x7fc7a0074ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:12.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.034+0000 7fc7a6e52700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a0112400 con 0x7fc7a0102430 2024-03-07T07:40:12.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.034+0000 7fc7a6e52700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a0112280 con 0x7fc7a0107460 2024-03-07T07:40:12.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.034+0000 7fc7a6e52700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a0112580 con 0x7fc7a0102de0 2024-03-07T07:40:12.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.034+0000 7fc7a4bee700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7a0102430 0x7fc7a0074450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:12.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.034+0000 7fc7a4bee700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7a0102430 0x7fc7a0074450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41246/0 (socket says 172.21.15.160:41246) 2024-03-07T07:40:12.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.034+0000 7fc7a4bee700 1 -- 172.21.15.160:0/2316795275 learned_addr learned my addr 172.21.15.160:0/2316795275 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:40:12.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.034+0000 7fc79ffff700 1 --2- 172.21.15.160:0/2316795275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7a0102de0 0x7fc7a0074990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:12.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.034+0000 7fc7a53ef700 1 --2- 172.21.15.160:0/2316795275 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7a0107460 0x7fc7a0074ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:12.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.034+0000 7fc7a4bee700 1 -- 172.21.15.160:0/2316795275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7a0102de0 msgr2=0x7fc7a0074990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:12.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.034+0000 7fc7a4bee700 1 --2- 172.21.15.160:0/2316795275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7a0102de0 0x7fc7a0074990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:12.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.034+0000 7fc7a4bee700 1 -- 172.21.15.160:0/2316795275 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7a0107460 msgr2=0x7fc7a0074ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:12.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.034+0000 7fc7a4bee700 1 --2- 172.21.15.160:0/2316795275 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7a0107460 0x7fc7a0074ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:12.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.035+0000 7fc7a4bee700 1 -- 172.21.15.160:0/2316795275 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a01a6e10 con 0x7fc7a0102430 2024-03-07T07:40:12.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.035+0000 7fc7a4bee700 1 --2- 172.21.15.160:0/2316795275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7a0102430 0x7fc7a0074450 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc79400e9a0 tx=0x7fc79400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:12.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.035+0000 7fc79dffb700 1 -- 172.21.15.160:0/2316795275 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79400cbd0 con 0x7fc7a0102430 2024-03-07T07:40:12.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.035+0000 7fc7a6e52700 1 -- 172.21.15.160:0/2316795275 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7a01a7100 con 0x7fc7a0102430 2024-03-07T07:40:12.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.035+0000 7fc79dffb700 1 -- 172.21.15.160:0/2316795275 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc79400cd70 con 0x7fc7a0102430 2024-03-07T07:40:12.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.035+0000 7fc79dffb700 1 -- 172.21.15.160:0/2316795275 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc794018750 con 0x7fc7a0102430 2024-03-07T07:40:12.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.035+0000 7fc7a6e52700 1 -- 172.21.15.160:0/2316795275 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7a01a7610 con 0x7fc7a0102430 2024-03-07T07:40:12.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.036+0000 7fc7a6e52700 1 -- 172.21.15.160:0/2316795275 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7a0103860 con 0x7fc7a0102430 2024-03-07T07:40:12.040 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.039+0000 7fc79dffb700 1 -- 172.21.15.160:0/2316795275 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) v1 ==== 136805+0+0 (secure 0 0 0) 0x7fc794020060 con 0x7fc7a0102430 2024-03-07T07:40:12.041 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.039+0000 7fc79dffb700 1 --2- 172.21.15.160:0/2316795275 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fc7900a1ea0 0x7fc7900a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:12.041 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.040+0000 7fc79ffff700 1 --2- 172.21.15.160:0/2316795275 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fc7900a1ea0 0x7fc7900a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:12.041 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.040+0000 7fc79dffb700 1 -- 172.21.15.160:0/2316795275 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc7940ceb30 con 0x7fc7a0102430 2024-03-07T07:40:12.041 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.040+0000 7fc79ffff700 1 --2- 172.21.15.160:0/2316795275 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fc7900a1ea0 0x7fc7900a4360 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc78c006010 tx=0x7fc78c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:12.044 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.043+0000 7fc79dffb700 1 -- 172.21.15.160:0/2316795275 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc794027070 con 0x7fc7a0102430 2024-03-07T07:40:12.044 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:40:12.248 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.246+0000 7fc7a6e52700 1 -- 172.21.15.160:0/2316795275 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc7a002d9f0 con 0x7fc7a0102430 2024-03-07T07:40:12.248 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.247+0000 7fc79dffb700 1 -- 172.21.15.160:0/2316795275 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc79409be00 con 0x7fc7a0102430 2024-03-07T07:40:12.248 INFO:teuthology.orchestra.run.smithi160.stdout:107374182466 2024-03-07T07:40:12.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.249+0000 7fc7a6e52700 1 -- 172.21.15.160:0/2316795275 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fc7900a1ea0 msgr2=0x7fc7900a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:12.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.249+0000 7fc7a6e52700 1 --2- 172.21.15.160:0/2316795275 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fc7900a1ea0 0x7fc7900a4360 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc78c006010 tx=0x7fc78c005960 comp rx=0 tx=0).stop 2024-03-07T07:40:12.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.249+0000 7fc7a6e52700 1 -- 172.21.15.160:0/2316795275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7a0102430 msgr2=0x7fc7a0074450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:12.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.249+0000 7fc7a6e52700 1 --2- 172.21.15.160:0/2316795275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7a0102430 0x7fc7a0074450 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc79400e9a0 tx=0x7fc79400ecb0 comp rx=0 tx=0).stop 2024-03-07T07:40:12.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.250+0000 7fc7a6e52700 1 -- 172.21.15.160:0/2316795275 shutdown_connections 2024-03-07T07:40:12.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.250+0000 7fc7a6e52700 1 --2- 172.21.15.160:0/2316795275 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7a0107460 0x7fc7a0074ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:12.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.250+0000 7fc7a6e52700 1 --2- 172.21.15.160:0/2316795275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7a0102430 0x7fc7a0074450 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:12.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.250+0000 7fc7a6e52700 1 --2- 172.21.15.160:0/2316795275 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fc7900a1ea0 0x7fc7900a4360 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:12.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.250+0000 7fc7a6e52700 1 --2- 172.21.15.160:0/2316795275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7a0102de0 0x7fc7a0074990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:12.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.250+0000 7fc7a6e52700 1 -- 172.21.15.160:0/2316795275 >> 172.21.15.160:0/2316795275 conn(0x7fc7a00791e0 msgr2=0x7fc7a0101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:12.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.250+0000 7fc7a6e52700 1 -- 172.21.15.160:0/2316795275 shutdown_connections 2024-03-07T07:40:12.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:12.250+0000 7fc7a6e52700 1 -- 172.21.15.160:0/2316795275 wait complete. 2024-03-07T07:40:12.531 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:12 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:40:12.531 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:12 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/2316795275' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-07T07:40:12.655 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:12 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:40:12.655 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:12 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/2316795275' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-07T07:40:12.742 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:12 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:40:12.742 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:12 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/2316795275' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-07T07:40:13.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.321+0000 7f96d411d700 1 -- 172.21.15.160:0/3825079845 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96cc102b60 msgr2=0x7f96cc10efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:13.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.321+0000 7f96d411d700 1 --2- 172.21.15.160:0/3825079845 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96cc102b60 0x7f96cc10efa0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f96c400b0d0 tx=0x7f96c400b3e0 comp rx=0 tx=0).stop 2024-03-07T07:40:13.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.321+0000 7f96d411d700 1 -- 172.21.15.160:0/3825079845 shutdown_connections 2024-03-07T07:40:13.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.321+0000 7f96d411d700 1 --2- 172.21.15.160:0/3825079845 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96cc102b60 0x7f96cc10efa0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:13.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.321+0000 7f96d411d700 1 --2- 172.21.15.160:0/3825079845 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96cc108180 0x7f96cc108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:13.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.321+0000 7f96d411d700 1 --2- 172.21.15.160:0/3825079845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96cc1021a0 0x7f96cc102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:13.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.321+0000 7f96d411d700 1 -- 172.21.15.160:0/3825079845 >> 172.21.15.160:0/3825079845 conn(0x7f96cc079280 msgr2=0x7f96cc079690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:13.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.322+0000 7f96d411d700 1 -- 172.21.15.160:0/3825079845 shutdown_connections 2024-03-07T07:40:13.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.322+0000 7f96d411d700 1 -- 172.21.15.160:0/3825079845 wait complete. 2024-03-07T07:40:13.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.322+0000 7f96d411d700 1 Processor -- start 2024-03-07T07:40:13.326 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.322+0000 7f96d411d700 1 -- start start 2024-03-07T07:40:13.326 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.322+0000 7f96d411d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96cc1021a0 0x7f96cc074450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:13.326 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.322+0000 7f96d411d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96cc102b60 0x7f96cc074990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:13.326 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.322+0000 7f96d411d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96cc108180 0x7f96cc074ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:13.326 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.322+0000 7f96d411d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96cc112400 con 0x7f96cc102b60 2024-03-07T07:40:13.326 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.322+0000 7f96d411d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96cc112280 con 0x7f96cc108180 2024-03-07T07:40:13.326 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.322+0000 7f96d411d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96cc112580 con 0x7f96cc1021a0 2024-03-07T07:40:13.326 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.322+0000 7f96d1eb9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96cc1021a0 0x7f96cc074450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:13.326 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.322+0000 7f96d16b8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96cc102b60 0x7f96cc074990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:13.326 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.322+0000 7f96d1eb9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96cc1021a0 0x7f96cc074450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.160:57876/0 (socket says 172.21.15.160:57876) 2024-03-07T07:40:13.326 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.322+0000 7f96d1eb9700 1 -- 172.21.15.160:0/88430877 learned_addr learned my addr 172.21.15.160:0/88430877 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:40:13.326 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.322+0000 7f96d26ba700 1 --2- 172.21.15.160:0/88430877 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96cc108180 0x7f96cc074ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:13.326 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.323+0000 7f96d1eb9700 1 -- 172.21.15.160:0/88430877 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96cc108180 msgr2=0x7f96cc074ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:13.327 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.323+0000 7f96d1eb9700 1 --2- 172.21.15.160:0/88430877 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96cc108180 0x7f96cc074ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:13.327 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.323+0000 7f96d1eb9700 1 -- 172.21.15.160:0/88430877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96cc102b60 msgr2=0x7f96cc074990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:13.327 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.323+0000 7f96d1eb9700 1 --2- 172.21.15.160:0/88430877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96cc102b60 0x7f96cc074990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:13.327 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.323+0000 7f96d1eb9700 1 -- 172.21.15.160:0/88430877 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96cc1a6d20 con 0x7f96cc1021a0 2024-03-07T07:40:13.327 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.323+0000 7f96d16b8700 1 --2- 172.21.15.160:0/88430877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96cc102b60 0x7f96cc074990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:40:13.327 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.323+0000 7f96d26ba700 1 --2- 172.21.15.160:0/88430877 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96cc108180 0x7f96cc074ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:40:13.327 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.323+0000 7f96d1eb9700 1 --2- 172.21.15.160:0/88430877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96cc1021a0 0x7f96cc074450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f96c800e9a0 tx=0x7f96c800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:13.327 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.323+0000 7f96c2ffd700 1 -- 172.21.15.160:0/88430877 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c800ca10 con 0x7f96cc1021a0 2024-03-07T07:40:13.327 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.323+0000 7f96d411d700 1 -- 172.21.15.160:0/88430877 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96cc1a6fb0 con 0x7f96cc1021a0 2024-03-07T07:40:13.327 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.323+0000 7f96c2ffd700 1 -- 172.21.15.160:0/88430877 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f96c800cbb0 con 0x7f96cc1021a0 2024-03-07T07:40:13.327 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.323+0000 7f96c2ffd700 1 -- 172.21.15.160:0/88430877 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c8018680 con 0x7f96cc1021a0 2024-03-07T07:40:13.327 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.323+0000 7f96d411d700 1 -- 172.21.15.160:0/88430877 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96cc1a7510 con 0x7f96cc1021a0 2024-03-07T07:40:13.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.324+0000 7f96d411d700 1 -- 172.21.15.160:0/88430877 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96cc068e10 con 0x7f96cc1021a0 2024-03-07T07:40:13.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.325+0000 7f96c2ffd700 1 -- 172.21.15.160:0/88430877 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) v1 ==== 136805+0+0 (secure 0 0 0) 0x7f96c8047880 con 0x7f96cc1021a0 2024-03-07T07:40:13.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.326+0000 7f96c2ffd700 1 --2- 172.21.15.160:0/88430877 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f96b80a1ea0 0x7f96b80a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:13.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.326+0000 7f96d16b8700 1 --2- 172.21.15.160:0/88430877 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f96b80a1ea0 0x7f96b80a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:13.330 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.326+0000 7f96c2ffd700 1 -- 172.21.15.160:0/88430877 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f96c80ce110 con 0x7f96cc1021a0 2024-03-07T07:40:13.330 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.326+0000 7f96d16b8700 1 --2- 172.21.15.160:0/88430877 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f96b80a1ea0 0x7f96b80a4360 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f96cc0797f0 tx=0x7f96bc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:13.333 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.332+0000 7f96c2ffd700 1 -- 172.21.15.160:0/88430877 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f96c809b3a0 con 0x7f96cc1021a0 2024-03-07T07:40:13.411 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:13 smithi160 ceph-mon[31963]: pgmap v319: 97 pgs: 1 peering, 96 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4 op/s; 7.2 KiB/s, 1 keys/s, 1 objects/s recovering 2024-03-07T07:40:13.442 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182462 got 107374182466 for osd.3 2024-03-07T07:40:13.442 DEBUG:teuthology.parallel:result is None 2024-03-07T07:40:13.538 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.536+0000 7f96d411d700 1 -- 172.21.15.160:0/88430877 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f96cc02dbb0 con 0x7f96cc1021a0 2024-03-07T07:40:13.538 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.537+0000 7f96c2ffd700 1 -- 172.21.15.160:0/88430877 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f96c809aca0 con 0x7f96cc1021a0 2024-03-07T07:40:13.538 INFO:teuthology.orchestra.run.smithi160.stdout:197568495666 2024-03-07T07:40:13.541 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.539+0000 7f96d411d700 1 -- 172.21.15.160:0/88430877 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f96b80a1ea0 msgr2=0x7f96b80a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:13.541 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.539+0000 7f96d411d700 1 --2- 172.21.15.160:0/88430877 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f96b80a1ea0 0x7f96b80a4360 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f96cc0797f0 tx=0x7f96bc009260 comp rx=0 tx=0).stop 2024-03-07T07:40:13.542 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.539+0000 7f96d411d700 1 -- 172.21.15.160:0/88430877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96cc1021a0 msgr2=0x7f96cc074450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:13.542 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.539+0000 7f96d411d700 1 --2- 172.21.15.160:0/88430877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96cc1021a0 0x7f96cc074450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f96c800e9a0 tx=0x7f96c800ecb0 comp rx=0 tx=0).stop 2024-03-07T07:40:13.542 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.539+0000 7f96d411d700 1 -- 172.21.15.160:0/88430877 shutdown_connections 2024-03-07T07:40:13.542 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.539+0000 7f96d411d700 1 --2- 172.21.15.160:0/88430877 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96cc108180 0x7f96cc074ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:13.542 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.539+0000 7f96d411d700 1 --2- 172.21.15.160:0/88430877 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f96b80a1ea0 0x7f96b80a4360 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:13.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.539+0000 7f96d411d700 1 --2- 172.21.15.160:0/88430877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96cc1021a0 0x7f96cc074450 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:13.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.539+0000 7f96d411d700 1 --2- 172.21.15.160:0/88430877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96cc102b60 0x7f96cc074990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:13.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.540+0000 7f96d411d700 1 -- 172.21.15.160:0/88430877 >> 172.21.15.160:0/88430877 conn(0x7f96cc079280 msgr2=0x7f96cc100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:13.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.540+0000 7f96d411d700 1 -- 172.21.15.160:0/88430877 shutdown_connections 2024-03-07T07:40:13.543 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:13.540+0000 7f96d411d700 1 -- 172.21.15.160:0/88430877 wait complete. 2024-03-07T07:40:13.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:13 smithi178 ceph-mon[36650]: pgmap v319: 97 pgs: 1 peering, 96 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4 op/s; 7.2 KiB/s, 1 keys/s, 1 objects/s recovering 2024-03-07T07:40:13.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:13 smithi177 ceph-mon[36578]: pgmap v319: 97 pgs: 1 peering, 96 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4 op/s; 7.2 KiB/s, 1 keys/s, 1 objects/s recovering 2024-03-07T07:40:14.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.062+0000 7f02beac8700 1 -- 172.21.15.160:0/349546525 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02b81019f0 msgr2=0x7f02b810ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:14.065 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.062+0000 7f02beac8700 1 --2- 172.21.15.160:0/349546525 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02b81019f0 0x7f02b810ded0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f02b000b0a0 tx=0x7f02b000b3b0 comp rx=0 tx=0).stop 2024-03-07T07:40:14.065 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.062+0000 7f02beac8700 1 -- 172.21.15.160:0/349546525 shutdown_connections 2024-03-07T07:40:14.065 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.062+0000 7f02beac8700 1 --2- 172.21.15.160:0/349546525 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02b81019f0 0x7f02b810ded0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:14.065 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.062+0000 7f02beac8700 1 --2- 172.21.15.160:0/349546525 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f02b8107090 0x7f02b8107470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:14.065 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.062+0000 7f02beac8700 1 --2- 172.21.15.160:0/349546525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02b8101050 0x7f02b81014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:14.065 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.062+0000 7f02beac8700 1 -- 172.21.15.160:0/349546525 >> 172.21.15.160:0/349546525 conn(0x7f02b8077f80 msgr2=0x7f02b8078390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:14.065 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.062+0000 7f02beac8700 1 -- 172.21.15.160:0/349546525 shutdown_connections 2024-03-07T07:40:14.065 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.062+0000 7f02beac8700 1 -- 172.21.15.160:0/349546525 wait complete. 2024-03-07T07:40:14.065 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.063+0000 7f02beac8700 1 Processor -- start 2024-03-07T07:40:14.065 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.063+0000 7f02beac8700 1 -- start start 2024-03-07T07:40:14.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.063+0000 7f02beac8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02b8101050 0x7f02b81996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:14.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.063+0000 7f02beac8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02b81019f0 0x7f02b8199c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:14.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.063+0000 7f02beac8700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f02b8107090 0x7f02b81a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:14.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.063+0000 7f02beac8700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b8111310 con 0x7f02b8101050 2024-03-07T07:40:14.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.063+0000 7f02beac8700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b8111190 con 0x7f02b8107090 2024-03-07T07:40:14.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.063+0000 7f02beac8700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b8111490 con 0x7f02b81019f0 2024-03-07T07:40:14.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.063+0000 7f02bc864700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02b8101050 0x7f02b81996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:14.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.063+0000 7f02bc864700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02b8101050 0x7f02b81996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41276/0 (socket says 172.21.15.160:41276) 2024-03-07T07:40:14.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.063+0000 7f02bc864700 1 -- 172.21.15.160:0/142388321 learned_addr learned my addr 172.21.15.160:0/142388321 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:40:14.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.063+0000 7f02b7fff700 1 --2- 172.21.15.160:0/142388321 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02b81019f0 0x7f02b8199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:14.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.063+0000 7f02bd065700 1 --2- 172.21.15.160:0/142388321 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f02b8107090 0x7f02b81a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:14.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.064+0000 7f02bc864700 1 -- 172.21.15.160:0/142388321 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02b81019f0 msgr2=0x7f02b8199c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:14.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.064+0000 7f02bc864700 1 --2- 172.21.15.160:0/142388321 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02b81019f0 0x7f02b8199c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:14.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.064+0000 7f02bc864700 1 -- 172.21.15.160:0/142388321 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f02b8107090 msgr2=0x7f02b81a4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:14.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.064+0000 7f02bc864700 1 --2- 172.21.15.160:0/142388321 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f02b8107090 0x7f02b81a4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:14.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.064+0000 7f02bc864700 1 -- 172.21.15.160:0/142388321 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b81a4c90 con 0x7f02b8101050 2024-03-07T07:40:14.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.064+0000 7f02bc864700 1 --2- 172.21.15.160:0/142388321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02b8101050 0x7f02b81996d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f02ac00e9a0 tx=0x7f02ac00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:14.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.064+0000 7f02b5ffb700 1 -- 172.21.15.160:0/142388321 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ac00ca10 con 0x7f02b8101050 2024-03-07T07:40:14.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.064+0000 7f02beac8700 1 -- 172.21.15.160:0/142388321 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02b81a4f20 con 0x7f02b8101050 2024-03-07T07:40:14.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.064+0000 7f02b5ffb700 1 -- 172.21.15.160:0/142388321 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f02ac00cbb0 con 0x7f02b8101050 2024-03-07T07:40:14.068 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.064+0000 7f02b5ffb700 1 -- 172.21.15.160:0/142388321 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ac018680 con 0x7f02b8101050 2024-03-07T07:40:14.068 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.064+0000 7f02beac8700 1 -- 172.21.15.160:0/142388321 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02b81a5420 con 0x7f02b8101050 2024-03-07T07:40:14.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.065+0000 7f02beac8700 1 -- 172.21.15.160:0/142388321 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02b8068e10 con 0x7f02b8101050 2024-03-07T07:40:14.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.066+0000 7f02b5ffb700 1 -- 172.21.15.160:0/142388321 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) v1 ==== 136805+0+0 (secure 0 0 0) 0x7f02ac010430 con 0x7f02b8101050 2024-03-07T07:40:14.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.067+0000 7f02b5ffb700 1 --2- 172.21.15.160:0/142388321 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f02a80a1ea0 0x7f02a80a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:14.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.067+0000 7f02b7fff700 1 --2- 172.21.15.160:0/142388321 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f02a80a1ea0 0x7f02a80a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:14.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.067+0000 7f02b5ffb700 1 -- 172.21.15.160:0/142388321 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f02ac0ce0f0 con 0x7f02b8101050 2024-03-07T07:40:14.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.067+0000 7f02b7fff700 1 --2- 172.21.15.160:0/142388321 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f02a80a1ea0 0x7f02a80a4360 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f02b8078580 tx=0x7f02a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:14.071 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.071+0000 7f02b5ffb700 1 -- 172.21.15.160:0/142388321 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f02ac09b3c0 con 0x7f02b8101050 2024-03-07T07:40:14.275 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.275+0000 7f02beac8700 1 -- 172.21.15.160:0/142388321 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f02b802d9f0 con 0x7f02b8101050 2024-03-07T07:40:14.275 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.275+0000 7f02b5ffb700 1 -- 172.21.15.160:0/142388321 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f02ac09acc0 con 0x7f02b8101050 2024-03-07T07:40:14.276 INFO:teuthology.orchestra.run.smithi160.stdout:34359738449 2024-03-07T07:40:14.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.277+0000 7f02beac8700 1 -- 172.21.15.160:0/142388321 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f02a80a1ea0 msgr2=0x7f02a80a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:14.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.277+0000 7f02beac8700 1 --2- 172.21.15.160:0/142388321 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f02a80a1ea0 0x7f02a80a4360 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f02b8078580 tx=0x7f02a4009260 comp rx=0 tx=0).stop 2024-03-07T07:40:14.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.277+0000 7f02beac8700 1 -- 172.21.15.160:0/142388321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02b8101050 msgr2=0x7f02b81996d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:14.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.277+0000 7f02beac8700 1 --2- 172.21.15.160:0/142388321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02b8101050 0x7f02b81996d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f02ac00e9a0 tx=0x7f02ac00ecb0 comp rx=0 tx=0).stop 2024-03-07T07:40:14.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.277+0000 7f02beac8700 1 -- 172.21.15.160:0/142388321 shutdown_connections 2024-03-07T07:40:14.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.277+0000 7f02beac8700 1 --2- 172.21.15.160:0/142388321 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f02b8107090 0x7f02b81a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:14.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.277+0000 7f02beac8700 1 --2- 172.21.15.160:0/142388321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02b8101050 0x7f02b81996d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:14.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.277+0000 7f02beac8700 1 --2- 172.21.15.160:0/142388321 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f02a80a1ea0 0x7f02a80a4360 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:14.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.277+0000 7f02beac8700 1 --2- 172.21.15.160:0/142388321 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02b81019f0 0x7f02b8199c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:14.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.277+0000 7f02beac8700 1 -- 172.21.15.160:0/142388321 >> 172.21.15.160:0/142388321 conn(0x7f02b8077f80 msgr2=0x7f02b80ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:14.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.278+0000 7f02beac8700 1 -- 172.21.15.160:0/142388321 shutdown_connections 2024-03-07T07:40:14.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.278+0000 7f02beac8700 1 -- 172.21.15.160:0/142388321 wait complete. 2024-03-07T07:40:14.573 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:14 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/88430877' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-07T07:40:14.612 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:14 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:40:14.612 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:14 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:40:14.612 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:14 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:40:14.612 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:14 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:40:14.612 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:14 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/142388321' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-07T07:40:14.612 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:14 smithi160 ceph-mon[31963]: pgmap v320: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 3 op/s; 7.2 KiB/s, 1 keys/s, 2 objects/s recovering 2024-03-07T07:40:14.684 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.682+0000 7ff0db72a700 1 -- 172.21.15.160:0/2401973214 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0d4107090 msgr2=0x7ff0d4107470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:14.685 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.682+0000 7ff0db72a700 1 --2- 172.21.15.160:0/2401973214 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0d4107090 0x7ff0d4107470 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff0d000b0a0 tx=0x7ff0d000b3b0 comp rx=0 tx=0).stop 2024-03-07T07:40:14.685 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.682+0000 7ff0db72a700 1 -- 172.21.15.160:0/2401973214 shutdown_connections 2024-03-07T07:40:14.685 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.682+0000 7ff0db72a700 1 --2- 172.21.15.160:0/2401973214 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0d41019f0 0x7ff0d410ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:14.685 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.682+0000 7ff0db72a700 1 --2- 172.21.15.160:0/2401973214 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0d4107090 0x7ff0d4107470 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:14.685 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.682+0000 7ff0db72a700 1 --2- 172.21.15.160:0/2401973214 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0d4101050 0x7ff0d41014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:14.685 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.682+0000 7ff0db72a700 1 -- 172.21.15.160:0/2401973214 >> 172.21.15.160:0/2401973214 conn(0x7ff0d4077f80 msgr2=0x7ff0d4078390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:14.685 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.682+0000 7ff0db72a700 1 -- 172.21.15.160:0/2401973214 shutdown_connections 2024-03-07T07:40:14.685 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.682+0000 7ff0db72a700 1 -- 172.21.15.160:0/2401973214 wait complete. 2024-03-07T07:40:14.685 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.683+0000 7ff0db72a700 1 Processor -- start 2024-03-07T07:40:14.685 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.683+0000 7ff0db72a700 1 -- start start 2024-03-07T07:40:14.685 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.683+0000 7ff0db72a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0d4101050 0x7ff0d419e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:14.685 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.683+0000 7ff0db72a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0d41019f0 0x7ff0d419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:14.685 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.683+0000 7ff0db72a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0d4107090 0x7ff0d4198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:14.686 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.683+0000 7ff0db72a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d41112f0 con 0x7ff0d4107090 2024-03-07T07:40:14.686 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.683+0000 7ff0db72a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d4111170 con 0x7ff0d41019f0 2024-03-07T07:40:14.686 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.683+0000 7ff0db72a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d4111470 con 0x7ff0d4101050 2024-03-07T07:40:14.686 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.683+0000 7ff0d9cc7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0d4107090 0x7ff0d4198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:14.686 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.683+0000 7ff0d94c6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0d4101050 0x7ff0d419e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:14.686 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.683+0000 7ff0d94c6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0d4101050 0x7ff0d419e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.160:57916/0 (socket says 172.21.15.160:57916) 2024-03-07T07:40:14.686 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.683+0000 7ff0d9cc7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0d4107090 0x7ff0d4198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41296/0 (socket says 172.21.15.160:41296) 2024-03-07T07:40:14.686 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.683+0000 7ff0d94c6700 1 -- 172.21.15.160:0/1625857405 learned_addr learned my addr 172.21.15.160:0/1625857405 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:40:14.686 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.683+0000 7ff0d8cc5700 1 --2- 172.21.15.160:0/1625857405 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0d41019f0 0x7ff0d419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:14.686 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.683+0000 7ff0d94c6700 1 -- 172.21.15.160:0/1625857405 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0d41019f0 msgr2=0x7ff0d419eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:14.686 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.683+0000 7ff0d94c6700 1 --2- 172.21.15.160:0/1625857405 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0d41019f0 0x7ff0d419eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:14.686 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.683+0000 7ff0d94c6700 1 -- 172.21.15.160:0/1625857405 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0d4107090 msgr2=0x7ff0d4198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:14.686 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.683+0000 7ff0d94c6700 1 --2- 172.21.15.160:0/1625857405 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0d4107090 0x7ff0d4198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:14.687 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.683+0000 7ff0d94c6700 1 -- 172.21.15.160:0/1625857405 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d4199020 con 0x7ff0d4101050 2024-03-07T07:40:14.687 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.683+0000 7ff0d9cc7700 1 --2- 172.21.15.160:0/1625857405 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0d4107090 0x7ff0d4198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:40:14.687 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.684+0000 7ff0d94c6700 1 --2- 172.21.15.160:0/1625857405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0d4101050 0x7ff0d419e620 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff0d00062a0 tx=0x7ff0d0004ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:14.687 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.684+0000 7ff0ca7fc700 1 -- 172.21.15.160:0/1625857405 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d0021070 con 0x7ff0d4101050 2024-03-07T07:40:14.687 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.684+0000 7ff0ca7fc700 1 -- 172.21.15.160:0/1625857405 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff0d0007b80 con 0x7ff0d4101050 2024-03-07T07:40:14.687 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.684+0000 7ff0ca7fc700 1 -- 172.21.15.160:0/1625857405 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d00052c0 con 0x7ff0d4101050 2024-03-07T07:40:14.687 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.684+0000 7ff0db72a700 1 -- 172.21.15.160:0/1625857405 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0d41992b0 con 0x7ff0d4101050 2024-03-07T07:40:14.687 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.684+0000 7ff0db72a700 1 -- 172.21.15.160:0/1625857405 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0d41a55a0 con 0x7ff0d4101050 2024-03-07T07:40:14.688 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.685+0000 7ff0db72a700 1 -- 172.21.15.160:0/1625857405 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0d4199620 con 0x7ff0d4101050 2024-03-07T07:40:14.688 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.686+0000 7ff0ca7fc700 1 -- 172.21.15.160:0/1625857405 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) v1 ==== 136805+0+0 (secure 0 0 0) 0x7ff0d0024070 con 0x7ff0d4101050 2024-03-07T07:40:14.688 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.686+0000 7ff0ca7fc700 1 --2- 172.21.15.160:0/1625857405 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7ff0c00a1ea0 0x7ff0c00a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:14.688 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.686+0000 7ff0d8cc5700 1 --2- 172.21.15.160:0/1625857405 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7ff0c00a1ea0 0x7ff0c00a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:14.689 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.686+0000 7ff0ca7fc700 1 -- 172.21.15.160:0/1625857405 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff0d000f020 con 0x7ff0d4101050 2024-03-07T07:40:14.689 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.687+0000 7ff0d8cc5700 1 --2- 172.21.15.160:0/1625857405 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7ff0c00a1ea0 0x7ff0c00a4360 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4199eb0 tx=0x7ff0c4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:14.691 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.691+0000 7ff0ca7fc700 1 -- 172.21.15.160:0/1625857405 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff0d4199620 con 0x7ff0d4101050 2024-03-07T07:40:14.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:14 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/88430877' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-07T07:40:14.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:14 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:40:14.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:14 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:40:14.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:14 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:40:14.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:14 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:40:14.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:14 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/142388321' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-07T07:40:14.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:14 smithi178 ceph-mon[36650]: pgmap v320: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 3 op/s; 7.2 KiB/s, 1 keys/s, 2 objects/s recovering 2024-03-07T07:40:14.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:14 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/88430877' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-07T07:40:14.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:14 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:40:14.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:14 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:40:14.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:14 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:40:14.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:14 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:40:14.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:14 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/142388321' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-07T07:40:14.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:14 smithi177 ceph-mon[36578]: pgmap v320: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 3 op/s; 7.2 KiB/s, 1 keys/s, 2 objects/s recovering 2024-03-07T07:40:14.891 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.890+0000 7ff0db72a700 1 -- 172.21.15.160:0/1625857405 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff0d4199c20 con 0x7ff0d4101050 2024-03-07T07:40:14.891 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.891+0000 7ff0ca7fc700 1 -- 172.21.15.160:0/1625857405 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff0d002b020 con 0x7ff0d4101050 2024-03-07T07:40:14.892 INFO:teuthology.orchestra.run.smithi160.stdout:77309411399 2024-03-07T07:40:14.893 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.893+0000 7ff0db72a700 1 -- 172.21.15.160:0/1625857405 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7ff0c00a1ea0 msgr2=0x7ff0c00a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:14.894 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.893+0000 7ff0db72a700 1 --2- 172.21.15.160:0/1625857405 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7ff0c00a1ea0 0x7ff0c00a4360 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4199eb0 tx=0x7ff0c4009260 comp rx=0 tx=0).stop 2024-03-07T07:40:14.894 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.893+0000 7ff0db72a700 1 -- 172.21.15.160:0/1625857405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0d4101050 msgr2=0x7ff0d419e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:14.894 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.893+0000 7ff0db72a700 1 --2- 172.21.15.160:0/1625857405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0d4101050 0x7ff0d419e620 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff0d00062a0 tx=0x7ff0d0004ce0 comp rx=0 tx=0).stop 2024-03-07T07:40:14.894 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.893+0000 7ff0db72a700 1 -- 172.21.15.160:0/1625857405 shutdown_connections 2024-03-07T07:40:14.894 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.893+0000 7ff0db72a700 1 --2- 172.21.15.160:0/1625857405 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0d4107090 0x7ff0d4198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:14.894 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.893+0000 7ff0db72a700 1 --2- 172.21.15.160:0/1625857405 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7ff0c00a1ea0 0x7ff0c00a4360 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:14.895 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.893+0000 7ff0db72a700 1 --2- 172.21.15.160:0/1625857405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0d4101050 0x7ff0d419e620 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:14.895 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.893+0000 7ff0db72a700 1 --2- 172.21.15.160:0/1625857405 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0d41019f0 0x7ff0d419eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:14.895 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.893+0000 7ff0db72a700 1 -- 172.21.15.160:0/1625857405 >> 172.21.15.160:0/1625857405 conn(0x7ff0d4077f80 msgr2=0x7ff0d4105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:14.895 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.893+0000 7ff0db72a700 1 -- 172.21.15.160:0/1625857405 shutdown_connections 2024-03-07T07:40:14.895 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:14.893+0000 7ff0db72a700 1 -- 172.21.15.160:0/1625857405 wait complete. 2024-03-07T07:40:15.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:15 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/1625857405' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-07T07:40:16.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:15 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/1625857405' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-07T07:40:16.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.068+0000 7ff8f6972700 1 -- 172.21.15.160:0/2667523013 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8f0102460 msgr2=0x7ff8f0105cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:16.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.068+0000 7ff8f6972700 1 --2- 172.21.15.160:0/2667523013 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8f0102460 0x7ff8f0105cf0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff8d80099c0 tx=0x7ff8d8009cd0 comp rx=0 tx=0).stop 2024-03-07T07:40:16.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.068+0000 7ff8f6972700 1 -- 172.21.15.160:0/2667523013 shutdown_connections 2024-03-07T07:40:16.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.068+0000 7ff8f6972700 1 --2- 172.21.15.160:0/2667523013 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff8f01063d0 0x7ff8f010dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:16.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.068+0000 7ff8f6972700 1 --2- 172.21.15.160:0/2667523013 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8f0101ab0 0x7ff8f0101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:16.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.068+0000 7ff8f6972700 1 --2- 172.21.15.160:0/2667523013 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8f0102460 0x7ff8f0105cf0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:16.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.068+0000 7ff8f6972700 1 -- 172.21.15.160:0/2667523013 >> 172.21.15.160:0/2667523013 conn(0x7ff8f0077650 msgr2=0x7ff8f00fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:16.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.068+0000 7ff8f6972700 1 -- 172.21.15.160:0/2667523013 shutdown_connections 2024-03-07T07:40:16.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.068+0000 7ff8f6972700 1 -- 172.21.15.160:0/2667523013 wait complete. 2024-03-07T07:40:16.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.069+0000 7ff8f6972700 1 Processor -- start 2024-03-07T07:40:16.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.069+0000 7ff8f6972700 1 -- start start 2024-03-07T07:40:16.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.069+0000 7ff8f6972700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff8f0101ab0 0x7ff8f019e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:16.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.069+0000 7ff8f6972700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8f0102460 0x7ff8f019eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:16.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.069+0000 7ff8f6972700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8f01063d0 0x7ff8f0198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:16.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.069+0000 7ff8f6972700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f01113b0 con 0x7ff8f0102460 2024-03-07T07:40:16.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.069+0000 7ff8f6972700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f0111230 con 0x7ff8f0101ab0 2024-03-07T07:40:16.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.069+0000 7ff8f6972700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f0111530 con 0x7ff8f01063d0 2024-03-07T07:40:16.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.069+0000 7ff8effff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff8f0101ab0 0x7ff8f019e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:16.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.069+0000 7ff8ef7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8f0102460 0x7ff8f019eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:16.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.069+0000 7ff8effff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff8f0101ab0 0x7ff8f019e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.160:39502/0 (socket says 172.21.15.160:39502) 2024-03-07T07:40:16.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.069+0000 7ff8effff700 1 -- 172.21.15.160:0/2934431127 learned_addr learned my addr 172.21.15.160:0/2934431127 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:40:16.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.069+0000 7ff8f4f0f700 1 --2- 172.21.15.160:0/2934431127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8f01063d0 0x7ff8f0198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:16.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.069+0000 7ff8effff700 1 -- 172.21.15.160:0/2934431127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8f01063d0 msgr2=0x7ff8f0198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:16.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.069+0000 7ff8effff700 1 --2- 172.21.15.160:0/2934431127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8f01063d0 0x7ff8f0198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:16.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.069+0000 7ff8effff700 1 -- 172.21.15.160:0/2934431127 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8f0102460 msgr2=0x7ff8f019eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:16.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.069+0000 7ff8effff700 1 --2- 172.21.15.160:0/2934431127 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8f0102460 0x7ff8f019eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:16.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.070+0000 7ff8effff700 1 -- 172.21.15.160:0/2934431127 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f0198f70 con 0x7ff8f0101ab0 2024-03-07T07:40:16.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.070+0000 7ff8effff700 1 --2- 172.21.15.160:0/2934431127 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff8f0101ab0 0x7ff8f019e610 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff8e000e9c0 tx=0x7ff8e000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:16.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.070+0000 7ff8ed7fa700 1 -- 172.21.15.160:0/2934431127 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e000cbd0 con 0x7ff8f0101ab0 2024-03-07T07:40:16.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.070+0000 7ff8f6972700 1 -- 172.21.15.160:0/2934431127 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8f0199260 con 0x7ff8f0101ab0 2024-03-07T07:40:16.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.070+0000 7ff8ed7fa700 1 -- 172.21.15.160:0/2934431127 <== mon.2 v2:172.21.15.177:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff8e000ee60 con 0x7ff8f0101ab0 2024-03-07T07:40:16.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.070+0000 7ff8ed7fa700 1 -- 172.21.15.160:0/2934431127 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0005630 con 0x7ff8f0101ab0 2024-03-07T07:40:16.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.070+0000 7ff8f6972700 1 -- 172.21.15.160:0/2934431127 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8f01a5470 con 0x7ff8f0101ab0 2024-03-07T07:40:16.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.071+0000 7ff8f6972700 1 -- 172.21.15.160:0/2934431127 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8f0068e10 con 0x7ff8f0101ab0 2024-03-07T07:40:16.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.072+0000 7ff8ed7fa700 1 -- 172.21.15.160:0/2934431127 <== mon.2 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 17) v1 ==== 136805+0+0 (secure 0 0 0) 0x7ff8e0004750 con 0x7ff8f0101ab0 2024-03-07T07:40:16.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.072+0000 7ff8ed7fa700 1 --2- 172.21.15.160:0/2934431127 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7ff8dc0a1ea0 0x7ff8dc0a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:16.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.072+0000 7ff8ef7fe700 1 --2- 172.21.15.160:0/2934431127 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7ff8dc0a1ea0 0x7ff8dc0a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:16.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.073+0000 7ff8ed7fa700 1 -- 172.21.15.160:0/2934431127 <== mon.2 v2:172.21.15.177:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff8e00ce180 con 0x7ff8f0101ab0 2024-03-07T07:40:16.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.073+0000 7ff8ef7fe700 1 --2- 172.21.15.160:0/2934431127 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7ff8dc0a1ea0 0x7ff8dc0a4360 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8000c00 tx=0x7ff8d8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:16.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.081+0000 7ff8ed7fa700 1 -- 172.21.15.160:0/2934431127 <== mon.2 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff8e000fe20 con 0x7ff8f0101ab0 2024-03-07T07:40:16.106 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:15 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/1625857405' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-07T07:40:16.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.293+0000 7ff8f6972700 1 -- 172.21.15.160:0/2934431127 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7ff8f01a56a0 con 0x7ff8f0101ab0 2024-03-07T07:40:16.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.293+0000 7ff8ed7fa700 1 -- 172.21.15.160:0/2934431127 <== mon.2 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff8e009b450 con 0x7ff8f0101ab0 2024-03-07T07:40:16.293 INFO:teuthology.orchestra.run.smithi160.stdout:163208757305 2024-03-07T07:40:16.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.295+0000 7ff8f6972700 1 -- 172.21.15.160:0/2934431127 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7ff8dc0a1ea0 msgr2=0x7ff8dc0a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:16.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.295+0000 7ff8f6972700 1 --2- 172.21.15.160:0/2934431127 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7ff8dc0a1ea0 0x7ff8dc0a4360 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8000c00 tx=0x7ff8d8005fb0 comp rx=0 tx=0).stop 2024-03-07T07:40:16.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.295+0000 7ff8f6972700 1 -- 172.21.15.160:0/2934431127 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff8f0101ab0 msgr2=0x7ff8f019e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:16.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.295+0000 7ff8f6972700 1 --2- 172.21.15.160:0/2934431127 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff8f0101ab0 0x7ff8f019e610 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff8e000e9c0 tx=0x7ff8e000ecd0 comp rx=0 tx=0).stop 2024-03-07T07:40:16.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.295+0000 7ff8f6972700 1 -- 172.21.15.160:0/2934431127 shutdown_connections 2024-03-07T07:40:16.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.295+0000 7ff8f6972700 1 --2- 172.21.15.160:0/2934431127 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7ff8dc0a1ea0 0x7ff8dc0a4360 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:16.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.295+0000 7ff8f6972700 1 --2- 172.21.15.160:0/2934431127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8f01063d0 0x7ff8f0198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:16.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.295+0000 7ff8f6972700 1 --2- 172.21.15.160:0/2934431127 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff8f0101ab0 0x7ff8f019e610 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:16.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.295+0000 7ff8f6972700 1 --2- 172.21.15.160:0/2934431127 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8f0102460 0x7ff8f019eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:16.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.295+0000 7ff8f6972700 1 -- 172.21.15.160:0/2934431127 >> 172.21.15.160:0/2934431127 conn(0x7ff8f0077650 msgr2=0x7ff8f01009f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:16.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.295+0000 7ff8f6972700 1 -- 172.21.15.160:0/2934431127 shutdown_connections 2024-03-07T07:40:16.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:16.295+0000 7ff8f6972700 1 -- 172.21.15.160:0/2934431127 wait complete. 2024-03-07T07:40:16.530 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411393 got 77309411399 for osd.2 2024-03-07T07:40:16.531 DEBUG:teuthology.parallel:result is None 2024-03-07T07:40:16.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:16 smithi160 ceph-mon[31963]: pgmap v321: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 7.2 KiB/s, 1 keys/s, 2 objects/s recovering 2024-03-07T07:40:16.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:16 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/2934431127' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-07T07:40:17.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:16 smithi178 ceph-mon[36650]: pgmap v321: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 7.2 KiB/s, 1 keys/s, 2 objects/s recovering 2024-03-07T07:40:17.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:16 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/2934431127' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-07T07:40:17.106 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:16 smithi177 ceph-mon[36578]: pgmap v321: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 7.2 KiB/s, 1 keys/s, 2 objects/s recovering 2024-03-07T07:40:17.106 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:16 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/2934431127' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-07T07:40:17.667 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757301 got 163208757305 for osd.5 2024-03-07T07:40:17.667 DEBUG:teuthology.parallel:result is None 2024-03-07T07:40:17.791 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495663 got 197568495666 for osd.6 2024-03-07T07:40:17.791 DEBUG:teuthology.parallel:result is None 2024-03-07T07:40:19.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:19 smithi177 ceph-mon[36578]: pgmap v322: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s; 5.8 KiB/s, 1 keys/s, 1 objects/s recovering 2024-03-07T07:40:19.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:19 smithi160 ceph-mon[31963]: pgmap v322: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s; 5.8 KiB/s, 1 keys/s, 1 objects/s recovering 2024-03-07T07:40:19.757 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:19 smithi178 ceph-mon[36650]: pgmap v322: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s; 5.8 KiB/s, 1 keys/s, 1 objects/s recovering 2024-03-07T07:40:20.043 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738445 got 34359738449 for osd.0 2024-03-07T07:40:20.044 DEBUG:teuthology.parallel:result is None 2024-03-07T07:40:20.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:20 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/3057835239' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-07T07:40:20.664 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:20 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/3057835239' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-07T07:40:20.668 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.668+0000 7f292796c700 1 -- 172.21.15.160:0/2990862992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29201082b0 msgr2=0x7f2920108690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:20.669 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.668+0000 7f292796c700 1 --2- 172.21.15.160:0/2990862992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29201082b0 0x7f2920108690 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2910009990 tx=0x7f2910009ca0 comp rx=0 tx=0).stop 2024-03-07T07:40:20.669 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.668+0000 7f292796c700 1 -- 172.21.15.160:0/2990862992 shutdown_connections 2024-03-07T07:40:20.669 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.668+0000 7f292796c700 1 --2- 172.21.15.160:0/2990862992 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2920102b90 0x7f292010eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:20.669 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.668+0000 7f292796c700 1 --2- 172.21.15.160:0/2990862992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29201082b0 0x7f2920108690 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:20.669 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.668+0000 7f292796c700 1 --2- 172.21.15.160:0/2990862992 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29201021d0 0x7f2920102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:20.669 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.668+0000 7f292796c700 1 -- 172.21.15.160:0/2990862992 >> 172.21.15.160:0/2990862992 conn(0x7f2920079210 msgr2=0x7f2920079620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:20.669 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.668+0000 7f292796c700 1 -- 172.21.15.160:0/2990862992 shutdown_connections 2024-03-07T07:40:20.669 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.668+0000 7f292796c700 1 -- 172.21.15.160:0/2990862992 wait complete. 2024-03-07T07:40:20.671 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.669+0000 7f292796c700 1 Processor -- start 2024-03-07T07:40:20.671 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.669+0000 7f292796c700 1 -- start start 2024-03-07T07:40:20.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.669+0000 7f292796c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29201021d0 0x7f2920074450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:20.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.669+0000 7f292796c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2920102b90 0x7f2920074990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:20.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.669+0000 7f292796c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29201082b0 0x7f2920074ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:20.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.669+0000 7f292796c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2920112430 con 0x7f29201021d0 2024-03-07T07:40:20.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.669+0000 7f292796c700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29201122b0 con 0x7f29201082b0 2024-03-07T07:40:20.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.669+0000 7f292796c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29201125b0 con 0x7f2920102b90 2024-03-07T07:40:20.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.669+0000 7f2924f07700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2920102b90 0x7f2920074990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:20.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.669+0000 7f2924f07700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2920102b90 0x7f2920074990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.160:43474/0 (socket says 172.21.15.160:43474) 2024-03-07T07:40:20.672 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.669+0000 7f2924f07700 1 -- 172.21.15.160:0/2481004754 learned_addr learned my addr 172.21.15.160:0/2481004754 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:40:20.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.669+0000 7f2925f09700 1 --2- 172.21.15.160:0/2481004754 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29201082b0 0x7f2920074ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:20.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.669+0000 7f2925708700 1 --2- 172.21.15.160:0/2481004754 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29201021d0 0x7f2920074450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:20.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.669+0000 7f2924f07700 1 -- 172.21.15.160:0/2481004754 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29201082b0 msgr2=0x7f2920074ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:20.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.669+0000 7f2924f07700 1 --2- 172.21.15.160:0/2481004754 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29201082b0 0x7f2920074ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:20.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.669+0000 7f2924f07700 1 -- 172.21.15.160:0/2481004754 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29201021d0 msgr2=0x7f2920074450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:20.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.669+0000 7f2924f07700 1 --2- 172.21.15.160:0/2481004754 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29201021d0 0x7f2920074450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:20.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.669+0000 7f2924f07700 1 -- 172.21.15.160:0/2481004754 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29201a6e10 con 0x7f2920102b90 2024-03-07T07:40:20.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.670+0000 7f2925708700 1 --2- 172.21.15.160:0/2481004754 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29201021d0 0x7f2920074450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T07:40:20.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.670+0000 7f2924f07700 1 --2- 172.21.15.160:0/2481004754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2920102b90 0x7f2920074990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f291c00e9c0 tx=0x7f291c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:20.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.670+0000 7f29167fc700 1 -- 172.21.15.160:0/2481004754 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291c00cbd0 con 0x7f2920102b90 2024-03-07T07:40:20.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.670+0000 7f29167fc700 1 -- 172.21.15.160:0/2481004754 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f291c00cd70 con 0x7f2920102b90 2024-03-07T07:40:20.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.670+0000 7f292796c700 1 -- 172.21.15.160:0/2481004754 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29201a7100 con 0x7f2920102b90 2024-03-07T07:40:20.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.670+0000 7f29167fc700 1 -- 172.21.15.160:0/2481004754 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291c010430 con 0x7f2920102b90 2024-03-07T07:40:20.673 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.670+0000 7f292796c700 1 -- 172.21.15.160:0/2481004754 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29201a75f0 con 0x7f2920102b90 2024-03-07T07:40:20.674 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.671+0000 7f292796c700 1 -- 172.21.15.160:0/2481004754 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2920068e10 con 0x7f2920102b90 2024-03-07T07:40:20.674 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.672+0000 7f29167fc700 1 -- 172.21.15.160:0/2481004754 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) v1 ==== 136805+0+0 (secure 0 0 0) 0x7f291c004750 con 0x7f2920102b90 2024-03-07T07:40:20.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.673+0000 7f29167fc700 1 --2- 172.21.15.160:0/2481004754 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f290c0a1ea0 0x7f290c0a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:20.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.673+0000 7f2925708700 1 --2- 172.21.15.160:0/2481004754 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f290c0a1ea0 0x7f290c0a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:20.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.673+0000 7f29167fc700 1 -- 172.21.15.160:0/2481004754 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f291c0ce1d0 con 0x7f2920102b90 2024-03-07T07:40:20.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.673+0000 7f2925708700 1 --2- 172.21.15.160:0/2481004754 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f290c0a1ea0 0x7f290c0a4360 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f291000b2e0 tx=0x7f2910005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:20.676 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.676+0000 7f29167fc700 1 -- 172.21.15.160:0/2481004754 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f291c0203a0 con 0x7f2920102b90 2024-03-07T07:40:20.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:20 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/3057835239' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-07T07:40:20.878 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.878+0000 7f292796c700 1 -- 172.21.15.160:0/2481004754 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f292002d9f0 con 0x7f2920102b90 2024-03-07T07:40:20.879 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.879+0000 7f29167fc700 1 -- 172.21.15.160:0/2481004754 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f291c09b4a0 con 0x7f2920102b90 2024-03-07T07:40:20.879 INFO:teuthology.orchestra.run.smithi160.stdout:141733920831 2024-03-07T07:40:20.881 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.881+0000 7f292796c700 1 -- 172.21.15.160:0/2481004754 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f290c0a1ea0 msgr2=0x7f290c0a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:20.882 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.881+0000 7f292796c700 1 --2- 172.21.15.160:0/2481004754 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f290c0a1ea0 0x7f290c0a4360 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f291000b2e0 tx=0x7f2910005e80 comp rx=0 tx=0).stop 2024-03-07T07:40:20.882 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.881+0000 7f292796c700 1 -- 172.21.15.160:0/2481004754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2920102b90 msgr2=0x7f2920074990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:20.882 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.881+0000 7f292796c700 1 --2- 172.21.15.160:0/2481004754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2920102b90 0x7f2920074990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f291c00e9c0 tx=0x7f291c00ecd0 comp rx=0 tx=0).stop 2024-03-07T07:40:20.882 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.881+0000 7f292796c700 1 -- 172.21.15.160:0/2481004754 shutdown_connections 2024-03-07T07:40:20.882 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.881+0000 7f292796c700 1 --2- 172.21.15.160:0/2481004754 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29201082b0 0x7f2920074ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:20.882 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.881+0000 7f292796c700 1 --2- 172.21.15.160:0/2481004754 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29201021d0 0x7f2920074450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:20.882 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.881+0000 7f292796c700 1 --2- 172.21.15.160:0/2481004754 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f290c0a1ea0 0x7f290c0a4360 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:20.882 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.881+0000 7f292796c700 1 --2- 172.21.15.160:0/2481004754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2920102b90 0x7f2920074990 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:20.882 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.881+0000 7f292796c700 1 -- 172.21.15.160:0/2481004754 >> 172.21.15.160:0/2481004754 conn(0x7f2920079210 msgr2=0x7f2920100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:20.882 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.881+0000 7f292796c700 1 -- 172.21.15.160:0/2481004754 shutdown_connections 2024-03-07T07:40:20.882 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.881+0000 7f292796c700 1 -- 172.21.15.160:0/2481004754 wait complete. 2024-03-07T07:40:21.041 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.909+0000 7f52a8b8e700 1 -- 172.21.15.160:0/1737193317 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52a41074d0 msgr2=0x7f52a410f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:21.042 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.909+0000 7f52a8b8e700 1 --2- 172.21.15.160:0/1737193317 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52a41074d0 0x7f52a410f050 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f529c00b0d0 tx=0x7f529c00b3e0 comp rx=0 tx=0).stop 2024-03-07T07:40:21.042 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.909+0000 7f52a8b8e700 1 -- 172.21.15.160:0/1737193317 shutdown_connections 2024-03-07T07:40:21.042 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.909+0000 7f52a8b8e700 1 --2- 172.21.15.160:0/1737193317 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52a41074d0 0x7f52a410f050 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:21.042 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.909+0000 7f52a8b8e700 1 --2- 172.21.15.160:0/1737193317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52a406a8f0 0x7f52a406acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:21.042 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.909+0000 7f52a8b8e700 1 --2- 172.21.15.160:0/1737193317 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f52a406b2a0 0x7f52a4106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:21.042 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.909+0000 7f52a8b8e700 1 -- 172.21.15.160:0/1737193317 >> 172.21.15.160:0/1737193317 conn(0x7f52a40770a0 msgr2=0x7f52a40774b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:21.042 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.909+0000 7f52a8b8e700 1 -- 172.21.15.160:0/1737193317 shutdown_connections 2024-03-07T07:40:21.042 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.909+0000 7f52a8b8e700 1 -- 172.21.15.160:0/1737193317 wait complete. 2024-03-07T07:40:21.042 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.910+0000 7f52a8b8e700 1 Processor -- start 2024-03-07T07:40:21.042 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.910+0000 7f52a8b8e700 1 -- start start 2024-03-07T07:40:21.042 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.910+0000 7f52a8b8e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52a406a8f0 0x7f52a419e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:21.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.910+0000 7f52a8b8e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f52a406b2a0 0x7f52a419eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:21.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.910+0000 7f52a8b8e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52a41074d0 0x7f52a4198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:21.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.910+0000 7f52a8b8e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a4112470 con 0x7f52a41074d0 2024-03-07T07:40:21.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.910+0000 7f52a8b8e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a41122f0 con 0x7f52a406b2a0 2024-03-07T07:40:21.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.910+0000 7f52a8b8e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a41125f0 con 0x7f52a406a8f0 2024-03-07T07:40:21.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.910+0000 7f52a2d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52a41074d0 0x7f52a4198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:21.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.910+0000 7f52a1d9b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f52a406b2a0 0x7f52a419eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:21.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.910+0000 7f52a259c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52a406a8f0 0x7f52a419e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:21.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.910+0000 7f52a1d9b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f52a406b2a0 0x7f52a419eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.160:50254/0 (socket says 172.21.15.160:50254) 2024-03-07T07:40:21.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.910+0000 7f52a259c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52a406a8f0 0x7f52a419e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.160:43500/0 (socket says 172.21.15.160:43500) 2024-03-07T07:40:21.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.910+0000 7f52a2d9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52a41074d0 0x7f52a4198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:56896/0 (socket says 172.21.15.160:56896) 2024-03-07T07:40:21.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.910+0000 7f52a1d9b700 1 -- 172.21.15.160:0/3163697740 learned_addr learned my addr 172.21.15.160:0/3163697740 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:40:21.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.911+0000 7f52a1d9b700 1 -- 172.21.15.160:0/3163697740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52a406a8f0 msgr2=0x7f52a419e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:21.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.911+0000 7f52a1d9b700 1 --2- 172.21.15.160:0/3163697740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52a406a8f0 0x7f52a419e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:21.044 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.911+0000 7f52a1d9b700 1 -- 172.21.15.160:0/3163697740 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52a41074d0 msgr2=0x7f52a4198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:21.044 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.911+0000 7f52a1d9b700 1 --2- 172.21.15.160:0/3163697740 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52a41074d0 0x7f52a4198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:21.044 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.911+0000 7f52a1d9b700 1 -- 172.21.15.160:0/3163697740 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a4198f70 con 0x7f52a406b2a0 2024-03-07T07:40:21.044 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.911+0000 7f52a1d9b700 1 --2- 172.21.15.160:0/3163697740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f52a406b2a0 0x7f52a419eb00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f529800e940 tx=0x7f529800ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:21.044 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.911+0000 7f52937fe700 1 -- 172.21.15.160:0/3163697740 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529800c9e0 con 0x7f52a406b2a0 2024-03-07T07:40:21.044 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.911+0000 7f52937fe700 1 -- 172.21.15.160:0/3163697740 <== mon.2 v2:172.21.15.177:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f529800cb80 con 0x7f52a406b2a0 2024-03-07T07:40:21.044 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.911+0000 7f52a8b8e700 1 -- 172.21.15.160:0/3163697740 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52a4199260 con 0x7f52a406b2a0 2024-03-07T07:40:21.044 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.911+0000 7f52937fe700 1 -- 172.21.15.160:0/3163697740 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5298018660 con 0x7f52a406b2a0 2024-03-07T07:40:21.044 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.911+0000 7f52a8b8e700 1 -- 172.21.15.160:0/3163697740 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52a41a5540 con 0x7f52a406b2a0 2024-03-07T07:40:21.044 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.912+0000 7f52a8b8e700 1 -- 172.21.15.160:0/3163697740 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52a4103880 con 0x7f52a406b2a0 2024-03-07T07:40:21.044 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.914+0000 7f52937fe700 1 -- 172.21.15.160:0/3163697740 <== mon.2 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 17) v1 ==== 136805+0+0 (secure 0 0 0) 0x7f5298010430 con 0x7f52a406b2a0 2024-03-07T07:40:21.044 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.915+0000 7f52937fe700 1 --2- 172.21.15.160:0/3163697740 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f528c0a1f20 0x7f528c0a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:21.044 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.915+0000 7f52a259c700 1 --2- 172.21.15.160:0/3163697740 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f528c0a1f20 0x7f528c0a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:21.045 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.915+0000 7f52937fe700 1 -- 172.21.15.160:0/3163697740 <== mon.2 v2:172.21.15.177:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f52980cdf40 con 0x7f52a406b2a0 2024-03-07T07:40:21.045 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.916+0000 7f52a259c700 1 --2- 172.21.15.160:0/3163697740 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f528c0a1f20 0x7f528c0a43e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5294009c60 tx=0x7f5294009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:21.045 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:20.917+0000 7f52937fe700 1 -- 172.21.15.160:0/3163697740 <== mon.2 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f529809b210 con 0x7f52a406b2a0 2024-03-07T07:40:21.124 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:21.123+0000 7f52a8b8e700 1 -- 172.21.15.160:0/3163697740 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f52a402d9f0 con 0x7f52a406b2a0 2024-03-07T07:40:21.124 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:21.123+0000 7f52937fe700 1 -- 172.21.15.160:0/3163697740 <== mon.2 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f529809ab10 con 0x7f52a406b2a0 2024-03-07T07:40:21.124 INFO:teuthology.orchestra.run.smithi160.stdout:219043332143 2024-03-07T07:40:21.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:21.125+0000 7f52a8b8e700 1 -- 172.21.15.160:0/3163697740 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f528c0a1f20 msgr2=0x7f528c0a43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:21.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:21.125+0000 7f52a8b8e700 1 --2- 172.21.15.160:0/3163697740 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f528c0a1f20 0x7f528c0a43e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5294009c60 tx=0x7f5294009260 comp rx=0 tx=0).stop 2024-03-07T07:40:21.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:21.125+0000 7f52a8b8e700 1 -- 172.21.15.160:0/3163697740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f52a406b2a0 msgr2=0x7f52a419eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:21.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:21.125+0000 7f52a8b8e700 1 --2- 172.21.15.160:0/3163697740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f52a406b2a0 0x7f52a419eb00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f529800e940 tx=0x7f529800ec50 comp rx=0 tx=0).stop 2024-03-07T07:40:21.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:21.125+0000 7f52a8b8e700 1 -- 172.21.15.160:0/3163697740 shutdown_connections 2024-03-07T07:40:21.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:21.125+0000 7f52a8b8e700 1 --2- 172.21.15.160:0/3163697740 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52a41074d0 0x7f52a4198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:21.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:21.125+0000 7f52a8b8e700 1 --2- 172.21.15.160:0/3163697740 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f528c0a1f20 0x7f528c0a43e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:21.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:21.125+0000 7f52a8b8e700 1 --2- 172.21.15.160:0/3163697740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52a406a8f0 0x7f52a419e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:21.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:21.125+0000 7f52a8b8e700 1 --2- 172.21.15.160:0/3163697740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f52a406b2a0 0x7f52a419eb00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:21.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:21.125+0000 7f52a8b8e700 1 -- 172.21.15.160:0/3163697740 >> 172.21.15.160:0/3163697740 conn(0x7f52a40770a0 msgr2=0x7f52a40ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:21.127 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:21.126+0000 7f52a8b8e700 1 -- 172.21.15.160:0/3163697740 shutdown_connections 2024-03-07T07:40:21.127 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:21.126+0000 7f52a8b8e700 1 -- 172.21.15.160:0/3163697740 wait complete. 2024-03-07T07:40:21.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.523+0000 7f290d1d7700 1 -- 172.21.15.160:0/630667402 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2908106d40 msgr2=0x7f29081071c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:21.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.523+0000 7f290d1d7700 1 --2- 172.21.15.160:0/630667402 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2908106d40 0x7f29081071c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f28fc0099c0 tx=0x7f28fc009cd0 comp rx=0 tx=0).stop 2024-03-07T07:40:21.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.524+0000 7f290d1d7700 1 -- 172.21.15.160:0/630667402 shutdown_connections 2024-03-07T07:40:21.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.524+0000 7f290d1d7700 1 --2- 172.21.15.160:0/630667402 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2908107700 0x7f290810dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:21.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.524+0000 7f290d1d7700 1 --2- 172.21.15.160:0/630667402 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2908106080 0x7f2908106460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:21.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.524+0000 7f290d1d7700 1 --2- 172.21.15.160:0/630667402 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2908106d40 0x7f29081071c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:21.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.524+0000 7f290d1d7700 1 -- 172.21.15.160:0/630667402 >> 172.21.15.160:0/630667402 conn(0x7f2908077220 msgr2=0x7f29080fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:21.143 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.524+0000 7f290d1d7700 1 -- 172.21.15.160:0/630667402 shutdown_connections 2024-03-07T07:40:21.143 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.524+0000 7f290d1d7700 1 -- 172.21.15.160:0/630667402 wait complete. 2024-03-07T07:40:21.143 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.524+0000 7f290d1d7700 1 Processor -- start 2024-03-07T07:40:21.143 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.525+0000 7f290d1d7700 1 -- start start 2024-03-07T07:40:21.143 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.525+0000 7f290d1d7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2908106080 0x7f2908199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:21.143 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.525+0000 7f290d1d7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2908106d40 0x7f2908199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:21.143 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.525+0000 7f290d1d7700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2908107700 0x7f29081a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:21.143 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.525+0000 7f290d1d7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2908111320 con 0x7f2908106d40 2024-03-07T07:40:21.143 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.525+0000 7f290d1d7700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29081111a0 con 0x7f2908107700 2024-03-07T07:40:21.143 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.525+0000 7f290d1d7700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29081114a0 con 0x7f2908106080 2024-03-07T07:40:21.143 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.525+0000 7f290659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2908106d40 0x7f2908199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:21.143 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.525+0000 7f2906d9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2908106080 0x7f2908199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:21.144 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.525+0000 7f2906d9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2908106080 0x7f2908199640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.160:43456/0 (socket says 172.21.15.160:43456) 2024-03-07T07:40:21.144 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.525+0000 7f290659c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2908106d40 0x7f2908199b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:56858/0 (socket says 172.21.15.160:56858) 2024-03-07T07:40:21.144 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.525+0000 7f2906d9d700 1 -- 172.21.15.160:0/3057835239 learned_addr learned my addr 172.21.15.160:0/3057835239 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:40:21.144 INFO:teuthology.orchestra.run.smithi160.stdout:309237645343 2024-03-07T07:40:21.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.525+0000 7f290759e700 1 --2- 172.21.15.160:0/3057835239 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2908107700 0x7f29081a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:21.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.525+0000 7f2906d9d700 1 -- 172.21.15.160:0/3057835239 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2908107700 msgr2=0x7f29081a46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:21.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.525+0000 7f2906d9d700 1 --2- 172.21.15.160:0/3057835239 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2908107700 0x7f29081a46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:21.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.525+0000 7f2906d9d700 1 -- 172.21.15.160:0/3057835239 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2908106d40 msgr2=0x7f2908199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:21.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.525+0000 7f2906d9d700 1 --2- 172.21.15.160:0/3057835239 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2908106d40 0x7f2908199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:21.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.525+0000 7f2906d9d700 1 -- 172.21.15.160:0/3057835239 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29081a4c00 con 0x7f2908106080 2024-03-07T07:40:21.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.525+0000 7f290659c700 1 --2- 172.21.15.160:0/3057835239 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2908106d40 0x7f2908199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:40:21.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.525+0000 7f2906d9d700 1 --2- 172.21.15.160:0/3057835239 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2908106080 0x7f2908199640 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f28f800e9a0 tx=0x7f28f800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:21.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.526+0000 7f28effff700 1 -- 172.21.15.160:0/3057835239 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f800ca10 con 0x7f2908106080 2024-03-07T07:40:21.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.526+0000 7f28effff700 1 -- 172.21.15.160:0/3057835239 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f28f800cbb0 con 0x7f2908106080 2024-03-07T07:40:21.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.526+0000 7f290d1d7700 1 -- 172.21.15.160:0/3057835239 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29081a4e30 con 0x7f2908106080 2024-03-07T07:40:21.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.526+0000 7f28effff700 1 -- 172.21.15.160:0/3057835239 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f8018680 con 0x7f2908106080 2024-03-07T07:40:21.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.526+0000 7f290d1d7700 1 -- 172.21.15.160:0/3057835239 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29081a5280 con 0x7f2908106080 2024-03-07T07:40:21.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.526+0000 7f290d1d7700 1 -- 172.21.15.160:0/3057835239 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2908109560 con 0x7f2908106080 2024-03-07T07:40:21.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.527+0000 7f28effff700 1 -- 172.21.15.160:0/3057835239 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) v1 ==== 136805+0+0 (secure 0 0 0) 0x7f28f8010430 con 0x7f2908106080 2024-03-07T07:40:21.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.528+0000 7f28effff700 1 --2- 172.21.15.160:0/3057835239 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f28f00a1ea0 0x7f28f00a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:21.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.528+0000 7f290659c700 1 --2- 172.21.15.160:0/3057835239 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f28f00a1ea0 0x7f28f00a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:21.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.528+0000 7f28effff700 1 -- 172.21.15.160:0/3057835239 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f28f80ce150 con 0x7f2908106080 2024-03-07T07:40:21.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.528+0000 7f290659c700 1 --2- 172.21.15.160:0/3057835239 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f28f00a1ea0 0x7f28f00a4360 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2908101660 tx=0x7f28fc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:21.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.533+0000 7f28effff700 1 -- 172.21.15.160:0/3057835239 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f28f809b3e0 con 0x7f2908106080 2024-03-07T07:40:21.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.736+0000 7f290d1d7700 1 -- 172.21.15.160:0/3057835239 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f290802da90 con 0x7f2908106080 2024-03-07T07:40:21.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.736+0000 7f28effff700 1 -- 172.21.15.160:0/3057835239 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f28f809ace0 con 0x7f2908106080 2024-03-07T07:40:21.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.739+0000 7f290d1d7700 1 -- 172.21.15.160:0/3057835239 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f28f00a1ea0 msgr2=0x7f28f00a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:21.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.739+0000 7f290d1d7700 1 --2- 172.21.15.160:0/3057835239 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f28f00a1ea0 0x7f28f00a4360 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2908101660 tx=0x7f28fc005fb0 comp rx=0 tx=0).stop 2024-03-07T07:40:21.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.739+0000 7f290d1d7700 1 -- 172.21.15.160:0/3057835239 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2908106080 msgr2=0x7f2908199640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:21.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.739+0000 7f290d1d7700 1 --2- 172.21.15.160:0/3057835239 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2908106080 0x7f2908199640 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f28f800e9a0 tx=0x7f28f800ecb0 comp rx=0 tx=0).stop 2024-03-07T07:40:21.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.739+0000 7f290d1d7700 1 -- 172.21.15.160:0/3057835239 shutdown_connections 2024-03-07T07:40:21.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.739+0000 7f290d1d7700 1 --2- 172.21.15.160:0/3057835239 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2908107700 0x7f29081a46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:21.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.739+0000 7f290d1d7700 1 --2- 172.21.15.160:0/3057835239 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f28f00a1ea0 0x7f28f00a4360 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:21.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.739+0000 7f290d1d7700 1 --2- 172.21.15.160:0/3057835239 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2908106080 0x7f2908199640 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:21.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.739+0000 7f290d1d7700 1 --2- 172.21.15.160:0/3057835239 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2908106d40 0x7f2908199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:21.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.739+0000 7f290d1d7700 1 -- 172.21.15.160:0/3057835239 >> 172.21.15.160:0/3057835239 conn(0x7f2908077220 msgr2=0x7f29080fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:21.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.739+0000 7f290d1d7700 1 -- 172.21.15.160:0/3057835239 shutdown_connections 2024-03-07T07:40:21.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:19.739+0000 7f290d1d7700 1 -- 172.21.15.160:0/3057835239 wait complete. 2024-03-07T07:40:21.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:21 smithi178 ceph-mon[36650]: pgmap v323: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 5.3 KiB/s, 0 keys/s, 1 objects/s recovering 2024-03-07T07:40:21.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:21 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/2481004754' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-07T07:40:21.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:21 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/3163697740' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-07T07:40:21.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:21 smithi177 ceph-mon[36578]: pgmap v323: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 5.3 KiB/s, 0 keys/s, 1 objects/s recovering 2024-03-07T07:40:21.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:21 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/2481004754' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-07T07:40:21.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:21 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/3163697740' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-07T07:40:21.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:21 smithi160 ceph-mon[31963]: pgmap v323: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 5.3 KiB/s, 0 keys/s, 1 objects/s recovering 2024-03-07T07:40:21.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:21 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/2481004754' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-07T07:40:21.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:21 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/3163697740' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-07T07:40:21.781 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645339 got 309237645343 for osd.10 2024-03-07T07:40:21.782 DEBUG:teuthology.parallel:result is None 2024-03-07T07:40:22.391 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332138 got 219043332143 for osd.7 2024-03-07T07:40:22.391 DEBUG:teuthology.parallel:result is None 2024-03-07T07:40:23.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:23 smithi160 ceph-mon[31963]: pgmap v324: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 9 B/s, 0 keys/s, 0 objects/s recovering 2024-03-07T07:40:23.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:23 smithi177 ceph-mon[36578]: pgmap v324: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 9 B/s, 0 keys/s, 0 objects/s recovering 2024-03-07T07:40:23.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:23 smithi178 ceph-mon[36650]: pgmap v324: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 9 B/s, 0 keys/s, 0 objects/s recovering 2024-03-07T07:40:23.895 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920825 got 141733920831 for osd.4 2024-03-07T07:40:23.896 DEBUG:teuthology.parallel:result is None 2024-03-07T07:40:24.528 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070499 got 253403070504 for osd.8 2024-03-07T07:40:24.529 DEBUG:teuthology.parallel:result is None 2024-03-07T07:40:25.384 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.041+0000 7f9806a61700 1 -- 172.21.15.160:0/2624622611 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9800108180 msgr2=0x7f9800108560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:25.384 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.041+0000 7f9806a61700 1 --2- 172.21.15.160:0/2624622611 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9800108180 0x7f9800108560 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f97f40099c0 tx=0x7f97f4009cd0 comp rx=0 tx=0).stop 2024-03-07T07:40:25.385 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.041+0000 7f9806a61700 1 -- 172.21.15.160:0/2624622611 shutdown_connections 2024-03-07T07:40:25.385 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.041+0000 7f9806a61700 1 --2- 172.21.15.160:0/2624622611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9800102b60 0x7f980010efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:25.385 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.041+0000 7f9806a61700 1 --2- 172.21.15.160:0/2624622611 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9800108180 0x7f9800108560 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:25.385 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.041+0000 7f9806a61700 1 --2- 172.21.15.160:0/2624622611 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98001021a0 0x7f9800102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:25.385 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.041+0000 7f9806a61700 1 -- 172.21.15.160:0/2624622611 >> 172.21.15.160:0/2624622611 conn(0x7f9800079280 msgr2=0x7f9800079690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:25.385 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.041+0000 7f9806a61700 1 -- 172.21.15.160:0/2624622611 shutdown_connections 2024-03-07T07:40:25.385 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.041+0000 7f9806a61700 1 -- 172.21.15.160:0/2624622611 wait complete. 2024-03-07T07:40:25.385 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.041+0000 7f9806a61700 1 Processor -- start 2024-03-07T07:40:25.385 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.041+0000 7f9806a61700 1 -- start start 2024-03-07T07:40:25.385 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.042+0000 7f9806a61700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98001021a0 0x7f9800074450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:25.385 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.042+0000 7f9806a61700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9800102b60 0x7f9800074990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:25.385 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.042+0000 7f9806a61700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9800108180 0x7f9800074ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:25.386 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.042+0000 7f9806a61700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9800112400 con 0x7f9800102b60 2024-03-07T07:40:25.386 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.042+0000 7f9806a61700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9800112280 con 0x7f98001021a0 2024-03-07T07:40:25.386 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.042+0000 7f9806a61700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9800112580 con 0x7f9800108180 2024-03-07T07:40:25.386 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.042+0000 7f97ff7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9800102b60 0x7f9800074990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:25.386 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.042+0000 7f97ff7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9800102b60 0x7f9800074990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:56916/0 (socket says 172.21.15.160:56916) 2024-03-07T07:40:25.386 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.042+0000 7f97ff7fe700 1 -- 172.21.15.160:0/2391615498 learned_addr learned my addr 172.21.15.160:0/2391615498 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:40:25.386 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.042+0000 7f97fffff700 1 --2- 172.21.15.160:0/2391615498 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98001021a0 0x7f9800074450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:25.386 INFO:teuthology.orchestra.run.smithi160.stdout:287762808869 2024-03-07T07:40:25.387 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.042+0000 7f9804ffe700 1 --2- 172.21.15.160:0/2391615498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9800108180 0x7f9800074ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:25.387 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.042+0000 7f97ff7fe700 1 -- 172.21.15.160:0/2391615498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9800108180 msgr2=0x7f9800074ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:25.387 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.042+0000 7f97ff7fe700 1 --2- 172.21.15.160:0/2391615498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9800108180 0x7f9800074ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:25.387 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.042+0000 7f97ff7fe700 1 -- 172.21.15.160:0/2391615498 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98001021a0 msgr2=0x7f9800074450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:25.387 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.042+0000 7f97ff7fe700 1 --2- 172.21.15.160:0/2391615498 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98001021a0 0x7f9800074450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:25.387 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.042+0000 7f97ff7fe700 1 -- 172.21.15.160:0/2391615498 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98001a6e00 con 0x7f9800102b60 2024-03-07T07:40:25.387 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.042+0000 7f97ff7fe700 1 --2- 172.21.15.160:0/2391615498 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9800102b60 0x7f9800074990 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f97f000e9a0 tx=0x7f97f000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:25.387 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.043+0000 7f97fd7fa700 1 -- 172.21.15.160:0/2391615498 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f000ca10 con 0x7f9800102b60 2024-03-07T07:40:25.387 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.043+0000 7f9806a61700 1 -- 172.21.15.160:0/2391615498 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98001a70f0 con 0x7f9800102b60 2024-03-07T07:40:25.387 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.043+0000 7f97fd7fa700 1 -- 172.21.15.160:0/2391615498 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f97f000cbb0 con 0x7f9800102b60 2024-03-07T07:40:25.387 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.043+0000 7f97fd7fa700 1 -- 172.21.15.160:0/2391615498 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f0018680 con 0x7f9800102b60 2024-03-07T07:40:25.387 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.043+0000 7f9806a61700 1 -- 172.21.15.160:0/2391615498 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98001a7680 con 0x7f9800102b60 2024-03-07T07:40:25.388 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.044+0000 7f9806a61700 1 -- 172.21.15.160:0/2391615498 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9800068e10 con 0x7f9800102b60 2024-03-07T07:40:25.388 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.045+0000 7f97fd7fa700 1 -- 172.21.15.160:0/2391615498 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) v1 ==== 136805+0+0 (secure 0 0 0) 0x7f97f0010430 con 0x7f9800102b60 2024-03-07T07:40:25.388 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.045+0000 7f97fd7fa700 1 --2- 172.21.15.160:0/2391615498 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f97e80a1ea0 0x7f97e80a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:25.388 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.045+0000 7f97fffff700 1 --2- 172.21.15.160:0/2391615498 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f97e80a1ea0 0x7f97e80a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:25.388 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.045+0000 7f97fd7fa700 1 -- 172.21.15.160:0/2391615498 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f97f00ce600 con 0x7f9800102b60 2024-03-07T07:40:25.388 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.046+0000 7f97fffff700 1 --2- 172.21.15.160:0/2391615498 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f97e80a1ea0 0x7f97e80a4360 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f97f4000c00 tx=0x7f97f4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:25.388 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.049+0000 7f97fd7fa700 1 -- 172.21.15.160:0/2391615498 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f97f009b8d0 con 0x7f9800102b60 2024-03-07T07:40:25.388 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.252+0000 7f9806a61700 1 -- 172.21.15.160:0/2391615498 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f9800075980 con 0x7f9800102b60 2024-03-07T07:40:25.388 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.253+0000 7f97fd7fa700 1 -- 172.21.15.160:0/2391615498 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f97f009b1d0 con 0x7f9800102b60 2024-03-07T07:40:25.388 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.256+0000 7f9806a61700 1 -- 172.21.15.160:0/2391615498 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f97e80a1ea0 msgr2=0x7f97e80a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:25.388 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.256+0000 7f9806a61700 1 --2- 172.21.15.160:0/2391615498 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f97e80a1ea0 0x7f97e80a4360 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f97f4000c00 tx=0x7f97f4005fb0 comp rx=0 tx=0).stop 2024-03-07T07:40:25.388 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.256+0000 7f9806a61700 1 -- 172.21.15.160:0/2391615498 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9800102b60 msgr2=0x7f9800074990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:25.388 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.256+0000 7f9806a61700 1 --2- 172.21.15.160:0/2391615498 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9800102b60 0x7f9800074990 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f97f000e9a0 tx=0x7f97f000ecb0 comp rx=0 tx=0).stop 2024-03-07T07:40:25.388 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.256+0000 7f9806a61700 1 -- 172.21.15.160:0/2391615498 shutdown_connections 2024-03-07T07:40:25.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.256+0000 7f9806a61700 1 --2- 172.21.15.160:0/2391615498 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f97e80a1ea0 0x7f97e80a4360 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:25.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.256+0000 7f9806a61700 1 --2- 172.21.15.160:0/2391615498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9800108180 0x7f9800074ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:25.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.256+0000 7f9806a61700 1 --2- 172.21.15.160:0/2391615498 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98001021a0 0x7f9800074450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:25.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.256+0000 7f9806a61700 1 --2- 172.21.15.160:0/2391615498 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9800102b60 0x7f9800074990 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:25.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.256+0000 7f9806a61700 1 -- 172.21.15.160:0/2391615498 >> 172.21.15.160:0/2391615498 conn(0x7f9800079280 msgr2=0x7f9800100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:25.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.256+0000 7f9806a61700 1 -- 172.21.15.160:0/2391615498 shutdown_connections 2024-03-07T07:40:25.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.256+0000 7f9806a61700 1 -- 172.21.15.160:0/2391615498 wait complete. 2024-03-07T07:40:25.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:25 smithi177 ceph-mon[36578]: pgmap v325: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-03-07T07:40:25.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:25 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/2391615498' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-07T07:40:25.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:25 smithi160 ceph-mon[31963]: pgmap v325: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-03-07T07:40:25.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:25 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/2391615498' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-07T07:40:25.791 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.790+0000 7fdff8d49700 1 -- 172.21.15.160:0/1689774877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdff41071c0 msgr2=0x7fdff41075a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:25.791 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.790+0000 7fdff8d49700 1 --2- 172.21.15.160:0/1689774877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdff41071c0 0x7fdff41075a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4009990 tx=0x7fdfe4009ca0 comp rx=0 tx=0).stop 2024-03-07T07:40:25.791 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.790+0000 7fdff8d49700 1 -- 172.21.15.160:0/1689774877 shutdown_connections 2024-03-07T07:40:25.791 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.790+0000 7fdff8d49700 1 --2- 172.21.15.160:0/1689774877 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdff4101100 0x7fdff410f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:25.791 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.790+0000 7fdff8d49700 1 --2- 172.21.15.160:0/1689774877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdff41071c0 0x7fdff41075a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:25.791 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.790+0000 7fdff8d49700 1 --2- 172.21.15.160:0/1689774877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdff4100740 0x7fdff4100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:25.791 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.790+0000 7fdff8d49700 1 -- 172.21.15.160:0/1689774877 >> 172.21.15.160:0/1689774877 conn(0x7fdff4077650 msgr2=0x7fdff40fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:25.791 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.790+0000 7fdff8d49700 1 -- 172.21.15.160:0/1689774877 shutdown_connections 2024-03-07T07:40:25.791 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.790+0000 7fdff8d49700 1 -- 172.21.15.160:0/1689774877 wait complete. 2024-03-07T07:40:25.793 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.791+0000 7fdff8d49700 1 Processor -- start 2024-03-07T07:40:25.793 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.791+0000 7fdff8d49700 1 -- start start 2024-03-07T07:40:25.794 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.791+0000 7fdff8d49700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdff4100740 0x7fdff419e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:25.794 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.791+0000 7fdff8d49700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdff4101100 0x7fdff419eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:25.794 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.791+0000 7fdff8d49700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdff41071c0 0x7fdff4198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:25.794 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.791+0000 7fdff8d49700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff4112580 con 0x7fdff4101100 2024-03-07T07:40:25.794 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.791+0000 7fdff8d49700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff4112400 con 0x7fdff4100740 2024-03-07T07:40:25.794 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.791+0000 7fdff8d49700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff4112700 con 0x7fdff41071c0 2024-03-07T07:40:25.794 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.791+0000 7fdff1d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdff4101100 0x7fdff419eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:25.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.791+0000 7fdff1d9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdff4101100 0x7fdff419eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:56940/0 (socket says 172.21.15.160:56940) 2024-03-07T07:40:25.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.791+0000 7fdff1d9b700 1 -- 172.21.15.160:0/3376294721 learned_addr learned my addr 172.21.15.160:0/3376294721 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:40:25.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.792+0000 7fdff259c700 1 --2- 172.21.15.160:0/3376294721 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdff4100740 0x7fdff419e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:25.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.792+0000 7fdff2d9d700 1 --2- 172.21.15.160:0/3376294721 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdff41071c0 0x7fdff4198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:25.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.792+0000 7fdff1d9b700 1 -- 172.21.15.160:0/3376294721 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdff41071c0 msgr2=0x7fdff4198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:25.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.792+0000 7fdff1d9b700 1 --2- 172.21.15.160:0/3376294721 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdff41071c0 0x7fdff4198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:25.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.792+0000 7fdff1d9b700 1 -- 172.21.15.160:0/3376294721 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdff4100740 msgr2=0x7fdff419e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:25.796 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.792+0000 7fdff1d9b700 1 --2- 172.21.15.160:0/3376294721 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdff4100740 0x7fdff419e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:25.796 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.792+0000 7fdff1d9b700 1 -- 172.21.15.160:0/3376294721 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff4198fe0 con 0x7fdff4101100 2024-03-07T07:40:25.796 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.792+0000 7fdff259c700 1 --2- 172.21.15.160:0/3376294721 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdff4100740 0x7fdff419e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:40:25.796 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.792+0000 7fdff1d9b700 1 --2- 172.21.15.160:0/3376294721 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdff4101100 0x7fdff419eb70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdfe800e9a0 tx=0x7fdfe800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:25.796 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.792+0000 7fdfe37fe700 1 -- 172.21.15.160:0/3376294721 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe800ca10 con 0x7fdff4101100 2024-03-07T07:40:25.796 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.792+0000 7fdff8d49700 1 -- 172.21.15.160:0/3376294721 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdff41992d0 con 0x7fdff4101100 2024-03-07T07:40:25.796 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.792+0000 7fdfe37fe700 1 -- 172.21.15.160:0/3376294721 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdfe800cbb0 con 0x7fdff4101100 2024-03-07T07:40:25.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.792+0000 7fdfe37fe700 1 -- 172.21.15.160:0/3376294721 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe8018680 con 0x7fdff4101100 2024-03-07T07:40:25.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.792+0000 7fdff8d49700 1 -- 172.21.15.160:0/3376294721 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdff41a5490 con 0x7fdff4101100 2024-03-07T07:40:25.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.793+0000 7fdff8d49700 1 -- 172.21.15.160:0/3376294721 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdff410a790 con 0x7fdff4101100 2024-03-07T07:40:25.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.794+0000 7fdfe37fe700 1 -- 172.21.15.160:0/3376294721 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) v1 ==== 136805+0+0 (secure 0 0 0) 0x7fdfe8010430 con 0x7fdff4101100 2024-03-07T07:40:25.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.795+0000 7fdfe37fe700 1 --2- 172.21.15.160:0/3376294721 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fdfdc0a1f20 0x7fdfdc0a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:25.799 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.795+0000 7fdff259c700 1 --2- 172.21.15.160:0/3376294721 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fdfdc0a1f20 0x7fdfdc0a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:25.799 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.795+0000 7fdfe37fe700 1 -- 172.21.15.160:0/3376294721 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fdfe80ce130 con 0x7fdff4101100 2024-03-07T07:40:25.799 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.795+0000 7fdff259c700 1 --2- 172.21.15.160:0/3376294721 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fdfdc0a1f20 0x7fdfdc0a43e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdfe400b2e0 tx=0x7fdfe4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:25.802 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:25.802+0000 7fdfe37fe700 1 -- 172.21.15.160:0/3376294721 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fdfe809b400 con 0x7fdff4101100 2024-03-07T07:40:25.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:25 smithi178 ceph-mon[36650]: pgmap v325: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-03-07T07:40:25.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:25 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/2391615498' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-07T07:40:26.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.011+0000 7fdff8d49700 1 -- 172.21.15.160:0/3376294721 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7fdff4199ce0 con 0x7fdff4101100 2024-03-07T07:40:26.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.011+0000 7fdfe37fe700 1 -- 172.21.15.160:0/3376294721 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fdfe809ad00 con 0x7fdff4101100 2024-03-07T07:40:26.012 INFO:teuthology.orchestra.run.smithi160.stdout:343597383708 2024-03-07T07:40:26.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.013+0000 7fdff8d49700 1 -- 172.21.15.160:0/3376294721 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fdfdc0a1f20 msgr2=0x7fdfdc0a43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:26.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.013+0000 7fdff8d49700 1 --2- 172.21.15.160:0/3376294721 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fdfdc0a1f20 0x7fdfdc0a43e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdfe400b2e0 tx=0x7fdfe4005960 comp rx=0 tx=0).stop 2024-03-07T07:40:26.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.013+0000 7fdff8d49700 1 -- 172.21.15.160:0/3376294721 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdff4101100 msgr2=0x7fdff419eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:26.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.013+0000 7fdff8d49700 1 --2- 172.21.15.160:0/3376294721 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdff4101100 0x7fdff419eb70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdfe800e9a0 tx=0x7fdfe800ecb0 comp rx=0 tx=0).stop 2024-03-07T07:40:26.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.013+0000 7fdff8d49700 1 -- 172.21.15.160:0/3376294721 shutdown_connections 2024-03-07T07:40:26.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.013+0000 7fdff8d49700 1 --2- 172.21.15.160:0/3376294721 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fdfdc0a1f20 0x7fdfdc0a43e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:26.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.013+0000 7fdff8d49700 1 --2- 172.21.15.160:0/3376294721 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdff41071c0 0x7fdff4198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:26.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.013+0000 7fdff8d49700 1 --2- 172.21.15.160:0/3376294721 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdff4100740 0x7fdff419e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:26.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.013+0000 7fdff8d49700 1 --2- 172.21.15.160:0/3376294721 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdff4101100 0x7fdff419eb70 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:26.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.013+0000 7fdff8d49700 1 -- 172.21.15.160:0/3376294721 >> 172.21.15.160:0/3376294721 conn(0x7fdff4077650 msgr2=0x7fdff4105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:26.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.013+0000 7fdff8d49700 1 -- 172.21.15.160:0/3376294721 shutdown_connections 2024-03-07T07:40:26.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.013+0000 7fdff8d49700 1 -- 172.21.15.160:0/3376294721 wait complete. 2024-03-07T07:40:26.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.421+0000 7fa2ab050700 1 -- 172.21.15.160:0/325735831 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2a4106810 msgr2=0x7fa2a4106bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:26.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.421+0000 7fa2ab050700 1 --2- 172.21.15.160:0/325735831 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2a4106810 0x7fa2a4106bf0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa298009990 tx=0x7fa298009ca0 comp rx=0 tx=0).stop 2024-03-07T07:40:26.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.421+0000 7fa2ab050700 1 -- 172.21.15.160:0/325735831 shutdown_connections 2024-03-07T07:40:26.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.421+0000 7fa2ab050700 1 --2- 172.21.15.160:0/325735831 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2a41011d0 0x7fa2a410d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:26.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.421+0000 7fa2ab050700 1 --2- 172.21.15.160:0/325735831 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2a4106810 0x7fa2a4106bf0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:26.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.421+0000 7fa2ab050700 1 --2- 172.21.15.160:0/325735831 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2a4100810 0x7fa2a4100c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:26.424 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.421+0000 7fa2ab050700 1 -- 172.21.15.160:0/325735831 >> 172.21.15.160:0/325735831 conn(0x7fa2a4077650 msgr2=0x7fa2a40fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:26.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.421+0000 7fa2ab050700 1 -- 172.21.15.160:0/325735831 shutdown_connections 2024-03-07T07:40:26.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.421+0000 7fa2ab050700 1 -- 172.21.15.160:0/325735831 wait complete. 2024-03-07T07:40:26.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.422+0000 7fa2ab050700 1 Processor -- start 2024-03-07T07:40:26.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.422+0000 7fa2ab050700 1 -- start start 2024-03-07T07:40:26.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.422+0000 7fa2ab050700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2a4100810 0x7fa2a41a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:26.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.422+0000 7fa2ab050700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2a41011d0 0x7fa2a41a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:26.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.422+0000 7fa2ab050700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2a4106810 0x7fa2a419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:26.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.422+0000 7fa2ab050700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a41102c0 con 0x7fa2a4100810 2024-03-07T07:40:26.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.422+0000 7fa2ab050700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a4110140 con 0x7fa2a41011d0 2024-03-07T07:40:26.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.422+0000 7fa2ab050700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a4110440 con 0x7fa2a4106810 2024-03-07T07:40:26.426 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.422+0000 7fa2a8dec700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2a4100810 0x7fa2a41a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:26.426 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.423+0000 7fa2a3fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2a41011d0 0x7fa2a41a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:26.426 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.423+0000 7fa2a8dec700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2a4100810 0x7fa2a41a0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:56958/0 (socket says 172.21.15.160:56958) 2024-03-07T07:40:26.426 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.423+0000 7fa2a8dec700 1 -- 172.21.15.160:0/4197788048 learned_addr learned my addr 172.21.15.160:0/4197788048 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:40:26.426 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.423+0000 7fa2a95ed700 1 --2- 172.21.15.160:0/4197788048 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2a4106810 0x7fa2a419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:26.426 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.423+0000 7fa2a3fff700 1 -- 172.21.15.160:0/4197788048 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2a4106810 msgr2=0x7fa2a419a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:26.426 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.423+0000 7fa2a3fff700 1 --2- 172.21.15.160:0/4197788048 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2a4106810 0x7fa2a419a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:26.426 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.423+0000 7fa2a3fff700 1 -- 172.21.15.160:0/4197788048 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2a4100810 msgr2=0x7fa2a41a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:26.426 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.423+0000 7fa2a3fff700 1 --2- 172.21.15.160:0/4197788048 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2a4100810 0x7fa2a41a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:26.426 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.423+0000 7fa2a3fff700 1 -- 172.21.15.160:0/4197788048 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a419b110 con 0x7fa2a41011d0 2024-03-07T07:40:26.426 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.423+0000 7fa2a3fff700 1 --2- 172.21.15.160:0/4197788048 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2a41011d0 0x7fa2a41a0c90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa29000e9f0 tx=0x7fa29000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:26.426 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.423+0000 7fa2a1ffb700 1 -- 172.21.15.160:0/4197788048 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29000c9c0 con 0x7fa2a41011d0 2024-03-07T07:40:26.426 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.423+0000 7fa2a1ffb700 1 -- 172.21.15.160:0/4197788048 <== mon.2 v2:172.21.15.177:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa29000cb60 con 0x7fa2a41011d0 2024-03-07T07:40:26.426 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.423+0000 7fa2ab050700 1 -- 172.21.15.160:0/4197788048 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2a419b400 con 0x7fa2a41011d0 2024-03-07T07:40:26.427 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.423+0000 7fa2a1ffb700 1 -- 172.21.15.160:0/4197788048 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa290018680 con 0x7fa2a41011d0 2024-03-07T07:40:26.427 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.423+0000 7fa2ab050700 1 -- 172.21.15.160:0/4197788048 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2a41a7620 con 0x7fa2a41011d0 2024-03-07T07:40:26.427 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.424+0000 7fa2ab050700 1 -- 172.21.15.160:0/4197788048 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2a4101f50 con 0x7fa2a41011d0 2024-03-07T07:40:26.428 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.425+0000 7fa2a1ffb700 1 -- 172.21.15.160:0/4197788048 <== mon.2 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 17) v1 ==== 136805+0+0 (secure 0 0 0) 0x7fa290010430 con 0x7fa2a41011d0 2024-03-07T07:40:26.428 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.426+0000 7fa2a1ffb700 1 --2- 172.21.15.160:0/4197788048 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa2940a1e80 0x7fa2940a4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:26.428 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.426+0000 7fa2a8dec700 1 --2- 172.21.15.160:0/4197788048 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa2940a1e80 0x7fa2940a4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:26.428 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.426+0000 7fa2a1ffb700 1 -- 172.21.15.160:0/4197788048 <== mon.2 v2:172.21.15.177:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa2900ce000 con 0x7fa2a41011d0 2024-03-07T07:40:26.428 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.426+0000 7fa2a8dec700 1 --2- 172.21.15.160:0/4197788048 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa2940a1e80 0x7fa2940a4340 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa298009750 tx=0x7fa29801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:26.431 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.430+0000 7fa2a1ffb700 1 -- 172.21.15.160:0/4197788048 <== mon.2 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa29009b310 con 0x7fa2a41011d0 2024-03-07T07:40:26.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:26 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/3376294721' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-07T07:40:26.632 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.630+0000 7fa2ab050700 1 -- 172.21.15.160:0/4197788048 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa2a402d9f0 con 0x7fa2a41011d0 2024-03-07T07:40:26.632 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:26 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/3376294721' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-07T07:40:26.632 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.631+0000 7fa2a1ffb700 1 -- 172.21.15.160:0/4197788048 <== mon.2 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa29009ac10 con 0x7fa2a41011d0 2024-03-07T07:40:26.633 INFO:teuthology.orchestra.run.smithi160.stdout:55834574927 2024-03-07T07:40:26.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.633+0000 7fa2ab050700 1 -- 172.21.15.160:0/4197788048 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa2940a1e80 msgr2=0x7fa2940a4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:26.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.633+0000 7fa2ab050700 1 --2- 172.21.15.160:0/4197788048 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa2940a1e80 0x7fa2940a4340 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa298009750 tx=0x7fa29801a040 comp rx=0 tx=0).stop 2024-03-07T07:40:26.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.633+0000 7fa2ab050700 1 -- 172.21.15.160:0/4197788048 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2a41011d0 msgr2=0x7fa2a41a0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:26.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.633+0000 7fa2ab050700 1 --2- 172.21.15.160:0/4197788048 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2a41011d0 0x7fa2a41a0c90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa29000e9f0 tx=0x7fa29000ed00 comp rx=0 tx=0).stop 2024-03-07T07:40:26.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.633+0000 7fa2ab050700 1 -- 172.21.15.160:0/4197788048 shutdown_connections 2024-03-07T07:40:26.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.633+0000 7fa2ab050700 1 --2- 172.21.15.160:0/4197788048 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa2940a1e80 0x7fa2940a4340 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:26.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.633+0000 7fa2ab050700 1 --2- 172.21.15.160:0/4197788048 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2a4106810 0x7fa2a419a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:26.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.633+0000 7fa2ab050700 1 --2- 172.21.15.160:0/4197788048 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2a4100810 0x7fa2a41a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:26.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.633+0000 7fa2ab050700 1 --2- 172.21.15.160:0/4197788048 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2a41011d0 0x7fa2a41a0c90 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:26.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.633+0000 7fa2ab050700 1 -- 172.21.15.160:0/4197788048 >> 172.21.15.160:0/4197788048 conn(0x7fa2a4077650 msgr2=0x7fa2a4103640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:26.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.633+0000 7fa2ab050700 1 -- 172.21.15.160:0/4197788048 shutdown_connections 2024-03-07T07:40:26.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:26.633+0000 7fa2ab050700 1 -- 172.21.15.160:0/4197788048 wait complete. 2024-03-07T07:40:26.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:26 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/3376294721' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-07T07:40:26.828 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383702 got 343597383708 for osd.11 2024-03-07T07:40:26.828 DEBUG:teuthology.parallel:result is None 2024-03-07T07:40:27.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:27 smithi160 ceph-mon[31963]: pgmap v326: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-07T07:40:27.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:27 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/4197788048' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-07T07:40:27.484 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808864 got 287762808869 for osd.9 2024-03-07T07:40:27.484 DEBUG:teuthology.parallel:result is None 2024-03-07T07:40:27.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:27 smithi177 ceph-mon[36578]: pgmap v326: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-07T07:40:27.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:27 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/4197788048' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-07T07:40:27.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:27 smithi178 ceph-mon[36650]: pgmap v326: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-07T07:40:27.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:27 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/4197788048' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-07T07:40:27.938 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574920 got 55834574927 for osd.1 2024-03-07T07:40:27.938 DEBUG:teuthology.parallel:result is None 2024-03-07T07:40:27.938 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-07T07:40:27.939 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph pg dump --format=json 2024-03-07T07:40:28.541 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:40:29.657 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:29 smithi160 ceph-mon[31963]: pgmap v327: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-07T07:40:29.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:29 smithi178 ceph-mon[36650]: pgmap v327: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-07T07:40:29.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:29 smithi177 ceph-mon[36578]: pgmap v327: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-07T07:40:30.203 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.203+0000 7f6e3849b700 1 -- 172.21.15.160:0/2043461384 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e301074b0 msgr2=0x7f6e3010f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:30.204 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.203+0000 7f6e3849b700 1 --2- 172.21.15.160:0/2043461384 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e301074b0 0x7f6e3010f080 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6e2800b0d0 tx=0x7f6e2800b3e0 comp rx=0 tx=0).stop 2024-03-07T07:40:30.204 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.203+0000 7f6e3849b700 1 -- 172.21.15.160:0/2043461384 shutdown_connections 2024-03-07T07:40:30.204 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.203+0000 7f6e3849b700 1 --2- 172.21.15.160:0/2043461384 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e301074b0 0x7f6e3010f080 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:30.204 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.203+0000 7f6e3849b700 1 --2- 172.21.15.160:0/2043461384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e3006ab10 0x7f6e3006aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:30.204 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.203+0000 7f6e3849b700 1 --2- 172.21.15.160:0/2043461384 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e3006b4c0 0x7f6e30106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:30.204 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.203+0000 7f6e3849b700 1 -- 172.21.15.160:0/2043461384 >> 172.21.15.160:0/2043461384 conn(0x7f6e30078150 msgr2=0x7f6e30078560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:30.204 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.203+0000 7f6e3849b700 1 -- 172.21.15.160:0/2043461384 shutdown_connections 2024-03-07T07:40:30.204 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.203+0000 7f6e3849b700 1 -- 172.21.15.160:0/2043461384 wait complete. 2024-03-07T07:40:30.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.204+0000 7f6e3849b700 1 Processor -- start 2024-03-07T07:40:30.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.204+0000 7f6e3849b700 1 -- start start 2024-03-07T07:40:30.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.204+0000 7f6e3849b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e3006ab10 0x7f6e3019e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:30.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.204+0000 7f6e3849b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e3006b4c0 0x7f6e3019eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:30.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.204+0000 7f6e3849b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e301074b0 0x7f6e30198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:30.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.204+0000 7f6e3849b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e301124b0 con 0x7f6e301074b0 2024-03-07T07:40:30.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.204+0000 7f6e3849b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e30112330 con 0x7f6e3006ab10 2024-03-07T07:40:30.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.204+0000 7f6e3849b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e30112630 con 0x7f6e3006b4c0 2024-03-07T07:40:30.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.204+0000 7f6e36a38700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e301074b0 0x7f6e30198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:30.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.204+0000 7f6e35a36700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e3006b4c0 0x7f6e3019eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:30.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.204+0000 7f6e36a38700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e301074b0 0x7f6e30198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:38284/0 (socket says 172.21.15.160:38284) 2024-03-07T07:40:30.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.204+0000 7f6e36a38700 1 -- 172.21.15.160:0/574020886 learned_addr learned my addr 172.21.15.160:0/574020886 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:40:30.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.204+0000 7f6e36237700 1 --2- 172.21.15.160:0/574020886 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e3006ab10 0x7f6e3019e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:30.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.204+0000 7f6e36a38700 1 -- 172.21.15.160:0/574020886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e3006b4c0 msgr2=0x7f6e3019eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:30.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.204+0000 7f6e36a38700 1 --2- 172.21.15.160:0/574020886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e3006b4c0 0x7f6e3019eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:30.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.204+0000 7f6e36a38700 1 -- 172.21.15.160:0/574020886 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e3006ab10 msgr2=0x7f6e3019e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:30.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.204+0000 7f6e36a38700 1 --2- 172.21.15.160:0/574020886 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e3006ab10 0x7f6e3019e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:30.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.204+0000 7f6e36a38700 1 -- 172.21.15.160:0/574020886 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e30198ff0 con 0x7f6e301074b0 2024-03-07T07:40:30.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.205+0000 7f6e36a38700 1 --2- 172.21.15.160:0/574020886 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e301074b0 0x7f6e30198710 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6e28006070 tx=0x7f6e28007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:30.208 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.205+0000 7f6e277fe700 1 -- 172.21.15.160:0/574020886 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e28003c00 con 0x7f6e301074b0 2024-03-07T07:40:30.208 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.205+0000 7f6e277fe700 1 -- 172.21.15.160:0/574020886 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6e28022070 con 0x7f6e301074b0 2024-03-07T07:40:30.208 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.205+0000 7f6e3849b700 1 -- 172.21.15.160:0/574020886 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e30199280 con 0x7f6e301074b0 2024-03-07T07:40:30.208 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.205+0000 7f6e277fe700 1 -- 172.21.15.160:0/574020886 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2801b6c0 con 0x7f6e301074b0 2024-03-07T07:40:30.208 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.205+0000 7f6e3849b700 1 -- 172.21.15.160:0/574020886 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e301a53a0 con 0x7f6e301074b0 2024-03-07T07:40:30.208 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.206+0000 7f6e3849b700 1 -- 172.21.15.160:0/574020886 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e301038b0 con 0x7f6e301074b0 2024-03-07T07:40:30.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.207+0000 7f6e277fe700 1 -- 172.21.15.160:0/574020886 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) v1 ==== 136805+0+0 (secure 0 0 0) 0x7f6e2801d470 con 0x7f6e301074b0 2024-03-07T07:40:30.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.207+0000 7f6e277fe700 1 --2- 172.21.15.160:0/574020886 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f6e1c0a1ea0 0x7f6e1c0a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:30.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.208+0000 7f6e36237700 1 --2- 172.21.15.160:0/574020886 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f6e1c0a1ea0 0x7f6e1c0a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:30.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.208+0000 7f6e277fe700 1 -- 172.21.15.160:0/574020886 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6e280cf130 con 0x7f6e301074b0 2024-03-07T07:40:30.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.208+0000 7f6e36237700 1 --2- 172.21.15.160:0/574020886 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f6e1c0a1ea0 0x7f6e1c0a4360 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c009c60 tx=0x7f6e2c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:30.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.211+0000 7f6e277fe700 1 -- 172.21.15.160:0/574020886 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6e2809c490 con 0x7f6e301074b0 2024-03-07T07:40:30.424 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.424+0000 7f6e3849b700 1 -- 172.21.15.160:0/574020886 --> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6e30199bf0 con 0x7f6e1c0a1ea0 2024-03-07T07:40:30.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.434+0000 7f6e277fe700 1 -- 172.21.15.160:0/574020886 <== mgr.14152 v2:172.21.15.160:6800/419630619 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+259032 (secure 0 0 0) 0x7f6e30199bf0 con 0x7f6e1c0a1ea0 2024-03-07T07:40:30.438 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:40:30.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.438+0000 7f6e3849b700 1 -- 172.21.15.160:0/574020886 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f6e1c0a1ea0 msgr2=0x7f6e1c0a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:30.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.438+0000 7f6e3849b700 1 --2- 172.21.15.160:0/574020886 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f6e1c0a1ea0 0x7f6e1c0a4360 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c009c60 tx=0x7f6e2c009260 comp rx=0 tx=0).stop 2024-03-07T07:40:30.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.438+0000 7f6e3849b700 1 -- 172.21.15.160:0/574020886 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e301074b0 msgr2=0x7f6e30198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:30.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.438+0000 7f6e3849b700 1 --2- 172.21.15.160:0/574020886 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e301074b0 0x7f6e30198710 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6e28006070 tx=0x7f6e28007a20 comp rx=0 tx=0).stop 2024-03-07T07:40:30.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.438+0000 7f6e3849b700 1 -- 172.21.15.160:0/574020886 shutdown_connections 2024-03-07T07:40:30.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.438+0000 7f6e3849b700 1 --2- 172.21.15.160:0/574020886 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e301074b0 0x7f6e30198710 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:30.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.438+0000 7f6e3849b700 1 --2- 172.21.15.160:0/574020886 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e3006ab10 0x7f6e3019e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:30.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.438+0000 7f6e3849b700 1 --2- 172.21.15.160:0/574020886 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f6e1c0a1ea0 0x7f6e1c0a4360 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:30.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.438+0000 7f6e3849b700 1 --2- 172.21.15.160:0/574020886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e3006b4c0 0x7f6e3019eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:30.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.438+0000 7f6e3849b700 1 -- 172.21.15.160:0/574020886 >> 172.21.15.160:0/574020886 conn(0x7f6e30078150 msgr2=0x7f6e300fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:30.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.438+0000 7f6e3849b700 1 -- 172.21.15.160:0/574020886 shutdown_connections 2024-03-07T07:40:30.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:30.438+0000 7f6e3849b700 1 -- 172.21.15.160:0/574020886 wait complete. 2024-03-07T07:40:30.441 INFO:teuthology.orchestra.run.smithi160.stderr:dumped all 2024-03-07T07:40:31.167 INFO:teuthology.orchestra.run.smithi160.stdout:{"pg_ready":true,"pg_map":{"version":328,"stamp":"2024-03-07T07:40:30.289551+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653515,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":562,"num_read_kb":392,"num_write":337,"num_write_kb":3045,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1239877,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":272,"ondisk_log_size":272,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360756,"kb_used_data":11516,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581068,"statfs":{"total":1151940427776,"available":1151571013632,"internally_reserved":0,"allocated":11792384,"data_stored":7428873,"data_compressed":190473,"data_compressed_allocated":253952,"data_compressed_original":2039808,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":45,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002966"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.358756+0000","last_change":"2024-03-07T07:39:14.614168+0000","last_active":"2024-03-07T07:40:09.358756+0000","last_peered":"2024-03-07T07:40:09.358756+0000","last_clean":"2024-03-07T07:40:09.358756+0000","last_became_active":"2024-03-07T07:39:14.613708+0000","last_became_peered":"2024-03-07T07:39:14.613708+0000","last_unstale":"2024-03-07T07:40:09.358756+0000","last_undegraded":"2024-03-07T07:40:09.358756+0000","last_fullsized":"2024-03-07T07:40:09.358756+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:08:07.286470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.357141+0000","last_change":"2024-03-07T07:39:14.609425+0000","last_active":"2024-03-07T07:40:09.357141+0000","last_peered":"2024-03-07T07:40:09.357141+0000","last_clean":"2024-03-07T07:40:09.357141+0000","last_became_active":"2024-03-07T07:39:14.609060+0000","last_became_peered":"2024-03-07T07:39:14.609060+0000","last_unstale":"2024-03-07T07:40:09.357141+0000","last_undegraded":"2024-03-07T07:40:09.357141+0000","last_fullsized":"2024-03-07T07:40:09.357141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:37:33.773358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.357094+0000","last_change":"2024-03-07T07:39:14.609985+0000","last_active":"2024-03-07T07:40:09.357094+0000","last_peered":"2024-03-07T07:40:09.357094+0000","last_clean":"2024-03-07T07:40:09.357094+0000","last_became_active":"2024-03-07T07:39:14.609490+0000","last_became_peered":"2024-03-07T07:39:14.609490+0000","last_unstale":"2024-03-07T07:40:09.357094+0000","last_undegraded":"2024-03-07T07:40:09.357094+0000","last_fullsized":"2024-03-07T07:40:09.357094+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T19:09:24.134718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.357909+0000","last_change":"2024-03-07T07:39:14.620482+0000","last_active":"2024-03-07T07:40:09.357909+0000","last_peered":"2024-03-07T07:40:09.357909+0000","last_clean":"2024-03-07T07:40:09.357909+0000","last_became_active":"2024-03-07T07:39:14.620100+0000","last_became_peered":"2024-03-07T07:39:14.620100+0000","last_unstale":"2024-03-07T07:40:09.357909+0000","last_undegraded":"2024-03-07T07:40:09.357909+0000","last_fullsized":"2024-03-07T07:40:09.357909+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:03:20.291833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.361639+0000","last_change":"2024-03-07T07:39:14.603228+0000","last_active":"2024-03-07T07:40:09.361639+0000","last_peered":"2024-03-07T07:40:09.361639+0000","last_clean":"2024-03-07T07:40:09.361639+0000","last_became_active":"2024-03-07T07:39:14.602874+0000","last_became_peered":"2024-03-07T07:39:14.602874+0000","last_unstale":"2024-03-07T07:40:09.361639+0000","last_undegraded":"2024-03-07T07:40:09.361639+0000","last_fullsized":"2024-03-07T07:40:09.361639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:26:17.762271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.456982+0000","last_change":"2024-03-07T07:40:09.456982+0000","last_active":"2024-03-07T07:40:09.456982+0000","last_peered":"2024-03-07T07:40:09.456982+0000","last_clean":"2024-03-07T07:40:09.456982+0000","last_became_active":"2024-03-07T07:40:09.456688+0000","last_became_peered":"2024-03-07T07:40:09.456688+0000","last_unstale":"2024-03-07T07:40:09.456982+0000","last_undegraded":"2024-03-07T07:40:09.456982+0000","last_fullsized":"2024-03-07T07:40:09.456982+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:00:34.723624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.359257+0000","last_change":"2024-03-07T07:39:14.615259+0000","last_active":"2024-03-07T07:40:09.359257+0000","last_peered":"2024-03-07T07:40:09.359257+0000","last_clean":"2024-03-07T07:40:09.359257+0000","last_became_active":"2024-03-07T07:39:14.614878+0000","last_became_peered":"2024-03-07T07:39:14.614878+0000","last_unstale":"2024-03-07T07:40:09.359257+0000","last_undegraded":"2024-03-07T07:40:09.359257+0000","last_fullsized":"2024-03-07T07:40:09.359257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:22:28.905360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.361729+0000","last_change":"2024-03-07T07:39:14.614518+0000","last_active":"2024-03-07T07:40:09.361729+0000","last_peered":"2024-03-07T07:40:09.361729+0000","last_clean":"2024-03-07T07:40:09.361729+0000","last_became_active":"2024-03-07T07:39:14.614308+0000","last_became_peered":"2024-03-07T07:39:14.614308+0000","last_unstale":"2024-03-07T07:40:09.361729+0000","last_undegraded":"2024-03-07T07:40:09.361729+0000","last_fullsized":"2024-03-07T07:40:09.361729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:54:54.908066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.363466+0000","last_change":"2024-03-07T07:39:14.615360+0000","last_active":"2024-03-07T07:40:09.363466+0000","last_peered":"2024-03-07T07:40:09.363466+0000","last_clean":"2024-03-07T07:40:09.363466+0000","last_became_active":"2024-03-07T07:39:14.615036+0000","last_became_peered":"2024-03-07T07:39:14.615036+0000","last_unstale":"2024-03-07T07:40:09.363466+0000","last_undegraded":"2024-03-07T07:40:09.363466+0000","last_fullsized":"2024-03-07T07:40:09.363466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T19:17:07.160773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.457981+0000","last_change":"2024-03-07T07:40:09.457981+0000","last_active":"2024-03-07T07:40:09.457981+0000","last_peered":"2024-03-07T07:40:09.457981+0000","last_clean":"2024-03-07T07:40:09.457981+0000","last_became_active":"2024-03-07T07:40:09.457624+0000","last_became_peered":"2024-03-07T07:40:09.457624+0000","last_unstale":"2024-03-07T07:40:09.457981+0000","last_undegraded":"2024-03-07T07:40:09.457981+0000","last_fullsized":"2024-03-07T07:40:09.457981+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:29:40.916809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.359183+0000","last_change":"2024-03-07T07:39:14.611800+0000","last_active":"2024-03-07T07:40:09.359183+0000","last_peered":"2024-03-07T07:40:09.359183+0000","last_clean":"2024-03-07T07:40:09.359183+0000","last_became_active":"2024-03-07T07:39:14.611482+0000","last_became_peered":"2024-03-07T07:39:14.611482+0000","last_unstale":"2024-03-07T07:40:09.359183+0000","last_undegraded":"2024-03-07T07:40:09.359183+0000","last_fullsized":"2024-03-07T07:40:09.359183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:26:36.361716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.357853+0000","last_change":"2024-03-07T07:39:14.621686+0000","last_active":"2024-03-07T07:40:09.357853+0000","last_peered":"2024-03-07T07:40:09.357853+0000","last_clean":"2024-03-07T07:40:09.357853+0000","last_became_active":"2024-03-07T07:39:14.620815+0000","last_became_peered":"2024-03-07T07:39:14.620815+0000","last_unstale":"2024-03-07T07:40:09.357853+0000","last_undegraded":"2024-03-07T07:40:09.357853+0000","last_fullsized":"2024-03-07T07:40:09.357853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:33:59.972215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.359614+0000","last_change":"2024-03-07T07:39:14.606045+0000","last_active":"2024-03-07T07:40:09.359614+0000","last_peered":"2024-03-07T07:40:09.359614+0000","last_clean":"2024-03-07T07:40:09.359614+0000","last_became_active":"2024-03-07T07:39:14.605726+0000","last_became_peered":"2024-03-07T07:39:14.605726+0000","last_unstale":"2024-03-07T07:40:09.359614+0000","last_undegraded":"2024-03-07T07:40:09.359614+0000","last_fullsized":"2024-03-07T07:40:09.359614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:13:51.899507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.356909+0000","last_change":"2024-03-07T07:39:14.605435+0000","last_active":"2024-03-07T07:40:09.356909+0000","last_peered":"2024-03-07T07:40:09.356909+0000","last_clean":"2024-03-07T07:40:09.356909+0000","last_became_active":"2024-03-07T07:39:14.605127+0000","last_became_peered":"2024-03-07T07:39:14.605127+0000","last_unstale":"2024-03-07T07:40:09.356909+0000","last_undegraded":"2024-03-07T07:40:09.356909+0000","last_fullsized":"2024-03-07T07:40:09.356909+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:15:58.206921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.356800+0000","last_change":"2024-03-07T07:39:14.617166+0000","last_active":"2024-03-07T07:40:09.356800+0000","last_peered":"2024-03-07T07:40:09.356800+0000","last_clean":"2024-03-07T07:40:09.356800+0000","last_became_active":"2024-03-07T07:39:14.616615+0000","last_became_peered":"2024-03-07T07:39:14.616615+0000","last_unstale":"2024-03-07T07:40:09.356800+0000","last_undegraded":"2024-03-07T07:40:09.356800+0000","last_fullsized":"2024-03-07T07:40:09.356800+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:25:05.086822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.362757+0000","last_change":"2024-03-07T07:39:14.611990+0000","last_active":"2024-03-07T07:40:09.362757+0000","last_peered":"2024-03-07T07:40:09.362757+0000","last_clean":"2024-03-07T07:40:09.362757+0000","last_became_active":"2024-03-07T07:39:14.610728+0000","last_became_peered":"2024-03-07T07:39:14.610728+0000","last_unstale":"2024-03-07T07:40:09.362757+0000","last_undegraded":"2024-03-07T07:40:09.362757+0000","last_fullsized":"2024-03-07T07:40:09.362757+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:12:54.850583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.358880+0000","last_change":"2024-03-07T07:39:14.615409+0000","last_active":"2024-03-07T07:40:09.358880+0000","last_peered":"2024-03-07T07:40:09.358880+0000","last_clean":"2024-03-07T07:40:09.358880+0000","last_became_active":"2024-03-07T07:39:14.615003+0000","last_became_peered":"2024-03-07T07:39:14.615003+0000","last_unstale":"2024-03-07T07:40:09.358880+0000","last_undegraded":"2024-03-07T07:40:09.358880+0000","last_fullsized":"2024-03-07T07:40:09.358880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:33:56.464718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.457012+0000","last_change":"2024-03-07T07:39:14.610501+0000","last_active":"2024-03-07T07:40:09.457012+0000","last_peered":"2024-03-07T07:40:09.457012+0000","last_clean":"2024-03-07T07:40:09.457012+0000","last_became_active":"2024-03-07T07:39:14.610190+0000","last_became_peered":"2024-03-07T07:39:14.610190+0000","last_unstale":"2024-03-07T07:40:09.457012+0000","last_undegraded":"2024-03-07T07:40:09.457012+0000","last_fullsized":"2024-03-07T07:40:09.457012+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:05:31.360270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.361517+0000","last_change":"2024-03-07T07:39:14.609373+0000","last_active":"2024-03-07T07:40:09.361517+0000","last_peered":"2024-03-07T07:40:09.361517+0000","last_clean":"2024-03-07T07:40:09.361517+0000","last_became_active":"2024-03-07T07:39:14.608174+0000","last_became_peered":"2024-03-07T07:39:14.608174+0000","last_unstale":"2024-03-07T07:40:09.361517+0000","last_undegraded":"2024-03-07T07:40:09.361517+0000","last_fullsized":"2024-03-07T07:40:09.361517+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:25:18.704239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.457099+0000","last_change":"2024-03-07T07:39:13.830499+0000","last_active":"2024-03-07T07:40:09.457099+0000","last_peered":"2024-03-07T07:40:09.457099+0000","last_clean":"2024-03-07T07:40:09.457099+0000","last_became_active":"2024-03-07T07:39:13.830131+0000","last_became_peered":"2024-03-07T07:39:13.830131+0000","last_unstale":"2024-03-07T07:40:09.457099+0000","last_undegraded":"2024-03-07T07:40:09.457099+0000","last_fullsized":"2024-03-07T07:40:09.457099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:48:14.338758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.362537+0000","last_change":"2024-03-07T07:39:14.612715+0000","last_active":"2024-03-07T07:40:09.362537+0000","last_peered":"2024-03-07T07:40:09.362537+0000","last_clean":"2024-03-07T07:40:09.362537+0000","last_became_active":"2024-03-07T07:39:14.612383+0000","last_became_peered":"2024-03-07T07:39:14.612383+0000","last_unstale":"2024-03-07T07:40:09.362537+0000","last_undegraded":"2024-03-07T07:40:09.362537+0000","last_fullsized":"2024-03-07T07:40:09.362537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:17:36.800396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.362682+0000","last_change":"2024-03-07T07:39:13.830901+0000","last_active":"2024-03-07T07:40:09.362682+0000","last_peered":"2024-03-07T07:40:09.362682+0000","last_clean":"2024-03-07T07:40:09.362682+0000","last_became_active":"2024-03-07T07:39:13.830581+0000","last_became_peered":"2024-03-07T07:39:13.830581+0000","last_unstale":"2024-03-07T07:40:09.362682+0000","last_undegraded":"2024-03-07T07:40:09.362682+0000","last_fullsized":"2024-03-07T07:40:09.362682+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:25:15.428381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.357233+0000","last_change":"2024-03-07T07:39:14.610876+0000","last_active":"2024-03-07T07:40:09.357233+0000","last_peered":"2024-03-07T07:40:09.357233+0000","last_clean":"2024-03-07T07:40:09.357233+0000","last_became_active":"2024-03-07T07:39:14.610584+0000","last_became_peered":"2024-03-07T07:39:14.610584+0000","last_unstale":"2024-03-07T07:40:09.357233+0000","last_undegraded":"2024-03-07T07:40:09.357233+0000","last_fullsized":"2024-03-07T07:40:09.357233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:55:11.818462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.377842+0000","last_change":"2024-03-07T07:40:09.377541+0000","last_active":"2024-03-07T07:40:09.377842+0000","last_peered":"2024-03-07T07:40:09.377842+0000","last_clean":"2024-03-07T07:40:09.377842+0000","last_became_active":"2024-03-07T07:40:09.362602+0000","last_became_peered":"2024-03-07T07:40:09.362602+0000","last_unstale":"2024-03-07T07:40:09.377842+0000","last_undegraded":"2024-03-07T07:40:09.377842+0000","last_fullsized":"2024-03-07T07:40:09.377842+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:50:47.570967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.363140+0000","last_change":"2024-03-07T07:39:14.607666+0000","last_active":"2024-03-07T07:40:09.363140+0000","last_peered":"2024-03-07T07:40:09.363140+0000","last_clean":"2024-03-07T07:40:09.363140+0000","last_became_active":"2024-03-07T07:39:14.607339+0000","last_became_peered":"2024-03-07T07:39:14.607339+0000","last_unstale":"2024-03-07T07:40:09.363140+0000","last_undegraded":"2024-03-07T07:40:09.363140+0000","last_fullsized":"2024-03-07T07:40:09.363140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:17:36.349529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.361319+0000","last_change":"2024-03-07T07:39:13.829179+0000","last_active":"2024-03-07T07:40:09.361319+0000","last_peered":"2024-03-07T07:40:09.361319+0000","last_clean":"2024-03-07T07:40:09.361319+0000","last_became_active":"2024-03-07T07:39:13.828162+0000","last_became_peered":"2024-03-07T07:39:13.828162+0000","last_unstale":"2024-03-07T07:40:09.361319+0000","last_undegraded":"2024-03-07T07:40:09.361319+0000","last_fullsized":"2024-03-07T07:40:09.361319+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:38:00.330980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.367311+0000","last_change":"2024-03-07T07:39:14.621496+0000","last_active":"2024-03-07T07:40:09.367311+0000","last_peered":"2024-03-07T07:40:09.367311+0000","last_clean":"2024-03-07T07:40:09.367311+0000","last_became_active":"2024-03-07T07:39:14.621139+0000","last_became_peered":"2024-03-07T07:39:14.621139+0000","last_unstale":"2024-03-07T07:40:09.367311+0000","last_undegraded":"2024-03-07T07:40:09.367311+0000","last_fullsized":"2024-03-07T07:40:09.367311+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:24:45.553536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":97,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:27.941588+0000","last_change":"2024-03-07T07:39:13.834999+0000","last_active":"2024-03-07T07:40:27.941588+0000","last_peered":"2024-03-07T07:40:27.941588+0000","last_clean":"2024-03-07T07:40:27.941588+0000","last_became_active":"2024-03-07T07:39:13.834672+0000","last_became_peered":"2024-03-07T07:39:13.834672+0000","last_unstale":"2024-03-07T07:40:27.941588+0000","last_undegraded":"2024-03-07T07:40:27.941588+0000","last_fullsized":"2024-03-07T07:40:27.941588+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:55:57.944992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.360955+0000","last_change":"2024-03-07T07:39:14.605519+0000","last_active":"2024-03-07T07:40:09.360955+0000","last_peered":"2024-03-07T07:40:09.360955+0000","last_clean":"2024-03-07T07:40:09.360955+0000","last_became_active":"2024-03-07T07:39:14.605185+0000","last_became_peered":"2024-03-07T07:39:14.605185+0000","last_unstale":"2024-03-07T07:40:09.360955+0000","last_undegraded":"2024-03-07T07:40:09.360955+0000","last_fullsized":"2024-03-07T07:40:09.360955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:54:33.622751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.360037+0000","last_change":"2024-03-07T07:39:13.837186+0000","last_active":"2024-03-07T07:40:09.360037+0000","last_peered":"2024-03-07T07:40:09.360037+0000","last_clean":"2024-03-07T07:40:09.360037+0000","last_became_active":"2024-03-07T07:39:13.836807+0000","last_became_peered":"2024-03-07T07:39:13.836807+0000","last_unstale":"2024-03-07T07:40:09.360037+0000","last_undegraded":"2024-03-07T07:40:09.360037+0000","last_fullsized":"2024-03-07T07:40:09.360037+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:32:39.574197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.367151+0000","last_change":"2024-03-07T07:39:14.621503+0000","last_active":"2024-03-07T07:40:09.367151+0000","last_peered":"2024-03-07T07:40:09.367151+0000","last_clean":"2024-03-07T07:40:09.367151+0000","last_became_active":"2024-03-07T07:39:14.621146+0000","last_became_peered":"2024-03-07T07:39:14.621146+0000","last_unstale":"2024-03-07T07:40:09.367151+0000","last_undegraded":"2024-03-07T07:40:09.367151+0000","last_fullsized":"2024-03-07T07:40:09.367151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:50:15.270355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.359502+0000","last_change":"2024-03-07T07:39:13.838146+0000","last_active":"2024-03-07T07:40:09.359502+0000","last_peered":"2024-03-07T07:40:09.359502+0000","last_clean":"2024-03-07T07:40:09.359502+0000","last_became_active":"2024-03-07T07:39:13.837788+0000","last_became_peered":"2024-03-07T07:39:13.837788+0000","last_unstale":"2024-03-07T07:40:09.359502+0000","last_undegraded":"2024-03-07T07:40:09.359502+0000","last_fullsized":"2024-03-07T07:40:09.359502+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:50:52.154146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.361831+0000","last_change":"2024-03-07T07:39:14.605069+0000","last_active":"2024-03-07T07:40:09.361831+0000","last_peered":"2024-03-07T07:40:09.361831+0000","last_clean":"2024-03-07T07:40:09.361831+0000","last_became_active":"2024-03-07T07:39:14.604733+0000","last_became_peered":"2024-03-07T07:39:14.604733+0000","last_unstale":"2024-03-07T07:40:09.361831+0000","last_undegraded":"2024-03-07T07:40:09.361831+0000","last_fullsized":"2024-03-07T07:40:09.361831+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:49:08.312385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.366786+0000","last_change":"2024-03-07T07:39:13.837854+0000","last_active":"2024-03-07T07:40:09.366786+0000","last_peered":"2024-03-07T07:40:09.366786+0000","last_clean":"2024-03-07T07:40:09.366786+0000","last_became_active":"2024-03-07T07:39:13.837428+0000","last_became_peered":"2024-03-07T07:39:13.837428+0000","last_unstale":"2024-03-07T07:40:09.366786+0000","last_undegraded":"2024-03-07T07:40:09.366786+0000","last_fullsized":"2024-03-07T07:40:09.366786+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:35:15.369936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.456624+0000","last_change":"2024-03-07T07:39:14.606299+0000","last_active":"2024-03-07T07:40:09.456624+0000","last_peered":"2024-03-07T07:40:09.456624+0000","last_clean":"2024-03-07T07:40:09.456624+0000","last_became_active":"2024-03-07T07:39:14.605983+0000","last_became_peered":"2024-03-07T07:39:14.605983+0000","last_unstale":"2024-03-07T07:40:09.456624+0000","last_undegraded":"2024-03-07T07:40:09.456624+0000","last_fullsized":"2024-03-07T07:40:09.456624+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:53:16.658862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":93,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:27.941119+0000","last_change":"2024-03-07T07:39:13.840129+0000","last_active":"2024-03-07T07:40:27.941119+0000","last_peered":"2024-03-07T07:40:27.941119+0000","last_clean":"2024-03-07T07:40:27.941119+0000","last_became_active":"2024-03-07T07:39:13.839735+0000","last_became_peered":"2024-03-07T07:39:13.839735+0000","last_unstale":"2024-03-07T07:40:27.941119+0000","last_undegraded":"2024-03-07T07:40:27.941119+0000","last_fullsized":"2024-03-07T07:40:27.941119+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:31:38.711810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":71,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.363334+0000","last_change":"2024-03-07T07:39:14.606449+0000","last_active":"2024-03-07T07:40:09.363334+0000","last_peered":"2024-03-07T07:40:09.363334+0000","last_clean":"2024-03-07T07:40:09.363334+0000","last_became_active":"2024-03-07T07:39:14.606009+0000","last_became_peered":"2024-03-07T07:39:14.606009+0000","last_unstale":"2024-03-07T07:40:09.363334+0000","last_undegraded":"2024-03-07T07:40:09.363334+0000","last_fullsized":"2024-03-07T07:40:09.363334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:55:49.237517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.359363+0000","last_change":"2024-03-07T07:39:13.834097+0000","last_active":"2024-03-07T07:40:09.359363+0000","last_peered":"2024-03-07T07:40:09.359363+0000","last_clean":"2024-03-07T07:40:09.359363+0000","last_became_active":"2024-03-07T07:39:13.833749+0000","last_became_peered":"2024-03-07T07:39:13.833749+0000","last_unstale":"2024-03-07T07:40:09.359363+0000","last_undegraded":"2024-03-07T07:40:09.359363+0000","last_fullsized":"2024-03-07T07:40:09.359363+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:12:22.471596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.357241+0000","last_change":"2024-03-07T07:39:14.621000+0000","last_active":"2024-03-07T07:40:09.357241+0000","last_peered":"2024-03-07T07:40:09.357241+0000","last_clean":"2024-03-07T07:40:09.357241+0000","last_became_active":"2024-03-07T07:39:14.620708+0000","last_became_peered":"2024-03-07T07:39:14.620708+0000","last_unstale":"2024-03-07T07:40:09.357241+0000","last_undegraded":"2024-03-07T07:40:09.357241+0000","last_fullsized":"2024-03-07T07:40:09.357241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:26:48.363406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.367483+0000","last_change":"2024-03-07T07:39:13.837442+0000","last_active":"2024-03-07T07:40:09.367483+0000","last_peered":"2024-03-07T07:40:09.367483+0000","last_clean":"2024-03-07T07:40:09.367483+0000","last_became_active":"2024-03-07T07:39:13.836515+0000","last_became_peered":"2024-03-07T07:39:13.836515+0000","last_unstale":"2024-03-07T07:40:09.367483+0000","last_undegraded":"2024-03-07T07:40:09.367483+0000","last_fullsized":"2024-03-07T07:40:09.367483+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:19:08.312359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.363253+0000","last_change":"2024-03-07T07:39:14.619130+0000","last_active":"2024-03-07T07:40:09.363253+0000","last_peered":"2024-03-07T07:40:09.363253+0000","last_clean":"2024-03-07T07:40:09.363253+0000","last_became_active":"2024-03-07T07:39:14.618814+0000","last_became_peered":"2024-03-07T07:39:14.618814+0000","last_unstale":"2024-03-07T07:40:09.363253+0000","last_undegraded":"2024-03-07T07:40:09.363253+0000","last_fullsized":"2024-03-07T07:40:09.363253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:52:06.323318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'26","reported_seq":212,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:29.387418+0000","last_change":"2024-03-07T07:40:09.370340+0000","last_active":"2024-03-07T07:40:29.387418+0000","last_peered":"2024-03-07T07:40:29.387418+0000","last_clean":"2024-03-07T07:40:29.387418+0000","last_became_active":"2024-03-07T07:40:09.363828+0000","last_became_peered":"2024-03-07T07:40:09.363828+0000","last_unstale":"2024-03-07T07:40:29.387418+0000","last_undegraded":"2024-03-07T07:40:29.387418+0000","last_fullsized":"2024-03-07T07:40:29.387418+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:42:45.203744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58912,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":135,"num_read_kb":124,"num_write":27,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":58912,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.361444+0000","last_change":"2024-03-07T07:39:14.606539+0000","last_active":"2024-03-07T07:40:09.361444+0000","last_peered":"2024-03-07T07:40:09.361444+0000","last_clean":"2024-03-07T07:40:09.361444+0000","last_became_active":"2024-03-07T07:39:14.605832+0000","last_became_peered":"2024-03-07T07:39:14.605832+0000","last_unstale":"2024-03-07T07:40:09.361444+0000","last_undegraded":"2024-03-07T07:40:09.361444+0000","last_fullsized":"2024-03-07T07:40:09.361444+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:08:10.781906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.361006+0000","last_change":"2024-03-07T07:39:13.841526+0000","last_active":"2024-03-07T07:40:09.361006+0000","last_peered":"2024-03-07T07:40:09.361006+0000","last_clean":"2024-03-07T07:40:09.361006+0000","last_became_active":"2024-03-07T07:39:13.841153+0000","last_became_peered":"2024-03-07T07:39:13.841153+0000","last_unstale":"2024-03-07T07:40:09.361006+0000","last_undegraded":"2024-03-07T07:40:09.361006+0000","last_fullsized":"2024-03-07T07:40:09.361006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T19:38:41.264033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.359467+0000","last_change":"2024-03-07T07:39:14.603948+0000","last_active":"2024-03-07T07:40:09.359467+0000","last_peered":"2024-03-07T07:40:09.359467+0000","last_clean":"2024-03-07T07:40:09.359467+0000","last_became_active":"2024-03-07T07:39:14.603724+0000","last_became_peered":"2024-03-07T07:39:14.603724+0000","last_unstale":"2024-03-07T07:40:09.359467+0000","last_undegraded":"2024-03-07T07:40:09.359467+0000","last_fullsized":"2024-03-07T07:40:09.359467+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T19:29:59.296537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.366969+0000","last_change":"2024-03-07T07:39:13.831965+0000","last_active":"2024-03-07T07:40:09.366969+0000","last_peered":"2024-03-07T07:40:09.366969+0000","last_clean":"2024-03-07T07:40:09.366969+0000","last_became_active":"2024-03-07T07:39:13.831617+0000","last_became_peered":"2024-03-07T07:39:13.831617+0000","last_unstale":"2024-03-07T07:40:09.366969+0000","last_undegraded":"2024-03-07T07:40:09.366969+0000","last_fullsized":"2024-03-07T07:40:09.366969+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:46:49.060697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.359245+0000","last_change":"2024-03-07T07:39:14.616265+0000","last_active":"2024-03-07T07:40:09.359245+0000","last_peered":"2024-03-07T07:40:09.359245+0000","last_clean":"2024-03-07T07:40:09.359245+0000","last_became_active":"2024-03-07T07:39:14.615206+0000","last_became_peered":"2024-03-07T07:39:14.615206+0000","last_unstale":"2024-03-07T07:40:09.359245+0000","last_undegraded":"2024-03-07T07:40:09.359245+0000","last_fullsized":"2024-03-07T07:40:09.359245+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:39:21.997327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'4","reported_seq":97,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:27.954886+0000","last_change":"2024-03-07T07:39:13.833311+0000","last_active":"2024-03-07T07:40:27.954886+0000","last_peered":"2024-03-07T07:40:27.954886+0000","last_clean":"2024-03-07T07:40:27.954886+0000","last_became_active":"2024-03-07T07:39:13.832863+0000","last_became_peered":"2024-03-07T07:39:13.832863+0000","last_unstale":"2024-03-07T07:40:27.954886+0000","last_undegraded":"2024-03-07T07:40:27.954886+0000","last_fullsized":"2024-03-07T07:40:27.954886+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:43:50.988905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":75,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.358786+0000","last_change":"2024-03-07T07:39:14.615740+0000","last_active":"2024-03-07T07:40:09.358786+0000","last_peered":"2024-03-07T07:40:09.358786+0000","last_clean":"2024-03-07T07:40:09.358786+0000","last_became_active":"2024-03-07T07:39:14.615394+0000","last_became_peered":"2024-03-07T07:39:14.615394+0000","last_unstale":"2024-03-07T07:40:09.358786+0000","last_undegraded":"2024-03-07T07:40:09.358786+0000","last_fullsized":"2024-03-07T07:40:09.358786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:17:19.107374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.374773+0000","last_change":"2024-03-07T07:40:09.374343+0000","last_active":"2024-03-07T07:40:09.374773+0000","last_peered":"2024-03-07T07:40:09.374773+0000","last_clean":"2024-03-07T07:40:09.374773+0000","last_became_active":"2024-03-07T07:40:09.362272+0000","last_became_peered":"2024-03-07T07:40:09.362272+0000","last_unstale":"2024-03-07T07:40:09.374773+0000","last_undegraded":"2024-03-07T07:40:09.374773+0000","last_fullsized":"2024-03-07T07:40:09.374773+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:42:17.076409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.366972+0000","last_change":"2024-03-07T07:39:14.617134+0000","last_active":"2024-03-07T07:40:09.366972+0000","last_peered":"2024-03-07T07:40:09.366972+0000","last_clean":"2024-03-07T07:40:09.366972+0000","last_became_active":"2024-03-07T07:39:14.616813+0000","last_became_peered":"2024-03-07T07:39:14.616813+0000","last_unstale":"2024-03-07T07:40:09.366972+0000","last_undegraded":"2024-03-07T07:40:09.366972+0000","last_fullsized":"2024-03-07T07:40:09.366972+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:12:18.477309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.362021+0000","last_change":"2024-03-07T07:39:13.828783+0000","last_active":"2024-03-07T07:40:09.362021+0000","last_peered":"2024-03-07T07:40:09.362021+0000","last_clean":"2024-03-07T07:40:09.362021+0000","last_became_active":"2024-03-07T07:39:13.828427+0000","last_became_peered":"2024-03-07T07:39:13.828427+0000","last_unstale":"2024-03-07T07:40:09.362021+0000","last_undegraded":"2024-03-07T07:40:09.362021+0000","last_fullsized":"2024-03-07T07:40:09.362021+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:49:21.074014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.458018+0000","last_change":"2024-03-07T07:40:09.458018+0000","last_active":"2024-03-07T07:40:09.458018+0000","last_peered":"2024-03-07T07:40:09.458018+0000","last_clean":"2024-03-07T07:40:09.458018+0000","last_became_active":"2024-03-07T07:40:09.457643+0000","last_became_peered":"2024-03-07T07:40:09.457643+0000","last_unstale":"2024-03-07T07:40:09.458018+0000","last_undegraded":"2024-03-07T07:40:09.458018+0000","last_fullsized":"2024-03-07T07:40:09.458018+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:18:33.643312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.456732+0000","last_change":"2024-03-07T07:39:13.833342+0000","last_active":"2024-03-07T07:40:09.456732+0000","last_peered":"2024-03-07T07:40:09.456732+0000","last_clean":"2024-03-07T07:40:09.456732+0000","last_became_active":"2024-03-07T07:39:13.833013+0000","last_became_peered":"2024-03-07T07:39:13.833013+0000","last_unstale":"2024-03-07T07:40:09.456732+0000","last_undegraded":"2024-03-07T07:40:09.456732+0000","last_fullsized":"2024-03-07T07:40:09.456732+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:39:59.821206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.460404+0000","last_change":"2024-03-07T07:40:09.460404+0000","last_active":"2024-03-07T07:40:09.460404+0000","last_peered":"2024-03-07T07:40:09.460404+0000","last_clean":"2024-03-07T07:40:09.460404+0000","last_became_active":"2024-03-07T07:40:09.457470+0000","last_became_peered":"2024-03-07T07:40:09.457470+0000","last_unstale":"2024-03-07T07:40:09.460404+0000","last_undegraded":"2024-03-07T07:40:09.460404+0000","last_fullsized":"2024-03-07T07:40:09.460404+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:37:30.728304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.358964+0000","last_change":"2024-03-07T07:39:13.836434+0000","last_active":"2024-03-07T07:40:09.358964+0000","last_peered":"2024-03-07T07:40:09.358964+0000","last_clean":"2024-03-07T07:40:09.358964+0000","last_became_active":"2024-03-07T07:39:13.835953+0000","last_became_peered":"2024-03-07T07:39:13.835953+0000","last_unstale":"2024-03-07T07:40:09.358964+0000","last_undegraded":"2024-03-07T07:40:09.358964+0000","last_fullsized":"2024-03-07T07:40:09.358964+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:23:20.941845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.360385+0000","last_change":"2024-03-07T07:39:14.618062+0000","last_active":"2024-03-07T07:40:09.360385+0000","last_peered":"2024-03-07T07:40:09.360385+0000","last_clean":"2024-03-07T07:40:09.360385+0000","last_became_active":"2024-03-07T07:39:14.617715+0000","last_became_peered":"2024-03-07T07:39:14.617715+0000","last_unstale":"2024-03-07T07:40:09.360385+0000","last_undegraded":"2024-03-07T07:40:09.360385+0000","last_fullsized":"2024-03-07T07:40:09.360385+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:51:29.200771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.366909+0000","last_change":"2024-03-07T07:39:14.617522+0000","last_active":"2024-03-07T07:40:09.366909+0000","last_peered":"2024-03-07T07:40:09.366909+0000","last_clean":"2024-03-07T07:40:09.366909+0000","last_became_active":"2024-03-07T07:39:14.616862+0000","last_became_peered":"2024-03-07T07:39:14.616862+0000","last_unstale":"2024-03-07T07:40:09.366909+0000","last_undegraded":"2024-03-07T07:40:09.366909+0000","last_fullsized":"2024-03-07T07:40:09.366909+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:37:58.473662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":93,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:26.953365+0000","last_change":"2024-03-07T07:39:13.835090+0000","last_active":"2024-03-07T07:40:26.953365+0000","last_peered":"2024-03-07T07:40:26.953365+0000","last_clean":"2024-03-07T07:40:26.953365+0000","last_became_active":"2024-03-07T07:39:13.834765+0000","last_became_peered":"2024-03-07T07:39:13.834765+0000","last_unstale":"2024-03-07T07:40:26.953365+0000","last_undegraded":"2024-03-07T07:40:26.953365+0000","last_fullsized":"2024-03-07T07:40:26.953365+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:50:41.530959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":70,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.362316+0000","last_change":"2024-03-07T07:39:14.615224+0000","last_active":"2024-03-07T07:40:09.362316+0000","last_peered":"2024-03-07T07:40:09.362316+0000","last_clean":"2024-03-07T07:40:09.362316+0000","last_became_active":"2024-03-07T07:39:14.614942+0000","last_became_peered":"2024-03-07T07:39:14.614942+0000","last_unstale":"2024-03-07T07:40:09.362316+0000","last_undegraded":"2024-03-07T07:40:09.362316+0000","last_fullsized":"2024-03-07T07:40:09.362316+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:30:48.321950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.357608+0000","last_change":"2024-03-07T07:39:13.833245+0000","last_active":"2024-03-07T07:40:09.357608+0000","last_peered":"2024-03-07T07:40:09.357608+0000","last_clean":"2024-03-07T07:40:09.357608+0000","last_became_active":"2024-03-07T07:39:13.832837+0000","last_became_peered":"2024-03-07T07:39:13.832837+0000","last_unstale":"2024-03-07T07:40:09.357608+0000","last_undegraded":"2024-03-07T07:40:09.357608+0000","last_fullsized":"2024-03-07T07:40:09.357608+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:47:28.204394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":343,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.358899+0000","last_change":"2024-03-07T07:37:58.585313+0000","last_active":"2024-03-07T07:40:09.358899+0000","last_peered":"2024-03-07T07:40:09.358899+0000","last_clean":"2024-03-07T07:40:09.358899+0000","last_became_active":"2024-03-07T07:37:58.584808+0000","last_became_peered":"2024-03-07T07:37:58.584808+0000","last_unstale":"2024-03-07T07:40:09.358899+0000","last_undegraded":"2024-03-07T07:40:09.358899+0000","last_fullsized":"2024-03-07T07:40:09.358899+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:34:34.612837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:34:34.612837+0000","last_clean_scrub_stamp":"2024-03-07T07:34:34.612837+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:56:56.998601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.359380+0000","last_change":"2024-03-07T07:39:14.615907+0000","last_active":"2024-03-07T07:40:09.359380+0000","last_peered":"2024-03-07T07:40:09.359380+0000","last_clean":"2024-03-07T07:40:09.359380+0000","last_became_active":"2024-03-07T07:39:14.615587+0000","last_became_peered":"2024-03-07T07:39:14.615587+0000","last_unstale":"2024-03-07T07:40:09.359380+0000","last_undegraded":"2024-03-07T07:40:09.359380+0000","last_fullsized":"2024-03-07T07:40:09.359380+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:08:20.363895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.358970+0000","last_change":"2024-03-07T07:39:13.832143+0000","last_active":"2024-03-07T07:40:09.358970+0000","last_peered":"2024-03-07T07:40:09.358970+0000","last_clean":"2024-03-07T07:40:09.358970+0000","last_became_active":"2024-03-07T07:39:13.831299+0000","last_became_peered":"2024-03-07T07:39:13.831299+0000","last_unstale":"2024-03-07T07:40:09.358970+0000","last_undegraded":"2024-03-07T07:40:09.358970+0000","last_fullsized":"2024-03-07T07:40:09.358970+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:25:08.965498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.361637+0000","last_change":"2024-03-07T07:39:13.827617+0000","last_active":"2024-03-07T07:40:09.361637+0000","last_peered":"2024-03-07T07:40:09.361637+0000","last_clean":"2024-03-07T07:40:09.361637+0000","last_became_active":"2024-03-07T07:39:13.827188+0000","last_became_peered":"2024-03-07T07:39:13.827188+0000","last_unstale":"2024-03-07T07:40:09.361637+0000","last_undegraded":"2024-03-07T07:40:09.361637+0000","last_fullsized":"2024-03-07T07:40:09.361637+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:35:27.634460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.359617+0000","last_change":"2024-03-07T07:39:14.608307+0000","last_active":"2024-03-07T07:40:09.359617+0000","last_peered":"2024-03-07T07:40:09.359617+0000","last_clean":"2024-03-07T07:40:09.359617+0000","last_became_active":"2024-03-07T07:39:14.607948+0000","last_became_peered":"2024-03-07T07:39:14.607948+0000","last_unstale":"2024-03-07T07:40:09.359617+0000","last_undegraded":"2024-03-07T07:40:09.359617+0000","last_fullsized":"2024-03-07T07:40:09.359617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:36:27.294925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.457084+0000","last_change":"2024-03-07T07:39:13.831123+0000","last_active":"2024-03-07T07:40:09.457084+0000","last_peered":"2024-03-07T07:40:09.457084+0000","last_clean":"2024-03-07T07:40:09.457084+0000","last_became_active":"2024-03-07T07:39:13.830801+0000","last_became_peered":"2024-03-07T07:39:13.830801+0000","last_unstale":"2024-03-07T07:40:09.457084+0000","last_undegraded":"2024-03-07T07:40:09.457084+0000","last_fullsized":"2024-03-07T07:40:09.457084+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:19:54.346449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.362610+0000","last_change":"2024-03-07T07:39:14.610510+0000","last_active":"2024-03-07T07:40:09.362610+0000","last_peered":"2024-03-07T07:40:09.362610+0000","last_clean":"2024-03-07T07:40:09.362610+0000","last_became_active":"2024-03-07T07:39:14.609908+0000","last_became_peered":"2024-03-07T07:39:14.609908+0000","last_unstale":"2024-03-07T07:40:09.362610+0000","last_undegraded":"2024-03-07T07:40:09.362610+0000","last_fullsized":"2024-03-07T07:40:09.362610+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:51:32.045296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.357673+0000","last_change":"2024-03-07T07:39:13.838605+0000","last_active":"2024-03-07T07:40:09.357673+0000","last_peered":"2024-03-07T07:40:09.357673+0000","last_clean":"2024-03-07T07:40:09.357673+0000","last_became_active":"2024-03-07T07:39:13.838362+0000","last_became_peered":"2024-03-07T07:39:13.838362+0000","last_unstale":"2024-03-07T07:40:09.357673+0000","last_undegraded":"2024-03-07T07:40:09.357673+0000","last_fullsized":"2024-03-07T07:40:09.357673+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:23:47.039496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.366588+0000","last_change":"2024-03-07T07:40:09.366588+0000","last_active":"2024-03-07T07:40:09.366588+0000","last_peered":"2024-03-07T07:40:09.366588+0000","last_clean":"2024-03-07T07:40:09.366588+0000","last_became_active":"2024-03-07T07:40:09.366133+0000","last_became_peered":"2024-03-07T07:40:09.366133+0000","last_unstale":"2024-03-07T07:40:09.366588+0000","last_undegraded":"2024-03-07T07:40:09.366588+0000","last_fullsized":"2024-03-07T07:40:09.366588+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:48:09.932164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.362890+0000","last_change":"2024-03-07T07:39:13.834640+0000","last_active":"2024-03-07T07:40:09.362890+0000","last_peered":"2024-03-07T07:40:09.362890+0000","last_clean":"2024-03-07T07:40:09.362890+0000","last_became_active":"2024-03-07T07:39:13.834247+0000","last_became_peered":"2024-03-07T07:39:13.834247+0000","last_unstale":"2024-03-07T07:40:09.362890+0000","last_undegraded":"2024-03-07T07:40:09.362890+0000","last_fullsized":"2024-03-07T07:40:09.362890+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:40:55.487824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.360713+0000","last_change":"2024-03-07T07:39:14.603762+0000","last_active":"2024-03-07T07:40:09.360713+0000","last_peered":"2024-03-07T07:40:09.360713+0000","last_clean":"2024-03-07T07:40:09.360713+0000","last_became_active":"2024-03-07T07:39:14.603444+0000","last_became_peered":"2024-03-07T07:39:14.603444+0000","last_unstale":"2024-03-07T07:40:09.360713+0000","last_undegraded":"2024-03-07T07:40:09.360713+0000","last_fullsized":"2024-03-07T07:40:09.360713+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:18:03.075996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.361831+0000","last_change":"2024-03-07T07:39:13.833864+0000","last_active":"2024-03-07T07:40:09.361831+0000","last_peered":"2024-03-07T07:40:09.361831+0000","last_clean":"2024-03-07T07:40:09.361831+0000","last_became_active":"2024-03-07T07:39:13.833570+0000","last_became_peered":"2024-03-07T07:39:13.833570+0000","last_unstale":"2024-03-07T07:40:09.361831+0000","last_undegraded":"2024-03-07T07:40:09.361831+0000","last_fullsized":"2024-03-07T07:40:09.361831+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:20:16.387319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.359020+0000","last_change":"2024-03-07T07:39:14.610023+0000","last_active":"2024-03-07T07:40:09.359020+0000","last_peered":"2024-03-07T07:40:09.359020+0000","last_clean":"2024-03-07T07:40:09.359020+0000","last_became_active":"2024-03-07T07:39:14.609671+0000","last_became_peered":"2024-03-07T07:39:14.609671+0000","last_unstale":"2024-03-07T07:40:09.359020+0000","last_undegraded":"2024-03-07T07:40:09.359020+0000","last_fullsized":"2024-03-07T07:40:09.359020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:41:48.384795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.361027+0000","last_change":"2024-03-07T07:39:13.841326+0000","last_active":"2024-03-07T07:40:09.361027+0000","last_peered":"2024-03-07T07:40:09.361027+0000","last_clean":"2024-03-07T07:40:09.361027+0000","last_became_active":"2024-03-07T07:39:13.840955+0000","last_became_peered":"2024-03-07T07:39:13.840955+0000","last_unstale":"2024-03-07T07:40:09.361027+0000","last_undegraded":"2024-03-07T07:40:09.361027+0000","last_fullsized":"2024-03-07T07:40:09.361027+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:56:25.944653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.362359+0000","last_change":"2024-03-07T07:39:14.617783+0000","last_active":"2024-03-07T07:40:09.362359+0000","last_peered":"2024-03-07T07:40:09.362359+0000","last_clean":"2024-03-07T07:40:09.362359+0000","last_became_active":"2024-03-07T07:39:14.617472+0000","last_became_peered":"2024-03-07T07:39:14.617472+0000","last_unstale":"2024-03-07T07:40:09.362359+0000","last_undegraded":"2024-03-07T07:40:09.362359+0000","last_fullsized":"2024-03-07T07:40:09.362359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:35:48.618321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.361690+0000","last_change":"2024-03-07T07:39:13.837602+0000","last_active":"2024-03-07T07:40:09.361690+0000","last_peered":"2024-03-07T07:40:09.361690+0000","last_clean":"2024-03-07T07:40:09.361690+0000","last_became_active":"2024-03-07T07:39:13.837275+0000","last_became_peered":"2024-03-07T07:39:13.837275+0000","last_unstale":"2024-03-07T07:40:09.361690+0000","last_undegraded":"2024-03-07T07:40:09.361690+0000","last_fullsized":"2024-03-07T07:40:09.361690+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:40:43.267842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.359069+0000","last_change":"2024-03-07T07:39:14.616702+0000","last_active":"2024-03-07T07:40:09.359069+0000","last_peered":"2024-03-07T07:40:09.359069+0000","last_clean":"2024-03-07T07:40:09.359069+0000","last_became_active":"2024-03-07T07:39:14.614720+0000","last_became_peered":"2024-03-07T07:39:14.614720+0000","last_unstale":"2024-03-07T07:40:09.359069+0000","last_undegraded":"2024-03-07T07:40:09.359069+0000","last_fullsized":"2024-03-07T07:40:09.359069+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:14:45.648231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.361725+0000","last_change":"2024-03-07T07:39:13.836641+0000","last_active":"2024-03-07T07:40:09.361725+0000","last_peered":"2024-03-07T07:40:09.361725+0000","last_clean":"2024-03-07T07:40:09.361725+0000","last_became_active":"2024-03-07T07:39:13.834012+0000","last_became_peered":"2024-03-07T07:39:13.834012+0000","last_unstale":"2024-03-07T07:40:09.361725+0000","last_undegraded":"2024-03-07T07:40:09.361725+0000","last_fullsized":"2024-03-07T07:40:09.361725+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:40:49.596816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.359265+0000","last_change":"2024-03-07T07:39:14.611074+0000","last_active":"2024-03-07T07:40:09.359265+0000","last_peered":"2024-03-07T07:40:09.359265+0000","last_clean":"2024-03-07T07:40:09.359265+0000","last_became_active":"2024-03-07T07:39:14.610651+0000","last_became_peered":"2024-03-07T07:39:14.610651+0000","last_unstale":"2024-03-07T07:40:09.359265+0000","last_undegraded":"2024-03-07T07:40:09.359265+0000","last_fullsized":"2024-03-07T07:40:09.359265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:18:29.684266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.367142+0000","last_change":"2024-03-07T07:39:13.839743+0000","last_active":"2024-03-07T07:40:09.367142+0000","last_peered":"2024-03-07T07:40:09.367142+0000","last_clean":"2024-03-07T07:40:09.367142+0000","last_became_active":"2024-03-07T07:39:13.839079+0000","last_became_peered":"2024-03-07T07:39:13.839079+0000","last_unstale":"2024-03-07T07:40:09.367142+0000","last_undegraded":"2024-03-07T07:40:09.367142+0000","last_fullsized":"2024-03-07T07:40:09.367142+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:11:00.757666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.359327+0000","last_change":"2024-03-07T07:39:14.615941+0000","last_active":"2024-03-07T07:40:09.359327+0000","last_peered":"2024-03-07T07:40:09.359327+0000","last_clean":"2024-03-07T07:40:09.359327+0000","last_became_active":"2024-03-07T07:39:14.615577+0000","last_became_peered":"2024-03-07T07:39:14.615577+0000","last_unstale":"2024-03-07T07:40:09.359327+0000","last_undegraded":"2024-03-07T07:40:09.359327+0000","last_fullsized":"2024-03-07T07:40:09.359327+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:06:05.457297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.371904+0000","last_change":"2024-03-07T07:40:09.371904+0000","last_active":"2024-03-07T07:40:09.371904+0000","last_peered":"2024-03-07T07:40:09.371904+0000","last_clean":"2024-03-07T07:40:09.371904+0000","last_became_active":"2024-03-07T07:40:09.366168+0000","last_became_peered":"2024-03-07T07:40:09.366168+0000","last_unstale":"2024-03-07T07:40:09.371904+0000","last_undegraded":"2024-03-07T07:40:09.371904+0000","last_fullsized":"2024-03-07T07:40:09.371904+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:08:59.697518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.359421+0000","last_change":"2024-03-07T07:39:14.611652+0000","last_active":"2024-03-07T07:40:09.359421+0000","last_peered":"2024-03-07T07:40:09.359421+0000","last_clean":"2024-03-07T07:40:09.359421+0000","last_became_active":"2024-03-07T07:39:14.611300+0000","last_became_peered":"2024-03-07T07:39:14.611300+0000","last_unstale":"2024-03-07T07:40:09.359421+0000","last_undegraded":"2024-03-07T07:40:09.359421+0000","last_fullsized":"2024-03-07T07:40:09.359421+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:30:44.240560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.359064+0000","last_change":"2024-03-07T07:39:14.617630+0000","last_active":"2024-03-07T07:40:09.359064+0000","last_peered":"2024-03-07T07:40:09.359064+0000","last_clean":"2024-03-07T07:40:09.359064+0000","last_became_active":"2024-03-07T07:39:14.616897+0000","last_became_peered":"2024-03-07T07:39:14.616897+0000","last_unstale":"2024-03-07T07:40:09.359064+0000","last_undegraded":"2024-03-07T07:40:09.359064+0000","last_fullsized":"2024-03-07T07:40:09.359064+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T19:24:35.522835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.361102+0000","last_change":"2024-03-07T07:39:14.616670+0000","last_active":"2024-03-07T07:40:09.361102+0000","last_peered":"2024-03-07T07:40:09.361102+0000","last_clean":"2024-03-07T07:40:09.361102+0000","last_became_active":"2024-03-07T07:39:14.615720+0000","last_became_peered":"2024-03-07T07:39:14.615720+0000","last_unstale":"2024-03-07T07:40:09.361102+0000","last_undegraded":"2024-03-07T07:40:09.361102+0000","last_fullsized":"2024-03-07T07:40:09.361102+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:51:59.533172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.361129+0000","last_change":"2024-03-07T07:39:14.606794+0000","last_active":"2024-03-07T07:40:09.361129+0000","last_peered":"2024-03-07T07:40:09.361129+0000","last_clean":"2024-03-07T07:40:09.361129+0000","last_became_active":"2024-03-07T07:39:14.606395+0000","last_became_peered":"2024-03-07T07:39:14.606395+0000","last_unstale":"2024-03-07T07:40:09.361129+0000","last_undegraded":"2024-03-07T07:40:09.361129+0000","last_fullsized":"2024-03-07T07:40:09.361129+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:54:26.263361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.356940+0000","last_change":"2024-03-07T07:39:14.616627+0000","last_active":"2024-03-07T07:40:09.356940+0000","last_peered":"2024-03-07T07:40:09.356940+0000","last_clean":"2024-03-07T07:40:09.356940+0000","last_became_active":"2024-03-07T07:39:14.615865+0000","last_became_peered":"2024-03-07T07:39:14.615865+0000","last_unstale":"2024-03-07T07:40:09.356940+0000","last_undegraded":"2024-03-07T07:40:09.356940+0000","last_fullsized":"2024-03-07T07:40:09.356940+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:15:46.997786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.359611+0000","last_change":"2024-03-07T07:39:14.616493+0000","last_active":"2024-03-07T07:40:09.359611+0000","last_peered":"2024-03-07T07:40:09.359611+0000","last_clean":"2024-03-07T07:40:09.359611+0000","last_became_active":"2024-03-07T07:39:14.616185+0000","last_became_peered":"2024-03-07T07:39:14.616185+0000","last_unstale":"2024-03-07T07:40:09.359611+0000","last_undegraded":"2024-03-07T07:40:09.359611+0000","last_fullsized":"2024-03-07T07:40:09.359611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:51:57.888472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.358622+0000","last_change":"2024-03-07T07:39:14.614720+0000","last_active":"2024-03-07T07:40:09.358622+0000","last_peered":"2024-03-07T07:40:09.358622+0000","last_clean":"2024-03-07T07:40:09.358622+0000","last_became_active":"2024-03-07T07:39:14.614256+0000","last_became_peered":"2024-03-07T07:39:14.614256+0000","last_unstale":"2024-03-07T07:40:09.358622+0000","last_undegraded":"2024-03-07T07:40:09.358622+0000","last_fullsized":"2024-03-07T07:40:09.358622+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:58:24.497664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.363002+0000","last_change":"2024-03-07T07:39:14.611152+0000","last_active":"2024-03-07T07:40:09.363002+0000","last_peered":"2024-03-07T07:40:09.363002+0000","last_clean":"2024-03-07T07:40:09.363002+0000","last_became_active":"2024-03-07T07:39:14.610616+0000","last_became_peered":"2024-03-07T07:39:14.610616+0000","last_unstale":"2024-03-07T07:40:09.363002+0000","last_undegraded":"2024-03-07T07:40:09.363002+0000","last_fullsized":"2024-03-07T07:40:09.363002+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:59:36.283163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.362611+0000","last_change":"2024-03-07T07:39:14.610762+0000","last_active":"2024-03-07T07:40:09.362611+0000","last_peered":"2024-03-07T07:40:09.362611+0000","last_clean":"2024-03-07T07:40:09.362611+0000","last_became_active":"2024-03-07T07:39:14.610440+0000","last_became_peered":"2024-03-07T07:39:14.610440+0000","last_unstale":"2024-03-07T07:40:09.362611+0000","last_undegraded":"2024-03-07T07:40:09.362611+0000","last_fullsized":"2024-03-07T07:40:09.362611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:45:18.046694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.366823+0000","last_change":"2024-03-07T07:39:14.610322+0000","last_active":"2024-03-07T07:40:09.366823+0000","last_peered":"2024-03-07T07:40:09.366823+0000","last_clean":"2024-03-07T07:40:09.366823+0000","last_became_active":"2024-03-07T07:39:14.610008+0000","last_became_peered":"2024-03-07T07:39:14.610008+0000","last_unstale":"2024-03-07T07:40:09.366823+0000","last_undegraded":"2024-03-07T07:40:09.366823+0000","last_fullsized":"2024-03-07T07:40:09.366823+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:35:46.071475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.361793+0000","last_change":"2024-03-07T07:39:14.610084+0000","last_active":"2024-03-07T07:40:09.361793+0000","last_peered":"2024-03-07T07:40:09.361793+0000","last_clean":"2024-03-07T07:40:09.361793+0000","last_became_active":"2024-03-07T07:39:14.609085+0000","last_became_peered":"2024-03-07T07:39:14.609085+0000","last_unstale":"2024-03-07T07:40:09.361793+0000","last_undegraded":"2024-03-07T07:40:09.361793+0000","last_fullsized":"2024-03-07T07:40:09.361793+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:24:31.206928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.363095+0000","last_change":"2024-03-07T07:39:14.611486+0000","last_active":"2024-03-07T07:40:09.363095+0000","last_peered":"2024-03-07T07:40:09.363095+0000","last_clean":"2024-03-07T07:40:09.363095+0000","last_became_active":"2024-03-07T07:39:14.611128+0000","last_became_peered":"2024-03-07T07:39:14.611128+0000","last_unstale":"2024-03-07T07:40:09.363095+0000","last_undegraded":"2024-03-07T07:40:09.363095+0000","last_fullsized":"2024-03-07T07:40:09.363095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:48:07.831530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.359722+0000","last_change":"2024-03-07T07:39:14.604771+0000","last_active":"2024-03-07T07:40:09.359722+0000","last_peered":"2024-03-07T07:40:09.359722+0000","last_clean":"2024-03-07T07:40:09.359722+0000","last_became_active":"2024-03-07T07:39:14.604494+0000","last_became_peered":"2024-03-07T07:39:14.604494+0000","last_unstale":"2024-03-07T07:40:09.359722+0000","last_undegraded":"2024-03-07T07:40:09.359722+0000","last_fullsized":"2024-03-07T07:40:09.359722+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:42:45.141833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.358435+0000","last_change":"2024-03-07T07:39:14.618689+0000","last_active":"2024-03-07T07:40:09.358435+0000","last_peered":"2024-03-07T07:40:09.358435+0000","last_clean":"2024-03-07T07:40:09.358435+0000","last_became_active":"2024-03-07T07:39:14.618330+0000","last_became_peered":"2024-03-07T07:39:14.618330+0000","last_unstale":"2024-03-07T07:40:09.358435+0000","last_undegraded":"2024-03-07T07:40:09.358435+0000","last_fullsized":"2024-03-07T07:40:09.358435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:23:07.431656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282597,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62913,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":436,"num_read_kb":283,"num_write":154,"num_write_kb":105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":59024,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":364544,"data_stored":188739,"data_compressed":13305,"data_compressed_allocated":20480,"data_compressed_original":73728,"omap_allocated":0,"internal_metadata":0},"log_size":133,"ondisk_log_size":133,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383709,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":950272,"data_stored":456388,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645346,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1089536,"data_stored":1107060,"data_compressed":43594,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808871,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":962560,"data_stored":456629,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070508,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":456320,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332145,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1007616,"data_stored":1048362,"data_compressed":41719,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495670,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":946176,"data_stored":455901,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":514606,"data_compressed":7654,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":457172,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411402,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":962560,"data_stored":457044,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182470,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":456095,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920833,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":974848,"data_stored":515641,"data_compressed":15334,"data_compressed_allocated":20480,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757308,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":1047655,"data_compressed":41719,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34299,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34299,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34299,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35940,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35940,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35940,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35940,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35940,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":58912,"data_compressed":1875,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":59947,"data_compressed":9555,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":60881,"data_compressed":1875,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-07T07:40:31.172 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph pg dump --format=json 2024-03-07T07:40:31.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:31 smithi160 ceph-mon[31963]: pgmap v328: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-07T07:40:31.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:31 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:40:31.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:31 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:40:31.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:31 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:40:31.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:31 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:40:31.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:31 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:40:31.428 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:40:31.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:31 smithi178 ceph-mon[36650]: pgmap v328: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-07T07:40:31.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:31 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:40:31.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:31 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:40:31.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:31 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:40:31.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:31 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:40:31.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:31 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:40:31.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:31 smithi177 ceph-mon[36578]: pgmap v328: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-07T07:40:31.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:31 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:40:31.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:31 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:40:31.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:31 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:40:31.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:31 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:40:31.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:31 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:40:32.532 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:32 smithi160 ceph-mon[31963]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-07T07:40:32.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:32 smithi178 ceph-mon[36650]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-07T07:40:32.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:32 smithi177 ceph-mon[36578]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-07T07:40:33.068 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.068+0000 7f5b1873f700 1 -- 172.21.15.160:0/3750016011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b10101910 msgr2=0x7f5b1010e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:33.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.068+0000 7f5b1873f700 1 --2- 172.21.15.160:0/3750016011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b10101910 0x7f5b1010e660 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5b00009990 tx=0x7f5b00009ca0 comp rx=0 tx=0).stop 2024-03-07T07:40:33.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.068+0000 7f5b1873f700 1 -- 172.21.15.160:0/3750016011 shutdown_connections 2024-03-07T07:40:33.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.068+0000 7f5b1873f700 1 --2- 172.21.15.160:0/3750016011 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b1010eba0 0x7f5b10110f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:33.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.068+0000 7f5b1873f700 1 --2- 172.21.15.160:0/3750016011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b10100ff0 0x7f5b101013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:33.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.068+0000 7f5b1873f700 1 --2- 172.21.15.160:0/3750016011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b10101910 0x7f5b1010e660 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:33.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.068+0000 7f5b1873f700 1 -- 172.21.15.160:0/3750016011 >> 172.21.15.160:0/3750016011 conn(0x7f5b10078180 msgr2=0x7f5b10078590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:33.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.068+0000 7f5b1873f700 1 -- 172.21.15.160:0/3750016011 shutdown_connections 2024-03-07T07:40:33.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.068+0000 7f5b1873f700 1 -- 172.21.15.160:0/3750016011 wait complete. 2024-03-07T07:40:33.071 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.069+0000 7f5b1873f700 1 Processor -- start 2024-03-07T07:40:33.071 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.069+0000 7f5b1873f700 1 -- start start 2024-03-07T07:40:33.071 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.069+0000 7f5b1873f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b10100ff0 0x7f5b101a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:33.071 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.069+0000 7f5b1873f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b10101910 0x7f5b101a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:33.071 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.069+0000 7f5b1873f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b1010eba0 0x7f5b1019aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:33.071 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.069+0000 7f5b1873f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b10114750 con 0x7f5b10100ff0 2024-03-07T07:40:33.071 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.069+0000 7f5b1873f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b101145d0 con 0x7f5b10101910 2024-03-07T07:40:33.071 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.069+0000 7f5b1873f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b101148d0 con 0x7f5b1010eba0 2024-03-07T07:40:33.071 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.069+0000 7f5b164db700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b10100ff0 0x7f5b101a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:33.072 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.069+0000 7f5b164db700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b10100ff0 0x7f5b101a0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:38290/0 (socket says 172.21.15.160:38290) 2024-03-07T07:40:33.072 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.069+0000 7f5b164db700 1 -- 172.21.15.160:0/3687181377 learned_addr learned my addr 172.21.15.160:0/3687181377 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:40:33.072 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.069+0000 7f5b16cdc700 1 --2- 172.21.15.160:0/3687181377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b1010eba0 0x7f5b1019aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:33.072 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.069+0000 7f5b15cda700 1 --2- 172.21.15.160:0/3687181377 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b10101910 0x7f5b101a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:33.072 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.069+0000 7f5b164db700 1 -- 172.21.15.160:0/3687181377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b1010eba0 msgr2=0x7f5b1019aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:33.072 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.069+0000 7f5b164db700 1 --2- 172.21.15.160:0/3687181377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b1010eba0 0x7f5b1019aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:33.072 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.069+0000 7f5b164db700 1 -- 172.21.15.160:0/3687181377 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b10101910 msgr2=0x7f5b101a0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:33.072 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.069+0000 7f5b164db700 1 --2- 172.21.15.160:0/3687181377 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b10101910 0x7f5b101a0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:33.072 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.069+0000 7f5b164db700 1 -- 172.21.15.160:0/3687181377 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b1019b1b0 con 0x7f5b10100ff0 2024-03-07T07:40:33.072 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.070+0000 7f5b164db700 1 --2- 172.21.15.160:0/3687181377 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b10100ff0 0x7f5b101a0890 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c00e9c0 tx=0x7f5b0c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:33.072 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.070+0000 7f5b077fe700 1 -- 172.21.15.160:0/3687181377 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c00cbd0 con 0x7f5b10100ff0 2024-03-07T07:40:33.072 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.070+0000 7f5b1873f700 1 -- 172.21.15.160:0/3687181377 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b1019b4a0 con 0x7f5b10100ff0 2024-03-07T07:40:33.072 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.070+0000 7f5b077fe700 1 -- 172.21.15.160:0/3687181377 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5b0c00ee60 con 0x7f5b10100ff0 2024-03-07T07:40:33.073 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.070+0000 7f5b077fe700 1 -- 172.21.15.160:0/3687181377 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c005630 con 0x7f5b10100ff0 2024-03-07T07:40:33.073 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.070+0000 7f5b1873f700 1 -- 172.21.15.160:0/3687181377 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b101a7760 con 0x7f5b10100ff0 2024-03-07T07:40:33.073 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.071+0000 7f5b1873f700 1 -- 172.21.15.160:0/3687181377 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b10068e10 con 0x7f5b10100ff0 2024-03-07T07:40:33.073 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.072+0000 7f5b077fe700 1 -- 172.21.15.160:0/3687181377 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) v1 ==== 136805+0+0 (secure 0 0 0) 0x7f5b0c004750 con 0x7f5b10100ff0 2024-03-07T07:40:33.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.072+0000 7f5b077fe700 1 --2- 172.21.15.160:0/3687181377 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f5afc0a1ea0 0x7f5afc0a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:33.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.072+0000 7f5b15cda700 1 --2- 172.21.15.160:0/3687181377 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f5afc0a1ea0 0x7f5afc0a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:33.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.072+0000 7f5b077fe700 1 -- 172.21.15.160:0/3687181377 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5b0c0ce1d0 con 0x7f5b10100ff0 2024-03-07T07:40:33.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.073+0000 7f5b15cda700 1 --2- 172.21.15.160:0/3687181377 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f5afc0a1ea0 0x7f5afc0a4360 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5b000098e0 tx=0x7f5b00005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:33.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.077+0000 7f5b077fe700 1 -- 172.21.15.160:0/3687181377 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5b0c09b4a0 con 0x7f5b10100ff0 2024-03-07T07:40:33.270 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.269+0000 7f5b1873f700 1 -- 172.21.15.160:0/3687181377 --> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5b1019bf80 con 0x7f5afc0a1ea0 2024-03-07T07:40:33.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.284+0000 7f5b077fe700 1 -- 172.21.15.160:0/3687181377 <== mgr.14152 v2:172.21.15.160:6800/419630619 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+259032 (secure 0 0 0) 0x7f5b1019bf80 con 0x7f5afc0a1ea0 2024-03-07T07:40:33.288 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:40:33.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.287+0000 7f5b1873f700 1 -- 172.21.15.160:0/3687181377 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f5afc0a1ea0 msgr2=0x7f5afc0a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:33.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.287+0000 7f5b1873f700 1 --2- 172.21.15.160:0/3687181377 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f5afc0a1ea0 0x7f5afc0a4360 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5b000098e0 tx=0x7f5b00005960 comp rx=0 tx=0).stop 2024-03-07T07:40:33.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.287+0000 7f5b1873f700 1 -- 172.21.15.160:0/3687181377 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b10100ff0 msgr2=0x7f5b101a0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:33.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.287+0000 7f5b1873f700 1 --2- 172.21.15.160:0/3687181377 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b10100ff0 0x7f5b101a0890 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c00e9c0 tx=0x7f5b0c00ecd0 comp rx=0 tx=0).stop 2024-03-07T07:40:33.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.287+0000 7f5b1873f700 1 -- 172.21.15.160:0/3687181377 shutdown_connections 2024-03-07T07:40:33.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.287+0000 7f5b1873f700 1 --2- 172.21.15.160:0/3687181377 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f5afc0a1ea0 0x7f5afc0a4360 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:33.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.287+0000 7f5b1873f700 1 --2- 172.21.15.160:0/3687181377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b1010eba0 0x7f5b1019aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:33.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.287+0000 7f5b1873f700 1 --2- 172.21.15.160:0/3687181377 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b10100ff0 0x7f5b101a0890 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:33.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.287+0000 7f5b1873f700 1 --2- 172.21.15.160:0/3687181377 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b10101910 0x7f5b101a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:33.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.287+0000 7f5b1873f700 1 -- 172.21.15.160:0/3687181377 >> 172.21.15.160:0/3687181377 conn(0x7f5b10078180 msgr2=0x7f5b10110090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:33.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.287+0000 7f5b1873f700 1 -- 172.21.15.160:0/3687181377 shutdown_connections 2024-03-07T07:40:33.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:33.287+0000 7f5b1873f700 1 -- 172.21.15.160:0/3687181377 wait complete. 2024-03-07T07:40:33.290 INFO:teuthology.orchestra.run.smithi160.stderr:dumped all 2024-03-07T07:40:33.465 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:33 smithi160 ceph-mon[31963]: pgmap v329: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-07T07:40:33.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:33 smithi178 ceph-mon[36650]: pgmap v329: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-07T07:40:33.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:33 smithi177 ceph-mon[36578]: pgmap v329: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-07T07:40:34.101 INFO:teuthology.orchestra.run.smithi160.stdout:{"pg_ready":true,"pg_map":{"version":329,"stamp":"2024-03-07T07:40:32.290057+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653515,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":562,"num_read_kb":392,"num_write":337,"num_write_kb":3045,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1239877,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":272,"ondisk_log_size":272,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360756,"kb_used_data":11516,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581068,"statfs":{"total":1151940427776,"available":1151571013632,"internally_reserved":0,"allocated":11792384,"data_stored":7428873,"data_compressed":190473,"data_compressed_allocated":253952,"data_compressed_original":2039808,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002983"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.358756+0000","last_change":"2024-03-07T07:39:14.614168+0000","last_active":"2024-03-07T07:40:09.358756+0000","last_peered":"2024-03-07T07:40:09.358756+0000","last_clean":"2024-03-07T07:40:09.358756+0000","last_became_active":"2024-03-07T07:39:14.613708+0000","last_became_peered":"2024-03-07T07:39:14.613708+0000","last_unstale":"2024-03-07T07:40:09.358756+0000","last_undegraded":"2024-03-07T07:40:09.358756+0000","last_fullsized":"2024-03-07T07:40:09.358756+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:08:07.286470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.357141+0000","last_change":"2024-03-07T07:39:14.609425+0000","last_active":"2024-03-07T07:40:09.357141+0000","last_peered":"2024-03-07T07:40:09.357141+0000","last_clean":"2024-03-07T07:40:09.357141+0000","last_became_active":"2024-03-07T07:39:14.609060+0000","last_became_peered":"2024-03-07T07:39:14.609060+0000","last_unstale":"2024-03-07T07:40:09.357141+0000","last_undegraded":"2024-03-07T07:40:09.357141+0000","last_fullsized":"2024-03-07T07:40:09.357141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:37:33.773358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.357094+0000","last_change":"2024-03-07T07:39:14.609985+0000","last_active":"2024-03-07T07:40:09.357094+0000","last_peered":"2024-03-07T07:40:09.357094+0000","last_clean":"2024-03-07T07:40:09.357094+0000","last_became_active":"2024-03-07T07:39:14.609490+0000","last_became_peered":"2024-03-07T07:39:14.609490+0000","last_unstale":"2024-03-07T07:40:09.357094+0000","last_undegraded":"2024-03-07T07:40:09.357094+0000","last_fullsized":"2024-03-07T07:40:09.357094+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T19:09:24.134718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.357909+0000","last_change":"2024-03-07T07:39:14.620482+0000","last_active":"2024-03-07T07:40:09.357909+0000","last_peered":"2024-03-07T07:40:09.357909+0000","last_clean":"2024-03-07T07:40:09.357909+0000","last_became_active":"2024-03-07T07:39:14.620100+0000","last_became_peered":"2024-03-07T07:39:14.620100+0000","last_unstale":"2024-03-07T07:40:09.357909+0000","last_undegraded":"2024-03-07T07:40:09.357909+0000","last_fullsized":"2024-03-07T07:40:09.357909+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:03:20.291833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.361639+0000","last_change":"2024-03-07T07:39:14.603228+0000","last_active":"2024-03-07T07:40:09.361639+0000","last_peered":"2024-03-07T07:40:09.361639+0000","last_clean":"2024-03-07T07:40:09.361639+0000","last_became_active":"2024-03-07T07:39:14.602874+0000","last_became_peered":"2024-03-07T07:39:14.602874+0000","last_unstale":"2024-03-07T07:40:09.361639+0000","last_undegraded":"2024-03-07T07:40:09.361639+0000","last_fullsized":"2024-03-07T07:40:09.361639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:26:17.762271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.456982+0000","last_change":"2024-03-07T07:40:09.456982+0000","last_active":"2024-03-07T07:40:09.456982+0000","last_peered":"2024-03-07T07:40:09.456982+0000","last_clean":"2024-03-07T07:40:09.456982+0000","last_became_active":"2024-03-07T07:40:09.456688+0000","last_became_peered":"2024-03-07T07:40:09.456688+0000","last_unstale":"2024-03-07T07:40:09.456982+0000","last_undegraded":"2024-03-07T07:40:09.456982+0000","last_fullsized":"2024-03-07T07:40:09.456982+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:00:34.723624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.359257+0000","last_change":"2024-03-07T07:39:14.615259+0000","last_active":"2024-03-07T07:40:09.359257+0000","last_peered":"2024-03-07T07:40:09.359257+0000","last_clean":"2024-03-07T07:40:09.359257+0000","last_became_active":"2024-03-07T07:39:14.614878+0000","last_became_peered":"2024-03-07T07:39:14.614878+0000","last_unstale":"2024-03-07T07:40:09.359257+0000","last_undegraded":"2024-03-07T07:40:09.359257+0000","last_fullsized":"2024-03-07T07:40:09.359257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:22:28.905360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.361729+0000","last_change":"2024-03-07T07:39:14.614518+0000","last_active":"2024-03-07T07:40:09.361729+0000","last_peered":"2024-03-07T07:40:09.361729+0000","last_clean":"2024-03-07T07:40:09.361729+0000","last_became_active":"2024-03-07T07:39:14.614308+0000","last_became_peered":"2024-03-07T07:39:14.614308+0000","last_unstale":"2024-03-07T07:40:09.361729+0000","last_undegraded":"2024-03-07T07:40:09.361729+0000","last_fullsized":"2024-03-07T07:40:09.361729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:54:54.908066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.363466+0000","last_change":"2024-03-07T07:39:14.615360+0000","last_active":"2024-03-07T07:40:09.363466+0000","last_peered":"2024-03-07T07:40:09.363466+0000","last_clean":"2024-03-07T07:40:09.363466+0000","last_became_active":"2024-03-07T07:39:14.615036+0000","last_became_peered":"2024-03-07T07:39:14.615036+0000","last_unstale":"2024-03-07T07:40:09.363466+0000","last_undegraded":"2024-03-07T07:40:09.363466+0000","last_fullsized":"2024-03-07T07:40:09.363466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T19:17:07.160773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.457981+0000","last_change":"2024-03-07T07:40:09.457981+0000","last_active":"2024-03-07T07:40:09.457981+0000","last_peered":"2024-03-07T07:40:09.457981+0000","last_clean":"2024-03-07T07:40:09.457981+0000","last_became_active":"2024-03-07T07:40:09.457624+0000","last_became_peered":"2024-03-07T07:40:09.457624+0000","last_unstale":"2024-03-07T07:40:09.457981+0000","last_undegraded":"2024-03-07T07:40:09.457981+0000","last_fullsized":"2024-03-07T07:40:09.457981+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:29:40.916809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.359183+0000","last_change":"2024-03-07T07:39:14.611800+0000","last_active":"2024-03-07T07:40:09.359183+0000","last_peered":"2024-03-07T07:40:09.359183+0000","last_clean":"2024-03-07T07:40:09.359183+0000","last_became_active":"2024-03-07T07:39:14.611482+0000","last_became_peered":"2024-03-07T07:39:14.611482+0000","last_unstale":"2024-03-07T07:40:09.359183+0000","last_undegraded":"2024-03-07T07:40:09.359183+0000","last_fullsized":"2024-03-07T07:40:09.359183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:26:36.361716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.357853+0000","last_change":"2024-03-07T07:39:14.621686+0000","last_active":"2024-03-07T07:40:09.357853+0000","last_peered":"2024-03-07T07:40:09.357853+0000","last_clean":"2024-03-07T07:40:09.357853+0000","last_became_active":"2024-03-07T07:39:14.620815+0000","last_became_peered":"2024-03-07T07:39:14.620815+0000","last_unstale":"2024-03-07T07:40:09.357853+0000","last_undegraded":"2024-03-07T07:40:09.357853+0000","last_fullsized":"2024-03-07T07:40:09.357853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:33:59.972215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.359614+0000","last_change":"2024-03-07T07:39:14.606045+0000","last_active":"2024-03-07T07:40:09.359614+0000","last_peered":"2024-03-07T07:40:09.359614+0000","last_clean":"2024-03-07T07:40:09.359614+0000","last_became_active":"2024-03-07T07:39:14.605726+0000","last_became_peered":"2024-03-07T07:39:14.605726+0000","last_unstale":"2024-03-07T07:40:09.359614+0000","last_undegraded":"2024-03-07T07:40:09.359614+0000","last_fullsized":"2024-03-07T07:40:09.359614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:13:51.899507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.356909+0000","last_change":"2024-03-07T07:39:14.605435+0000","last_active":"2024-03-07T07:40:09.356909+0000","last_peered":"2024-03-07T07:40:09.356909+0000","last_clean":"2024-03-07T07:40:09.356909+0000","last_became_active":"2024-03-07T07:39:14.605127+0000","last_became_peered":"2024-03-07T07:39:14.605127+0000","last_unstale":"2024-03-07T07:40:09.356909+0000","last_undegraded":"2024-03-07T07:40:09.356909+0000","last_fullsized":"2024-03-07T07:40:09.356909+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:15:58.206921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.356800+0000","last_change":"2024-03-07T07:39:14.617166+0000","last_active":"2024-03-07T07:40:09.356800+0000","last_peered":"2024-03-07T07:40:09.356800+0000","last_clean":"2024-03-07T07:40:09.356800+0000","last_became_active":"2024-03-07T07:39:14.616615+0000","last_became_peered":"2024-03-07T07:39:14.616615+0000","last_unstale":"2024-03-07T07:40:09.356800+0000","last_undegraded":"2024-03-07T07:40:09.356800+0000","last_fullsized":"2024-03-07T07:40:09.356800+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:25:05.086822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.362757+0000","last_change":"2024-03-07T07:39:14.611990+0000","last_active":"2024-03-07T07:40:09.362757+0000","last_peered":"2024-03-07T07:40:09.362757+0000","last_clean":"2024-03-07T07:40:09.362757+0000","last_became_active":"2024-03-07T07:39:14.610728+0000","last_became_peered":"2024-03-07T07:39:14.610728+0000","last_unstale":"2024-03-07T07:40:09.362757+0000","last_undegraded":"2024-03-07T07:40:09.362757+0000","last_fullsized":"2024-03-07T07:40:09.362757+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:12:54.850583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.358880+0000","last_change":"2024-03-07T07:39:14.615409+0000","last_active":"2024-03-07T07:40:09.358880+0000","last_peered":"2024-03-07T07:40:09.358880+0000","last_clean":"2024-03-07T07:40:09.358880+0000","last_became_active":"2024-03-07T07:39:14.615003+0000","last_became_peered":"2024-03-07T07:39:14.615003+0000","last_unstale":"2024-03-07T07:40:09.358880+0000","last_undegraded":"2024-03-07T07:40:09.358880+0000","last_fullsized":"2024-03-07T07:40:09.358880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:33:56.464718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.457012+0000","last_change":"2024-03-07T07:39:14.610501+0000","last_active":"2024-03-07T07:40:09.457012+0000","last_peered":"2024-03-07T07:40:09.457012+0000","last_clean":"2024-03-07T07:40:09.457012+0000","last_became_active":"2024-03-07T07:39:14.610190+0000","last_became_peered":"2024-03-07T07:39:14.610190+0000","last_unstale":"2024-03-07T07:40:09.457012+0000","last_undegraded":"2024-03-07T07:40:09.457012+0000","last_fullsized":"2024-03-07T07:40:09.457012+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:05:31.360270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.361517+0000","last_change":"2024-03-07T07:39:14.609373+0000","last_active":"2024-03-07T07:40:09.361517+0000","last_peered":"2024-03-07T07:40:09.361517+0000","last_clean":"2024-03-07T07:40:09.361517+0000","last_became_active":"2024-03-07T07:39:14.608174+0000","last_became_peered":"2024-03-07T07:39:14.608174+0000","last_unstale":"2024-03-07T07:40:09.361517+0000","last_undegraded":"2024-03-07T07:40:09.361517+0000","last_fullsized":"2024-03-07T07:40:09.361517+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:25:18.704239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.457099+0000","last_change":"2024-03-07T07:39:13.830499+0000","last_active":"2024-03-07T07:40:09.457099+0000","last_peered":"2024-03-07T07:40:09.457099+0000","last_clean":"2024-03-07T07:40:09.457099+0000","last_became_active":"2024-03-07T07:39:13.830131+0000","last_became_peered":"2024-03-07T07:39:13.830131+0000","last_unstale":"2024-03-07T07:40:09.457099+0000","last_undegraded":"2024-03-07T07:40:09.457099+0000","last_fullsized":"2024-03-07T07:40:09.457099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:48:14.338758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.362537+0000","last_change":"2024-03-07T07:39:14.612715+0000","last_active":"2024-03-07T07:40:09.362537+0000","last_peered":"2024-03-07T07:40:09.362537+0000","last_clean":"2024-03-07T07:40:09.362537+0000","last_became_active":"2024-03-07T07:39:14.612383+0000","last_became_peered":"2024-03-07T07:39:14.612383+0000","last_unstale":"2024-03-07T07:40:09.362537+0000","last_undegraded":"2024-03-07T07:40:09.362537+0000","last_fullsized":"2024-03-07T07:40:09.362537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:17:36.800396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.362682+0000","last_change":"2024-03-07T07:39:13.830901+0000","last_active":"2024-03-07T07:40:09.362682+0000","last_peered":"2024-03-07T07:40:09.362682+0000","last_clean":"2024-03-07T07:40:09.362682+0000","last_became_active":"2024-03-07T07:39:13.830581+0000","last_became_peered":"2024-03-07T07:39:13.830581+0000","last_unstale":"2024-03-07T07:40:09.362682+0000","last_undegraded":"2024-03-07T07:40:09.362682+0000","last_fullsized":"2024-03-07T07:40:09.362682+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:25:15.428381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.357233+0000","last_change":"2024-03-07T07:39:14.610876+0000","last_active":"2024-03-07T07:40:09.357233+0000","last_peered":"2024-03-07T07:40:09.357233+0000","last_clean":"2024-03-07T07:40:09.357233+0000","last_became_active":"2024-03-07T07:39:14.610584+0000","last_became_peered":"2024-03-07T07:39:14.610584+0000","last_unstale":"2024-03-07T07:40:09.357233+0000","last_undegraded":"2024-03-07T07:40:09.357233+0000","last_fullsized":"2024-03-07T07:40:09.357233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:55:11.818462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.377842+0000","last_change":"2024-03-07T07:40:09.377541+0000","last_active":"2024-03-07T07:40:09.377842+0000","last_peered":"2024-03-07T07:40:09.377842+0000","last_clean":"2024-03-07T07:40:09.377842+0000","last_became_active":"2024-03-07T07:40:09.362602+0000","last_became_peered":"2024-03-07T07:40:09.362602+0000","last_unstale":"2024-03-07T07:40:09.377842+0000","last_undegraded":"2024-03-07T07:40:09.377842+0000","last_fullsized":"2024-03-07T07:40:09.377842+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:50:47.570967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.363140+0000","last_change":"2024-03-07T07:39:14.607666+0000","last_active":"2024-03-07T07:40:09.363140+0000","last_peered":"2024-03-07T07:40:09.363140+0000","last_clean":"2024-03-07T07:40:09.363140+0000","last_became_active":"2024-03-07T07:39:14.607339+0000","last_became_peered":"2024-03-07T07:39:14.607339+0000","last_unstale":"2024-03-07T07:40:09.363140+0000","last_undegraded":"2024-03-07T07:40:09.363140+0000","last_fullsized":"2024-03-07T07:40:09.363140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:17:36.349529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.361319+0000","last_change":"2024-03-07T07:39:13.829179+0000","last_active":"2024-03-07T07:40:09.361319+0000","last_peered":"2024-03-07T07:40:09.361319+0000","last_clean":"2024-03-07T07:40:09.361319+0000","last_became_active":"2024-03-07T07:39:13.828162+0000","last_became_peered":"2024-03-07T07:39:13.828162+0000","last_unstale":"2024-03-07T07:40:09.361319+0000","last_undegraded":"2024-03-07T07:40:09.361319+0000","last_fullsized":"2024-03-07T07:40:09.361319+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:38:00.330980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.367311+0000","last_change":"2024-03-07T07:39:14.621496+0000","last_active":"2024-03-07T07:40:09.367311+0000","last_peered":"2024-03-07T07:40:09.367311+0000","last_clean":"2024-03-07T07:40:09.367311+0000","last_became_active":"2024-03-07T07:39:14.621139+0000","last_became_peered":"2024-03-07T07:39:14.621139+0000","last_unstale":"2024-03-07T07:40:09.367311+0000","last_undegraded":"2024-03-07T07:40:09.367311+0000","last_fullsized":"2024-03-07T07:40:09.367311+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:24:45.553536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":97,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:27.941588+0000","last_change":"2024-03-07T07:39:13.834999+0000","last_active":"2024-03-07T07:40:27.941588+0000","last_peered":"2024-03-07T07:40:27.941588+0000","last_clean":"2024-03-07T07:40:27.941588+0000","last_became_active":"2024-03-07T07:39:13.834672+0000","last_became_peered":"2024-03-07T07:39:13.834672+0000","last_unstale":"2024-03-07T07:40:27.941588+0000","last_undegraded":"2024-03-07T07:40:27.941588+0000","last_fullsized":"2024-03-07T07:40:27.941588+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:55:57.944992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.360955+0000","last_change":"2024-03-07T07:39:14.605519+0000","last_active":"2024-03-07T07:40:09.360955+0000","last_peered":"2024-03-07T07:40:09.360955+0000","last_clean":"2024-03-07T07:40:09.360955+0000","last_became_active":"2024-03-07T07:39:14.605185+0000","last_became_peered":"2024-03-07T07:39:14.605185+0000","last_unstale":"2024-03-07T07:40:09.360955+0000","last_undegraded":"2024-03-07T07:40:09.360955+0000","last_fullsized":"2024-03-07T07:40:09.360955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:54:33.622751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.360037+0000","last_change":"2024-03-07T07:39:13.837186+0000","last_active":"2024-03-07T07:40:09.360037+0000","last_peered":"2024-03-07T07:40:09.360037+0000","last_clean":"2024-03-07T07:40:09.360037+0000","last_became_active":"2024-03-07T07:39:13.836807+0000","last_became_peered":"2024-03-07T07:39:13.836807+0000","last_unstale":"2024-03-07T07:40:09.360037+0000","last_undegraded":"2024-03-07T07:40:09.360037+0000","last_fullsized":"2024-03-07T07:40:09.360037+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:32:39.574197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.367151+0000","last_change":"2024-03-07T07:39:14.621503+0000","last_active":"2024-03-07T07:40:09.367151+0000","last_peered":"2024-03-07T07:40:09.367151+0000","last_clean":"2024-03-07T07:40:09.367151+0000","last_became_active":"2024-03-07T07:39:14.621146+0000","last_became_peered":"2024-03-07T07:39:14.621146+0000","last_unstale":"2024-03-07T07:40:09.367151+0000","last_undegraded":"2024-03-07T07:40:09.367151+0000","last_fullsized":"2024-03-07T07:40:09.367151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:50:15.270355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.359502+0000","last_change":"2024-03-07T07:39:13.838146+0000","last_active":"2024-03-07T07:40:09.359502+0000","last_peered":"2024-03-07T07:40:09.359502+0000","last_clean":"2024-03-07T07:40:09.359502+0000","last_became_active":"2024-03-07T07:39:13.837788+0000","last_became_peered":"2024-03-07T07:39:13.837788+0000","last_unstale":"2024-03-07T07:40:09.359502+0000","last_undegraded":"2024-03-07T07:40:09.359502+0000","last_fullsized":"2024-03-07T07:40:09.359502+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:50:52.154146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.361831+0000","last_change":"2024-03-07T07:39:14.605069+0000","last_active":"2024-03-07T07:40:09.361831+0000","last_peered":"2024-03-07T07:40:09.361831+0000","last_clean":"2024-03-07T07:40:09.361831+0000","last_became_active":"2024-03-07T07:39:14.604733+0000","last_became_peered":"2024-03-07T07:39:14.604733+0000","last_unstale":"2024-03-07T07:40:09.361831+0000","last_undegraded":"2024-03-07T07:40:09.361831+0000","last_fullsized":"2024-03-07T07:40:09.361831+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:49:08.312385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.366786+0000","last_change":"2024-03-07T07:39:13.837854+0000","last_active":"2024-03-07T07:40:09.366786+0000","last_peered":"2024-03-07T07:40:09.366786+0000","last_clean":"2024-03-07T07:40:09.366786+0000","last_became_active":"2024-03-07T07:39:13.837428+0000","last_became_peered":"2024-03-07T07:39:13.837428+0000","last_unstale":"2024-03-07T07:40:09.366786+0000","last_undegraded":"2024-03-07T07:40:09.366786+0000","last_fullsized":"2024-03-07T07:40:09.366786+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:35:15.369936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.456624+0000","last_change":"2024-03-07T07:39:14.606299+0000","last_active":"2024-03-07T07:40:09.456624+0000","last_peered":"2024-03-07T07:40:09.456624+0000","last_clean":"2024-03-07T07:40:09.456624+0000","last_became_active":"2024-03-07T07:39:14.605983+0000","last_became_peered":"2024-03-07T07:39:14.605983+0000","last_unstale":"2024-03-07T07:40:09.456624+0000","last_undegraded":"2024-03-07T07:40:09.456624+0000","last_fullsized":"2024-03-07T07:40:09.456624+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:53:16.658862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":93,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:27.941119+0000","last_change":"2024-03-07T07:39:13.840129+0000","last_active":"2024-03-07T07:40:27.941119+0000","last_peered":"2024-03-07T07:40:27.941119+0000","last_clean":"2024-03-07T07:40:27.941119+0000","last_became_active":"2024-03-07T07:39:13.839735+0000","last_became_peered":"2024-03-07T07:39:13.839735+0000","last_unstale":"2024-03-07T07:40:27.941119+0000","last_undegraded":"2024-03-07T07:40:27.941119+0000","last_fullsized":"2024-03-07T07:40:27.941119+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:31:38.711810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":71,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.363334+0000","last_change":"2024-03-07T07:39:14.606449+0000","last_active":"2024-03-07T07:40:09.363334+0000","last_peered":"2024-03-07T07:40:09.363334+0000","last_clean":"2024-03-07T07:40:09.363334+0000","last_became_active":"2024-03-07T07:39:14.606009+0000","last_became_peered":"2024-03-07T07:39:14.606009+0000","last_unstale":"2024-03-07T07:40:09.363334+0000","last_undegraded":"2024-03-07T07:40:09.363334+0000","last_fullsized":"2024-03-07T07:40:09.363334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:55:49.237517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.359363+0000","last_change":"2024-03-07T07:39:13.834097+0000","last_active":"2024-03-07T07:40:09.359363+0000","last_peered":"2024-03-07T07:40:09.359363+0000","last_clean":"2024-03-07T07:40:09.359363+0000","last_became_active":"2024-03-07T07:39:13.833749+0000","last_became_peered":"2024-03-07T07:39:13.833749+0000","last_unstale":"2024-03-07T07:40:09.359363+0000","last_undegraded":"2024-03-07T07:40:09.359363+0000","last_fullsized":"2024-03-07T07:40:09.359363+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:12:22.471596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.357241+0000","last_change":"2024-03-07T07:39:14.621000+0000","last_active":"2024-03-07T07:40:09.357241+0000","last_peered":"2024-03-07T07:40:09.357241+0000","last_clean":"2024-03-07T07:40:09.357241+0000","last_became_active":"2024-03-07T07:39:14.620708+0000","last_became_peered":"2024-03-07T07:39:14.620708+0000","last_unstale":"2024-03-07T07:40:09.357241+0000","last_undegraded":"2024-03-07T07:40:09.357241+0000","last_fullsized":"2024-03-07T07:40:09.357241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:26:48.363406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.367483+0000","last_change":"2024-03-07T07:39:13.837442+0000","last_active":"2024-03-07T07:40:09.367483+0000","last_peered":"2024-03-07T07:40:09.367483+0000","last_clean":"2024-03-07T07:40:09.367483+0000","last_became_active":"2024-03-07T07:39:13.836515+0000","last_became_peered":"2024-03-07T07:39:13.836515+0000","last_unstale":"2024-03-07T07:40:09.367483+0000","last_undegraded":"2024-03-07T07:40:09.367483+0000","last_fullsized":"2024-03-07T07:40:09.367483+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:19:08.312359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.363253+0000","last_change":"2024-03-07T07:39:14.619130+0000","last_active":"2024-03-07T07:40:09.363253+0000","last_peered":"2024-03-07T07:40:09.363253+0000","last_clean":"2024-03-07T07:40:09.363253+0000","last_became_active":"2024-03-07T07:39:14.618814+0000","last_became_peered":"2024-03-07T07:39:14.618814+0000","last_unstale":"2024-03-07T07:40:09.363253+0000","last_undegraded":"2024-03-07T07:40:09.363253+0000","last_fullsized":"2024-03-07T07:40:09.363253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:52:06.323318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'26","reported_seq":212,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:29.387418+0000","last_change":"2024-03-07T07:40:09.370340+0000","last_active":"2024-03-07T07:40:29.387418+0000","last_peered":"2024-03-07T07:40:29.387418+0000","last_clean":"2024-03-07T07:40:29.387418+0000","last_became_active":"2024-03-07T07:40:09.363828+0000","last_became_peered":"2024-03-07T07:40:09.363828+0000","last_unstale":"2024-03-07T07:40:29.387418+0000","last_undegraded":"2024-03-07T07:40:29.387418+0000","last_fullsized":"2024-03-07T07:40:29.387418+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:42:45.203744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58912,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":135,"num_read_kb":124,"num_write":27,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":58912,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.361444+0000","last_change":"2024-03-07T07:39:14.606539+0000","last_active":"2024-03-07T07:40:09.361444+0000","last_peered":"2024-03-07T07:40:09.361444+0000","last_clean":"2024-03-07T07:40:09.361444+0000","last_became_active":"2024-03-07T07:39:14.605832+0000","last_became_peered":"2024-03-07T07:39:14.605832+0000","last_unstale":"2024-03-07T07:40:09.361444+0000","last_undegraded":"2024-03-07T07:40:09.361444+0000","last_fullsized":"2024-03-07T07:40:09.361444+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:08:10.781906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.361006+0000","last_change":"2024-03-07T07:39:13.841526+0000","last_active":"2024-03-07T07:40:09.361006+0000","last_peered":"2024-03-07T07:40:09.361006+0000","last_clean":"2024-03-07T07:40:09.361006+0000","last_became_active":"2024-03-07T07:39:13.841153+0000","last_became_peered":"2024-03-07T07:39:13.841153+0000","last_unstale":"2024-03-07T07:40:09.361006+0000","last_undegraded":"2024-03-07T07:40:09.361006+0000","last_fullsized":"2024-03-07T07:40:09.361006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T19:38:41.264033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.359467+0000","last_change":"2024-03-07T07:39:14.603948+0000","last_active":"2024-03-07T07:40:09.359467+0000","last_peered":"2024-03-07T07:40:09.359467+0000","last_clean":"2024-03-07T07:40:09.359467+0000","last_became_active":"2024-03-07T07:39:14.603724+0000","last_became_peered":"2024-03-07T07:39:14.603724+0000","last_unstale":"2024-03-07T07:40:09.359467+0000","last_undegraded":"2024-03-07T07:40:09.359467+0000","last_fullsized":"2024-03-07T07:40:09.359467+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T19:29:59.296537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.366969+0000","last_change":"2024-03-07T07:39:13.831965+0000","last_active":"2024-03-07T07:40:09.366969+0000","last_peered":"2024-03-07T07:40:09.366969+0000","last_clean":"2024-03-07T07:40:09.366969+0000","last_became_active":"2024-03-07T07:39:13.831617+0000","last_became_peered":"2024-03-07T07:39:13.831617+0000","last_unstale":"2024-03-07T07:40:09.366969+0000","last_undegraded":"2024-03-07T07:40:09.366969+0000","last_fullsized":"2024-03-07T07:40:09.366969+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:46:49.060697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.359245+0000","last_change":"2024-03-07T07:39:14.616265+0000","last_active":"2024-03-07T07:40:09.359245+0000","last_peered":"2024-03-07T07:40:09.359245+0000","last_clean":"2024-03-07T07:40:09.359245+0000","last_became_active":"2024-03-07T07:39:14.615206+0000","last_became_peered":"2024-03-07T07:39:14.615206+0000","last_unstale":"2024-03-07T07:40:09.359245+0000","last_undegraded":"2024-03-07T07:40:09.359245+0000","last_fullsized":"2024-03-07T07:40:09.359245+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:39:21.997327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'4","reported_seq":97,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:27.954886+0000","last_change":"2024-03-07T07:39:13.833311+0000","last_active":"2024-03-07T07:40:27.954886+0000","last_peered":"2024-03-07T07:40:27.954886+0000","last_clean":"2024-03-07T07:40:27.954886+0000","last_became_active":"2024-03-07T07:39:13.832863+0000","last_became_peered":"2024-03-07T07:39:13.832863+0000","last_unstale":"2024-03-07T07:40:27.954886+0000","last_undegraded":"2024-03-07T07:40:27.954886+0000","last_fullsized":"2024-03-07T07:40:27.954886+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:43:50.988905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":75,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.358786+0000","last_change":"2024-03-07T07:39:14.615740+0000","last_active":"2024-03-07T07:40:09.358786+0000","last_peered":"2024-03-07T07:40:09.358786+0000","last_clean":"2024-03-07T07:40:09.358786+0000","last_became_active":"2024-03-07T07:39:14.615394+0000","last_became_peered":"2024-03-07T07:39:14.615394+0000","last_unstale":"2024-03-07T07:40:09.358786+0000","last_undegraded":"2024-03-07T07:40:09.358786+0000","last_fullsized":"2024-03-07T07:40:09.358786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:17:19.107374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.374773+0000","last_change":"2024-03-07T07:40:09.374343+0000","last_active":"2024-03-07T07:40:09.374773+0000","last_peered":"2024-03-07T07:40:09.374773+0000","last_clean":"2024-03-07T07:40:09.374773+0000","last_became_active":"2024-03-07T07:40:09.362272+0000","last_became_peered":"2024-03-07T07:40:09.362272+0000","last_unstale":"2024-03-07T07:40:09.374773+0000","last_undegraded":"2024-03-07T07:40:09.374773+0000","last_fullsized":"2024-03-07T07:40:09.374773+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:42:17.076409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.366972+0000","last_change":"2024-03-07T07:39:14.617134+0000","last_active":"2024-03-07T07:40:09.366972+0000","last_peered":"2024-03-07T07:40:09.366972+0000","last_clean":"2024-03-07T07:40:09.366972+0000","last_became_active":"2024-03-07T07:39:14.616813+0000","last_became_peered":"2024-03-07T07:39:14.616813+0000","last_unstale":"2024-03-07T07:40:09.366972+0000","last_undegraded":"2024-03-07T07:40:09.366972+0000","last_fullsized":"2024-03-07T07:40:09.366972+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:12:18.477309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.362021+0000","last_change":"2024-03-07T07:39:13.828783+0000","last_active":"2024-03-07T07:40:09.362021+0000","last_peered":"2024-03-07T07:40:09.362021+0000","last_clean":"2024-03-07T07:40:09.362021+0000","last_became_active":"2024-03-07T07:39:13.828427+0000","last_became_peered":"2024-03-07T07:39:13.828427+0000","last_unstale":"2024-03-07T07:40:09.362021+0000","last_undegraded":"2024-03-07T07:40:09.362021+0000","last_fullsized":"2024-03-07T07:40:09.362021+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:49:21.074014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.458018+0000","last_change":"2024-03-07T07:40:09.458018+0000","last_active":"2024-03-07T07:40:09.458018+0000","last_peered":"2024-03-07T07:40:09.458018+0000","last_clean":"2024-03-07T07:40:09.458018+0000","last_became_active":"2024-03-07T07:40:09.457643+0000","last_became_peered":"2024-03-07T07:40:09.457643+0000","last_unstale":"2024-03-07T07:40:09.458018+0000","last_undegraded":"2024-03-07T07:40:09.458018+0000","last_fullsized":"2024-03-07T07:40:09.458018+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:18:33.643312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.456732+0000","last_change":"2024-03-07T07:39:13.833342+0000","last_active":"2024-03-07T07:40:09.456732+0000","last_peered":"2024-03-07T07:40:09.456732+0000","last_clean":"2024-03-07T07:40:09.456732+0000","last_became_active":"2024-03-07T07:39:13.833013+0000","last_became_peered":"2024-03-07T07:39:13.833013+0000","last_unstale":"2024-03-07T07:40:09.456732+0000","last_undegraded":"2024-03-07T07:40:09.456732+0000","last_fullsized":"2024-03-07T07:40:09.456732+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:39:59.821206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.460404+0000","last_change":"2024-03-07T07:40:09.460404+0000","last_active":"2024-03-07T07:40:09.460404+0000","last_peered":"2024-03-07T07:40:09.460404+0000","last_clean":"2024-03-07T07:40:09.460404+0000","last_became_active":"2024-03-07T07:40:09.457470+0000","last_became_peered":"2024-03-07T07:40:09.457470+0000","last_unstale":"2024-03-07T07:40:09.460404+0000","last_undegraded":"2024-03-07T07:40:09.460404+0000","last_fullsized":"2024-03-07T07:40:09.460404+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:37:30.728304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.358964+0000","last_change":"2024-03-07T07:39:13.836434+0000","last_active":"2024-03-07T07:40:09.358964+0000","last_peered":"2024-03-07T07:40:09.358964+0000","last_clean":"2024-03-07T07:40:09.358964+0000","last_became_active":"2024-03-07T07:39:13.835953+0000","last_became_peered":"2024-03-07T07:39:13.835953+0000","last_unstale":"2024-03-07T07:40:09.358964+0000","last_undegraded":"2024-03-07T07:40:09.358964+0000","last_fullsized":"2024-03-07T07:40:09.358964+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:23:20.941845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.360385+0000","last_change":"2024-03-07T07:39:14.618062+0000","last_active":"2024-03-07T07:40:09.360385+0000","last_peered":"2024-03-07T07:40:09.360385+0000","last_clean":"2024-03-07T07:40:09.360385+0000","last_became_active":"2024-03-07T07:39:14.617715+0000","last_became_peered":"2024-03-07T07:39:14.617715+0000","last_unstale":"2024-03-07T07:40:09.360385+0000","last_undegraded":"2024-03-07T07:40:09.360385+0000","last_fullsized":"2024-03-07T07:40:09.360385+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:51:29.200771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.366909+0000","last_change":"2024-03-07T07:39:14.617522+0000","last_active":"2024-03-07T07:40:09.366909+0000","last_peered":"2024-03-07T07:40:09.366909+0000","last_clean":"2024-03-07T07:40:09.366909+0000","last_became_active":"2024-03-07T07:39:14.616862+0000","last_became_peered":"2024-03-07T07:39:14.616862+0000","last_unstale":"2024-03-07T07:40:09.366909+0000","last_undegraded":"2024-03-07T07:40:09.366909+0000","last_fullsized":"2024-03-07T07:40:09.366909+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:37:58.473662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":93,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:26.953365+0000","last_change":"2024-03-07T07:39:13.835090+0000","last_active":"2024-03-07T07:40:26.953365+0000","last_peered":"2024-03-07T07:40:26.953365+0000","last_clean":"2024-03-07T07:40:26.953365+0000","last_became_active":"2024-03-07T07:39:13.834765+0000","last_became_peered":"2024-03-07T07:39:13.834765+0000","last_unstale":"2024-03-07T07:40:26.953365+0000","last_undegraded":"2024-03-07T07:40:26.953365+0000","last_fullsized":"2024-03-07T07:40:26.953365+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:50:41.530959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":70,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.362316+0000","last_change":"2024-03-07T07:39:14.615224+0000","last_active":"2024-03-07T07:40:09.362316+0000","last_peered":"2024-03-07T07:40:09.362316+0000","last_clean":"2024-03-07T07:40:09.362316+0000","last_became_active":"2024-03-07T07:39:14.614942+0000","last_became_peered":"2024-03-07T07:39:14.614942+0000","last_unstale":"2024-03-07T07:40:09.362316+0000","last_undegraded":"2024-03-07T07:40:09.362316+0000","last_fullsized":"2024-03-07T07:40:09.362316+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:30:48.321950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.357608+0000","last_change":"2024-03-07T07:39:13.833245+0000","last_active":"2024-03-07T07:40:09.357608+0000","last_peered":"2024-03-07T07:40:09.357608+0000","last_clean":"2024-03-07T07:40:09.357608+0000","last_became_active":"2024-03-07T07:39:13.832837+0000","last_became_peered":"2024-03-07T07:39:13.832837+0000","last_unstale":"2024-03-07T07:40:09.357608+0000","last_undegraded":"2024-03-07T07:40:09.357608+0000","last_fullsized":"2024-03-07T07:40:09.357608+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:47:28.204394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":343,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.358899+0000","last_change":"2024-03-07T07:37:58.585313+0000","last_active":"2024-03-07T07:40:09.358899+0000","last_peered":"2024-03-07T07:40:09.358899+0000","last_clean":"2024-03-07T07:40:09.358899+0000","last_became_active":"2024-03-07T07:37:58.584808+0000","last_became_peered":"2024-03-07T07:37:58.584808+0000","last_unstale":"2024-03-07T07:40:09.358899+0000","last_undegraded":"2024-03-07T07:40:09.358899+0000","last_fullsized":"2024-03-07T07:40:09.358899+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:34:34.612837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:34:34.612837+0000","last_clean_scrub_stamp":"2024-03-07T07:34:34.612837+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:56:56.998601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.359380+0000","last_change":"2024-03-07T07:39:14.615907+0000","last_active":"2024-03-07T07:40:09.359380+0000","last_peered":"2024-03-07T07:40:09.359380+0000","last_clean":"2024-03-07T07:40:09.359380+0000","last_became_active":"2024-03-07T07:39:14.615587+0000","last_became_peered":"2024-03-07T07:39:14.615587+0000","last_unstale":"2024-03-07T07:40:09.359380+0000","last_undegraded":"2024-03-07T07:40:09.359380+0000","last_fullsized":"2024-03-07T07:40:09.359380+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:08:20.363895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.358970+0000","last_change":"2024-03-07T07:39:13.832143+0000","last_active":"2024-03-07T07:40:09.358970+0000","last_peered":"2024-03-07T07:40:09.358970+0000","last_clean":"2024-03-07T07:40:09.358970+0000","last_became_active":"2024-03-07T07:39:13.831299+0000","last_became_peered":"2024-03-07T07:39:13.831299+0000","last_unstale":"2024-03-07T07:40:09.358970+0000","last_undegraded":"2024-03-07T07:40:09.358970+0000","last_fullsized":"2024-03-07T07:40:09.358970+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:25:08.965498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.361637+0000","last_change":"2024-03-07T07:39:13.827617+0000","last_active":"2024-03-07T07:40:09.361637+0000","last_peered":"2024-03-07T07:40:09.361637+0000","last_clean":"2024-03-07T07:40:09.361637+0000","last_became_active":"2024-03-07T07:39:13.827188+0000","last_became_peered":"2024-03-07T07:39:13.827188+0000","last_unstale":"2024-03-07T07:40:09.361637+0000","last_undegraded":"2024-03-07T07:40:09.361637+0000","last_fullsized":"2024-03-07T07:40:09.361637+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:35:27.634460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.359617+0000","last_change":"2024-03-07T07:39:14.608307+0000","last_active":"2024-03-07T07:40:09.359617+0000","last_peered":"2024-03-07T07:40:09.359617+0000","last_clean":"2024-03-07T07:40:09.359617+0000","last_became_active":"2024-03-07T07:39:14.607948+0000","last_became_peered":"2024-03-07T07:39:14.607948+0000","last_unstale":"2024-03-07T07:40:09.359617+0000","last_undegraded":"2024-03-07T07:40:09.359617+0000","last_fullsized":"2024-03-07T07:40:09.359617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:36:27.294925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.457084+0000","last_change":"2024-03-07T07:39:13.831123+0000","last_active":"2024-03-07T07:40:09.457084+0000","last_peered":"2024-03-07T07:40:09.457084+0000","last_clean":"2024-03-07T07:40:09.457084+0000","last_became_active":"2024-03-07T07:39:13.830801+0000","last_became_peered":"2024-03-07T07:39:13.830801+0000","last_unstale":"2024-03-07T07:40:09.457084+0000","last_undegraded":"2024-03-07T07:40:09.457084+0000","last_fullsized":"2024-03-07T07:40:09.457084+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:19:54.346449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.362610+0000","last_change":"2024-03-07T07:39:14.610510+0000","last_active":"2024-03-07T07:40:09.362610+0000","last_peered":"2024-03-07T07:40:09.362610+0000","last_clean":"2024-03-07T07:40:09.362610+0000","last_became_active":"2024-03-07T07:39:14.609908+0000","last_became_peered":"2024-03-07T07:39:14.609908+0000","last_unstale":"2024-03-07T07:40:09.362610+0000","last_undegraded":"2024-03-07T07:40:09.362610+0000","last_fullsized":"2024-03-07T07:40:09.362610+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:51:32.045296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.357673+0000","last_change":"2024-03-07T07:39:13.838605+0000","last_active":"2024-03-07T07:40:09.357673+0000","last_peered":"2024-03-07T07:40:09.357673+0000","last_clean":"2024-03-07T07:40:09.357673+0000","last_became_active":"2024-03-07T07:39:13.838362+0000","last_became_peered":"2024-03-07T07:39:13.838362+0000","last_unstale":"2024-03-07T07:40:09.357673+0000","last_undegraded":"2024-03-07T07:40:09.357673+0000","last_fullsized":"2024-03-07T07:40:09.357673+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:23:47.039496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.366588+0000","last_change":"2024-03-07T07:40:09.366588+0000","last_active":"2024-03-07T07:40:09.366588+0000","last_peered":"2024-03-07T07:40:09.366588+0000","last_clean":"2024-03-07T07:40:09.366588+0000","last_became_active":"2024-03-07T07:40:09.366133+0000","last_became_peered":"2024-03-07T07:40:09.366133+0000","last_unstale":"2024-03-07T07:40:09.366588+0000","last_undegraded":"2024-03-07T07:40:09.366588+0000","last_fullsized":"2024-03-07T07:40:09.366588+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:48:09.932164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.362890+0000","last_change":"2024-03-07T07:39:13.834640+0000","last_active":"2024-03-07T07:40:09.362890+0000","last_peered":"2024-03-07T07:40:09.362890+0000","last_clean":"2024-03-07T07:40:09.362890+0000","last_became_active":"2024-03-07T07:39:13.834247+0000","last_became_peered":"2024-03-07T07:39:13.834247+0000","last_unstale":"2024-03-07T07:40:09.362890+0000","last_undegraded":"2024-03-07T07:40:09.362890+0000","last_fullsized":"2024-03-07T07:40:09.362890+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:40:55.487824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.360713+0000","last_change":"2024-03-07T07:39:14.603762+0000","last_active":"2024-03-07T07:40:09.360713+0000","last_peered":"2024-03-07T07:40:09.360713+0000","last_clean":"2024-03-07T07:40:09.360713+0000","last_became_active":"2024-03-07T07:39:14.603444+0000","last_became_peered":"2024-03-07T07:39:14.603444+0000","last_unstale":"2024-03-07T07:40:09.360713+0000","last_undegraded":"2024-03-07T07:40:09.360713+0000","last_fullsized":"2024-03-07T07:40:09.360713+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:18:03.075996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.361831+0000","last_change":"2024-03-07T07:39:13.833864+0000","last_active":"2024-03-07T07:40:09.361831+0000","last_peered":"2024-03-07T07:40:09.361831+0000","last_clean":"2024-03-07T07:40:09.361831+0000","last_became_active":"2024-03-07T07:39:13.833570+0000","last_became_peered":"2024-03-07T07:39:13.833570+0000","last_unstale":"2024-03-07T07:40:09.361831+0000","last_undegraded":"2024-03-07T07:40:09.361831+0000","last_fullsized":"2024-03-07T07:40:09.361831+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:20:16.387319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.359020+0000","last_change":"2024-03-07T07:39:14.610023+0000","last_active":"2024-03-07T07:40:09.359020+0000","last_peered":"2024-03-07T07:40:09.359020+0000","last_clean":"2024-03-07T07:40:09.359020+0000","last_became_active":"2024-03-07T07:39:14.609671+0000","last_became_peered":"2024-03-07T07:39:14.609671+0000","last_unstale":"2024-03-07T07:40:09.359020+0000","last_undegraded":"2024-03-07T07:40:09.359020+0000","last_fullsized":"2024-03-07T07:40:09.359020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:41:48.384795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.361027+0000","last_change":"2024-03-07T07:39:13.841326+0000","last_active":"2024-03-07T07:40:09.361027+0000","last_peered":"2024-03-07T07:40:09.361027+0000","last_clean":"2024-03-07T07:40:09.361027+0000","last_became_active":"2024-03-07T07:39:13.840955+0000","last_became_peered":"2024-03-07T07:39:13.840955+0000","last_unstale":"2024-03-07T07:40:09.361027+0000","last_undegraded":"2024-03-07T07:40:09.361027+0000","last_fullsized":"2024-03-07T07:40:09.361027+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:56:25.944653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.362359+0000","last_change":"2024-03-07T07:39:14.617783+0000","last_active":"2024-03-07T07:40:09.362359+0000","last_peered":"2024-03-07T07:40:09.362359+0000","last_clean":"2024-03-07T07:40:09.362359+0000","last_became_active":"2024-03-07T07:39:14.617472+0000","last_became_peered":"2024-03-07T07:39:14.617472+0000","last_unstale":"2024-03-07T07:40:09.362359+0000","last_undegraded":"2024-03-07T07:40:09.362359+0000","last_fullsized":"2024-03-07T07:40:09.362359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:35:48.618321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.361690+0000","last_change":"2024-03-07T07:39:13.837602+0000","last_active":"2024-03-07T07:40:09.361690+0000","last_peered":"2024-03-07T07:40:09.361690+0000","last_clean":"2024-03-07T07:40:09.361690+0000","last_became_active":"2024-03-07T07:39:13.837275+0000","last_became_peered":"2024-03-07T07:39:13.837275+0000","last_unstale":"2024-03-07T07:40:09.361690+0000","last_undegraded":"2024-03-07T07:40:09.361690+0000","last_fullsized":"2024-03-07T07:40:09.361690+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:40:43.267842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.359069+0000","last_change":"2024-03-07T07:39:14.616702+0000","last_active":"2024-03-07T07:40:09.359069+0000","last_peered":"2024-03-07T07:40:09.359069+0000","last_clean":"2024-03-07T07:40:09.359069+0000","last_became_active":"2024-03-07T07:39:14.614720+0000","last_became_peered":"2024-03-07T07:39:14.614720+0000","last_unstale":"2024-03-07T07:40:09.359069+0000","last_undegraded":"2024-03-07T07:40:09.359069+0000","last_fullsized":"2024-03-07T07:40:09.359069+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:14:45.648231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.361725+0000","last_change":"2024-03-07T07:39:13.836641+0000","last_active":"2024-03-07T07:40:09.361725+0000","last_peered":"2024-03-07T07:40:09.361725+0000","last_clean":"2024-03-07T07:40:09.361725+0000","last_became_active":"2024-03-07T07:39:13.834012+0000","last_became_peered":"2024-03-07T07:39:13.834012+0000","last_unstale":"2024-03-07T07:40:09.361725+0000","last_undegraded":"2024-03-07T07:40:09.361725+0000","last_fullsized":"2024-03-07T07:40:09.361725+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:40:49.596816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.359265+0000","last_change":"2024-03-07T07:39:14.611074+0000","last_active":"2024-03-07T07:40:09.359265+0000","last_peered":"2024-03-07T07:40:09.359265+0000","last_clean":"2024-03-07T07:40:09.359265+0000","last_became_active":"2024-03-07T07:39:14.610651+0000","last_became_peered":"2024-03-07T07:39:14.610651+0000","last_unstale":"2024-03-07T07:40:09.359265+0000","last_undegraded":"2024-03-07T07:40:09.359265+0000","last_fullsized":"2024-03-07T07:40:09.359265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:18:29.684266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.367142+0000","last_change":"2024-03-07T07:39:13.839743+0000","last_active":"2024-03-07T07:40:09.367142+0000","last_peered":"2024-03-07T07:40:09.367142+0000","last_clean":"2024-03-07T07:40:09.367142+0000","last_became_active":"2024-03-07T07:39:13.839079+0000","last_became_peered":"2024-03-07T07:39:13.839079+0000","last_unstale":"2024-03-07T07:40:09.367142+0000","last_undegraded":"2024-03-07T07:40:09.367142+0000","last_fullsized":"2024-03-07T07:40:09.367142+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:11:00.757666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.359327+0000","last_change":"2024-03-07T07:39:14.615941+0000","last_active":"2024-03-07T07:40:09.359327+0000","last_peered":"2024-03-07T07:40:09.359327+0000","last_clean":"2024-03-07T07:40:09.359327+0000","last_became_active":"2024-03-07T07:39:14.615577+0000","last_became_peered":"2024-03-07T07:39:14.615577+0000","last_unstale":"2024-03-07T07:40:09.359327+0000","last_undegraded":"2024-03-07T07:40:09.359327+0000","last_fullsized":"2024-03-07T07:40:09.359327+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:06:05.457297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.371904+0000","last_change":"2024-03-07T07:40:09.371904+0000","last_active":"2024-03-07T07:40:09.371904+0000","last_peered":"2024-03-07T07:40:09.371904+0000","last_clean":"2024-03-07T07:40:09.371904+0000","last_became_active":"2024-03-07T07:40:09.366168+0000","last_became_peered":"2024-03-07T07:40:09.366168+0000","last_unstale":"2024-03-07T07:40:09.371904+0000","last_undegraded":"2024-03-07T07:40:09.371904+0000","last_fullsized":"2024-03-07T07:40:09.371904+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:12.808038+0000","last_clean_scrub_stamp":"2024-03-07T07:39:12.808038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:08:59.697518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.359421+0000","last_change":"2024-03-07T07:39:14.611652+0000","last_active":"2024-03-07T07:40:09.359421+0000","last_peered":"2024-03-07T07:40:09.359421+0000","last_clean":"2024-03-07T07:40:09.359421+0000","last_became_active":"2024-03-07T07:39:14.611300+0000","last_became_peered":"2024-03-07T07:39:14.611300+0000","last_unstale":"2024-03-07T07:40:09.359421+0000","last_undegraded":"2024-03-07T07:40:09.359421+0000","last_fullsized":"2024-03-07T07:40:09.359421+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:30:44.240560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.359064+0000","last_change":"2024-03-07T07:39:14.617630+0000","last_active":"2024-03-07T07:40:09.359064+0000","last_peered":"2024-03-07T07:40:09.359064+0000","last_clean":"2024-03-07T07:40:09.359064+0000","last_became_active":"2024-03-07T07:39:14.616897+0000","last_became_peered":"2024-03-07T07:39:14.616897+0000","last_unstale":"2024-03-07T07:40:09.359064+0000","last_undegraded":"2024-03-07T07:40:09.359064+0000","last_fullsized":"2024-03-07T07:40:09.359064+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T19:24:35.522835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.361102+0000","last_change":"2024-03-07T07:39:14.616670+0000","last_active":"2024-03-07T07:40:09.361102+0000","last_peered":"2024-03-07T07:40:09.361102+0000","last_clean":"2024-03-07T07:40:09.361102+0000","last_became_active":"2024-03-07T07:39:14.615720+0000","last_became_peered":"2024-03-07T07:39:14.615720+0000","last_unstale":"2024-03-07T07:40:09.361102+0000","last_undegraded":"2024-03-07T07:40:09.361102+0000","last_fullsized":"2024-03-07T07:40:09.361102+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:51:59.533172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.361129+0000","last_change":"2024-03-07T07:39:14.606794+0000","last_active":"2024-03-07T07:40:09.361129+0000","last_peered":"2024-03-07T07:40:09.361129+0000","last_clean":"2024-03-07T07:40:09.361129+0000","last_became_active":"2024-03-07T07:39:14.606395+0000","last_became_peered":"2024-03-07T07:39:14.606395+0000","last_unstale":"2024-03-07T07:40:09.361129+0000","last_undegraded":"2024-03-07T07:40:09.361129+0000","last_fullsized":"2024-03-07T07:40:09.361129+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:54:26.263361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.356940+0000","last_change":"2024-03-07T07:39:14.616627+0000","last_active":"2024-03-07T07:40:09.356940+0000","last_peered":"2024-03-07T07:40:09.356940+0000","last_clean":"2024-03-07T07:40:09.356940+0000","last_became_active":"2024-03-07T07:39:14.615865+0000","last_became_peered":"2024-03-07T07:39:14.615865+0000","last_unstale":"2024-03-07T07:40:09.356940+0000","last_undegraded":"2024-03-07T07:40:09.356940+0000","last_fullsized":"2024-03-07T07:40:09.356940+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:15:46.997786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.359611+0000","last_change":"2024-03-07T07:39:14.616493+0000","last_active":"2024-03-07T07:40:09.359611+0000","last_peered":"2024-03-07T07:40:09.359611+0000","last_clean":"2024-03-07T07:40:09.359611+0000","last_became_active":"2024-03-07T07:39:14.616185+0000","last_became_peered":"2024-03-07T07:39:14.616185+0000","last_unstale":"2024-03-07T07:40:09.359611+0000","last_undegraded":"2024-03-07T07:40:09.359611+0000","last_fullsized":"2024-03-07T07:40:09.359611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:51:57.888472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.358622+0000","last_change":"2024-03-07T07:39:14.614720+0000","last_active":"2024-03-07T07:40:09.358622+0000","last_peered":"2024-03-07T07:40:09.358622+0000","last_clean":"2024-03-07T07:40:09.358622+0000","last_became_active":"2024-03-07T07:39:14.614256+0000","last_became_peered":"2024-03-07T07:39:14.614256+0000","last_unstale":"2024-03-07T07:40:09.358622+0000","last_undegraded":"2024-03-07T07:40:09.358622+0000","last_fullsized":"2024-03-07T07:40:09.358622+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:58:24.497664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.363002+0000","last_change":"2024-03-07T07:39:14.611152+0000","last_active":"2024-03-07T07:40:09.363002+0000","last_peered":"2024-03-07T07:40:09.363002+0000","last_clean":"2024-03-07T07:40:09.363002+0000","last_became_active":"2024-03-07T07:39:14.610616+0000","last_became_peered":"2024-03-07T07:39:14.610616+0000","last_unstale":"2024-03-07T07:40:09.363002+0000","last_undegraded":"2024-03-07T07:40:09.363002+0000","last_fullsized":"2024-03-07T07:40:09.363002+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:59:36.283163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.362611+0000","last_change":"2024-03-07T07:39:14.610762+0000","last_active":"2024-03-07T07:40:09.362611+0000","last_peered":"2024-03-07T07:40:09.362611+0000","last_clean":"2024-03-07T07:40:09.362611+0000","last_became_active":"2024-03-07T07:39:14.610440+0000","last_became_peered":"2024-03-07T07:39:14.610440+0000","last_unstale":"2024-03-07T07:40:09.362611+0000","last_undegraded":"2024-03-07T07:40:09.362611+0000","last_fullsized":"2024-03-07T07:40:09.362611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:45:18.046694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.366823+0000","last_change":"2024-03-07T07:39:14.610322+0000","last_active":"2024-03-07T07:40:09.366823+0000","last_peered":"2024-03-07T07:40:09.366823+0000","last_clean":"2024-03-07T07:40:09.366823+0000","last_became_active":"2024-03-07T07:39:14.610008+0000","last_became_peered":"2024-03-07T07:39:14.610008+0000","last_unstale":"2024-03-07T07:40:09.366823+0000","last_undegraded":"2024-03-07T07:40:09.366823+0000","last_fullsized":"2024-03-07T07:40:09.366823+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:35:46.071475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.361793+0000","last_change":"2024-03-07T07:39:14.610084+0000","last_active":"2024-03-07T07:40:09.361793+0000","last_peered":"2024-03-07T07:40:09.361793+0000","last_clean":"2024-03-07T07:40:09.361793+0000","last_became_active":"2024-03-07T07:39:14.609085+0000","last_became_peered":"2024-03-07T07:39:14.609085+0000","last_unstale":"2024-03-07T07:40:09.361793+0000","last_undegraded":"2024-03-07T07:40:09.361793+0000","last_fullsized":"2024-03-07T07:40:09.361793+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:24:31.206928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.363095+0000","last_change":"2024-03-07T07:39:14.611486+0000","last_active":"2024-03-07T07:40:09.363095+0000","last_peered":"2024-03-07T07:40:09.363095+0000","last_clean":"2024-03-07T07:40:09.363095+0000","last_became_active":"2024-03-07T07:39:14.611128+0000","last_became_peered":"2024-03-07T07:39:14.611128+0000","last_unstale":"2024-03-07T07:40:09.363095+0000","last_undegraded":"2024-03-07T07:40:09.363095+0000","last_fullsized":"2024-03-07T07:40:09.363095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:48:07.831530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.359722+0000","last_change":"2024-03-07T07:39:14.604771+0000","last_active":"2024-03-07T07:40:09.359722+0000","last_peered":"2024-03-07T07:40:09.359722+0000","last_clean":"2024-03-07T07:40:09.359722+0000","last_became_active":"2024-03-07T07:39:14.604494+0000","last_became_peered":"2024-03-07T07:39:14.604494+0000","last_unstale":"2024-03-07T07:40:09.359722+0000","last_undegraded":"2024-03-07T07:40:09.359722+0000","last_fullsized":"2024-03-07T07:40:09.359722+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:42:45.141833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-07T07:40:09.358435+0000","last_change":"2024-03-07T07:39:14.618689+0000","last_active":"2024-03-07T07:40:09.358435+0000","last_peered":"2024-03-07T07:40:09.358435+0000","last_clean":"2024-03-07T07:40:09.358435+0000","last_became_active":"2024-03-07T07:39:14.618330+0000","last_became_peered":"2024-03-07T07:39:14.618330+0000","last_unstale":"2024-03-07T07:40:09.358435+0000","last_undegraded":"2024-03-07T07:40:09.358435+0000","last_fullsized":"2024-03-07T07:40:09.358435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:39:13.812982+0000","last_clean_scrub_stamp":"2024-03-07T07:39:13.812982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:23:07.431656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282597,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62913,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":436,"num_read_kb":283,"num_write":154,"num_write_kb":105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":59024,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":364544,"data_stored":188739,"data_compressed":13305,"data_compressed_allocated":20480,"data_compressed_original":73728,"omap_allocated":0,"internal_metadata":0},"log_size":133,"ondisk_log_size":133,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383710,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":950272,"data_stored":456388,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645346,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1089536,"data_stored":1107060,"data_compressed":43594,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808871,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":962560,"data_stored":456629,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070508,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":456320,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332145,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1007616,"data_stored":1048362,"data_compressed":41719,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495670,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":946176,"data_stored":455901,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":514606,"data_compressed":7654,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":457172,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411402,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":962560,"data_stored":457044,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182470,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":456095,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920833,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":974848,"data_stored":515641,"data_compressed":15334,"data_compressed_allocated":20480,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757308,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":1047655,"data_compressed":41719,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34299,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34299,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34299,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35940,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35940,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35940,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35940,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35940,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":58912,"data_compressed":1875,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":59947,"data_compressed":9555,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":60881,"data_compressed":1875,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-07T07:40:34.105 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-07T07:40:34.105 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-07T07:40:34.105 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-07T07:40:34.105 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph health --format=json 2024-03-07T07:40:34.356 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:40:34.371 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:34 smithi160 ceph-mon[31963]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-07T07:40:34.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:34 smithi178 ceph-mon[36650]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-07T07:40:34.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:34 smithi177 ceph-mon[36578]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-07T07:40:35.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:35 smithi160 ceph-mon[31963]: pgmap v330: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-07T07:40:35.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:35 smithi178 ceph-mon[36650]: pgmap v330: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-07T07:40:35.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:35 smithi177 ceph-mon[36578]: pgmap v330: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-07T07:40:36.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.061+0000 7f6ac9ebe700 1 -- 172.21.15.160:0/1541975238 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ac406a8d0 msgr2=0x7f6ac406ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:36.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.061+0000 7f6ac9ebe700 1 --2- 172.21.15.160:0/1541975238 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ac406a8d0 0x7f6ac406ad50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8009990 tx=0x7f6ab8009ca0 comp rx=0 tx=0).stop 2024-03-07T07:40:36.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.061+0000 7f6ac9ebe700 1 -- 172.21.15.160:0/1541975238 shutdown_connections 2024-03-07T07:40:36.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.061+0000 7f6ac9ebe700 1 --2- 172.21.15.160:0/1541975238 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ac406b290 0x7f6ac410efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:36.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.061+0000 7f6ac9ebe700 1 --2- 172.21.15.160:0/1541975238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ac4107160 0x7f6ac4107540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:36.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.061+0000 7f6ac9ebe700 1 --2- 172.21.15.160:0/1541975238 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ac406a8d0 0x7f6ac406ad50 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:36.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.061+0000 7f6ac9ebe700 1 -- 172.21.15.160:0/1541975238 >> 172.21.15.160:0/1541975238 conn(0x7f6ac40780b0 msgr2=0x7f6ac40784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:36.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.061+0000 7f6ac9ebe700 1 -- 172.21.15.160:0/1541975238 shutdown_connections 2024-03-07T07:40:36.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.061+0000 7f6ac9ebe700 1 -- 172.21.15.160:0/1541975238 wait complete. 2024-03-07T07:40:36.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.061+0000 7f6ac9ebe700 1 Processor -- start 2024-03-07T07:40:36.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.062+0000 7f6ac9ebe700 1 -- start start 2024-03-07T07:40:36.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.062+0000 7f6ac9ebe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ac406a8d0 0x7f6ac4198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:36.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.062+0000 7f6ac9ebe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ac406b290 0x7f6ac4198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:36.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.062+0000 7f6ac9ebe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ac4107160 0x7f6ac419d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:36.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.062+0000 7f6ac9ebe700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac4112560 con 0x7f6ac4107160 2024-03-07T07:40:36.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.062+0000 7f6ac9ebe700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac41123e0 con 0x7f6ac406a8d0 2024-03-07T07:40:36.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.062+0000 7f6ac9ebe700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac41126e0 con 0x7f6ac406b290 2024-03-07T07:40:36.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.062+0000 7f6ac3fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ac4107160 0x7f6ac419d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:36.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.062+0000 7f6ac3fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ac4107160 0x7f6ac419d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:38296/0 (socket says 172.21.15.160:38296) 2024-03-07T07:40:36.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.062+0000 7f6ac3fff700 1 -- 172.21.15.160:0/2703546788 learned_addr learned my addr 172.21.15.160:0/2703546788 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:40:36.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.062+0000 7f6ac2ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ac406b290 0x7f6ac4198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:36.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.062+0000 7f6ac37fe700 1 --2- 172.21.15.160:0/2703546788 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ac406a8d0 0x7f6ac4198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:36.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.062+0000 7f6ac2ffd700 1 -- 172.21.15.160:0/2703546788 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ac406a8d0 msgr2=0x7f6ac4198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:36.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.062+0000 7f6ac2ffd700 1 --2- 172.21.15.160:0/2703546788 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ac406a8d0 0x7f6ac4198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:36.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.062+0000 7f6ac2ffd700 1 -- 172.21.15.160:0/2703546788 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ac4107160 msgr2=0x7f6ac419d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:36.065 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.062+0000 7f6ac2ffd700 1 --2- 172.21.15.160:0/2703546788 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ac4107160 0x7f6ac419d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:36.065 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.062+0000 7f6ac2ffd700 1 -- 172.21.15.160:0/2703546788 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac419da30 con 0x7f6ac406b290 2024-03-07T07:40:36.065 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.063+0000 7f6ac2ffd700 1 --2- 172.21.15.160:0/2703546788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ac406b290 0x7f6ac4198f70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6ab80059d0 tx=0x7f6ab800bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:36.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.063+0000 7f6ac0ff9700 1 -- 172.21.15.160:0/2703546788 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab8027070 con 0x7f6ac406b290 2024-03-07T07:40:36.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.063+0000 7f6ac0ff9700 1 -- 172.21.15.160:0/2703546788 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ab8003d70 con 0x7f6ac406b290 2024-03-07T07:40:36.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.063+0000 7f6ac9ebe700 1 -- 172.21.15.160:0/2703546788 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ac419dcc0 con 0x7f6ac406b290 2024-03-07T07:40:36.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.063+0000 7f6ac0ff9700 1 -- 172.21.15.160:0/2703546788 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab8022e30 con 0x7f6ac406b290 2024-03-07T07:40:36.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.063+0000 7f6ac9ebe700 1 -- 172.21.15.160:0/2703546788 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ac419e170 con 0x7f6ac406b290 2024-03-07T07:40:36.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.064+0000 7f6ac9ebe700 1 -- 172.21.15.160:0/2703546788 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ac4068e10 con 0x7f6ac406b290 2024-03-07T07:40:36.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.065+0000 7f6ac0ff9700 1 -- 172.21.15.160:0/2703546788 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) v1 ==== 136805+0+0 (secure 0 0 0) 0x7f6ab802a070 con 0x7f6ac406b290 2024-03-07T07:40:36.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.065+0000 7f6ac0ff9700 1 --2- 172.21.15.160:0/2703546788 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f6aac0a1ea0 0x7f6aac0a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:36.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.065+0000 7f6ac37fe700 1 --2- 172.21.15.160:0/2703546788 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f6aac0a1ea0 0x7f6aac0a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:36.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.065+0000 7f6ac0ff9700 1 -- 172.21.15.160:0/2703546788 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6ab80cecb0 con 0x7f6ac406b290 2024-03-07T07:40:36.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.066+0000 7f6ac37fe700 1 --2- 172.21.15.160:0/2703546788 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f6aac0a1ea0 0x7f6aac0a4360 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4009be0 tx=0x7f6ab4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:36.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.069+0000 7f6ac0ff9700 1 -- 172.21.15.160:0/2703546788 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6ab8013070 con 0x7f6ac406b290 2024-03-07T07:40:36.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.311+0000 7f6ac9ebe700 1 -- 172.21.15.160:0/2703546788 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f6ac402d9f0 con 0x7f6ac406b290 2024-03-07T07:40:36.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.311+0000 7f6ac0ff9700 1 -- 172.21.15.160:0/2703546788 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f6ab809c010 con 0x7f6ac406b290 2024-03-07T07:40:36.312 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:40:36.312 INFO:teuthology.orchestra.run.smithi160.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-07T07:40:36.314 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.313+0000 7f6ac9ebe700 1 -- 172.21.15.160:0/2703546788 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f6aac0a1ea0 msgr2=0x7f6aac0a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:36.314 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.313+0000 7f6ac9ebe700 1 --2- 172.21.15.160:0/2703546788 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f6aac0a1ea0 0x7f6aac0a4360 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4009be0 tx=0x7f6ab4009260 comp rx=0 tx=0).stop 2024-03-07T07:40:36.314 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.313+0000 7f6ac9ebe700 1 -- 172.21.15.160:0/2703546788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ac406b290 msgr2=0x7f6ac4198f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:36.314 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.313+0000 7f6ac9ebe700 1 --2- 172.21.15.160:0/2703546788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ac406b290 0x7f6ac4198f70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6ab80059d0 tx=0x7f6ab800bab0 comp rx=0 tx=0).stop 2024-03-07T07:40:36.314 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.313+0000 7f6ac9ebe700 1 -- 172.21.15.160:0/2703546788 shutdown_connections 2024-03-07T07:40:36.314 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.313+0000 7f6ac9ebe700 1 --2- 172.21.15.160:0/2703546788 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ac4107160 0x7f6ac419d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:36.314 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.313+0000 7f6ac9ebe700 1 --2- 172.21.15.160:0/2703546788 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ac406a8d0 0x7f6ac4198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:36.314 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.313+0000 7f6ac9ebe700 1 --2- 172.21.15.160:0/2703546788 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f6aac0a1ea0 0x7f6aac0a4360 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:36.314 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.314+0000 7f6ac9ebe700 1 --2- 172.21.15.160:0/2703546788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ac406b290 0x7f6ac4198f70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:36.314 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.314+0000 7f6ac9ebe700 1 -- 172.21.15.160:0/2703546788 >> 172.21.15.160:0/2703546788 conn(0x7f6ac40780b0 msgr2=0x7f6ac40ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:36.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.314+0000 7f6ac9ebe700 1 -- 172.21.15.160:0/2703546788 shutdown_connections 2024-03-07T07:40:36.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:36.314+0000 7f6ac9ebe700 1 -- 172.21.15.160:0/2703546788 wait complete. 2024-03-07T07:40:36.673 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:36 smithi160 ceph-mon[31963]: pgmap v331: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-07T07:40:36.673 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:36 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/2703546788' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-07T07:40:36.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:36 smithi178 ceph-mon[36650]: pgmap v331: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-07T07:40:36.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:36 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/2703546788' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-07T07:40:36.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:36 smithi177 ceph-mon[36578]: pgmap v331: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-07T07:40:36.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:36 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/2703546788' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-07T07:40:36.958 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-07T07:40:36.959 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T07:40:37.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.064+0000 7fcdb6e87640 1 -- 172.21.15.160:0/67125855 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdb0102140 msgr2=0x7fcdb01089d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:37.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.064+0000 7fcdb6e87640 1 --2- 172.21.15.160:0/67125855 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdb0102140 0x7fcdb01089d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fcda8009ca0 tx=0x7fcda802c0f0 comp rx=0 tx=0).stop 2024-03-07T07:40:37.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.064+0000 7fcdb6e87640 1 -- 172.21.15.160:0/67125855 shutdown_connections 2024-03-07T07:40:37.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.064+0000 7fcdb6e87640 1 --2- 172.21.15.160:0/67125855 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdb0102140 0x7fcdb01089d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:37.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.064+0000 7fcdb6e87640 1 --2- 172.21.15.160:0/67125855 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcdb0101780 0x7fcdb0101c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:37.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.064+0000 7fcdb6e87640 1 --2- 172.21.15.160:0/67125855 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcdb0100590 0x7fcdb01009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:37.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.064+0000 7fcdb6e87640 1 -- 172.21.15.160:0/67125855 >> 172.21.15.160:0/67125855 conn(0x7fcdb00fbb40 msgr2=0x7fcdb00fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:37.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.064+0000 7fcdb6e87640 1 -- 172.21.15.160:0/67125855 shutdown_connections 2024-03-07T07:40:37.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.064+0000 7fcdb6e87640 1 -- 172.21.15.160:0/67125855 wait complete. 2024-03-07T07:40:37.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.064+0000 7fcdb6e87640 1 Processor -- start 2024-03-07T07:40:37.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.064+0000 7fcdb6e87640 1 -- start start 2024-03-07T07:40:37.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.064+0000 7fcdb6e87640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdb0100590 0x7fcdb006a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:37.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.064+0000 7fcdb6e87640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcdb0101780 0x7fcdb019c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:37.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.064+0000 7fcdb6e87640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcdb0102140 0x7fcdb019ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:37.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.064+0000 7fcdb6e87640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb010c220 con 0x7fcdb0100590 2024-03-07T07:40:37.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.064+0000 7fcdb6e87640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb010c0a0 con 0x7fcdb0102140 2024-03-07T07:40:37.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.064+0000 7fcdb6e87640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb010c3a0 con 0x7fcdb0101780 2024-03-07T07:40:37.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.064+0000 7fcdb4c23640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdb0100590 0x7fcdb006a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:37.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.064+0000 7fcdb4c23640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdb0100590 0x7fcdb006a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:38324/0 (socket says 172.21.15.160:38324) 2024-03-07T07:40:37.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.064+0000 7fcdb4c23640 1 -- 172.21.15.160:0/1158109503 learned_addr learned my addr 172.21.15.160:0/1158109503 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:40:37.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.065+0000 7fcdaffff640 1 --2- 172.21.15.160:0/1158109503 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcdb0101780 0x7fcdb019c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:37.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.065+0000 7fcdb5424640 1 --2- 172.21.15.160:0/1158109503 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcdb0102140 0x7fcdb019ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:37.068 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.065+0000 7fcdb4c23640 1 -- 172.21.15.160:0/1158109503 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcdb0101780 msgr2=0x7fcdb019c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:37.068 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.065+0000 7fcdb4c23640 1 --2- 172.21.15.160:0/1158109503 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcdb0101780 0x7fcdb019c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:37.068 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.065+0000 7fcdb4c23640 1 -- 172.21.15.160:0/1158109503 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcdb0102140 msgr2=0x7fcdb019ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:37.068 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.065+0000 7fcdb4c23640 1 --2- 172.21.15.160:0/1158109503 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcdb0102140 0x7fcdb019ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:37.068 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.065+0000 7fcdb4c23640 1 -- 172.21.15.160:0/1158109503 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb019f570 con 0x7fcdb0100590 2024-03-07T07:40:37.068 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.065+0000 7fcdb4c23640 1 --2- 172.21.15.160:0/1158109503 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdb0100590 0x7fcdb006a820 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fcda000e9c0 tx=0x7fcda000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:37.068 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.065+0000 7fcdadffb640 1 -- 172.21.15.160:0/1158109503 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda000cd80 con 0x7fcdb0100590 2024-03-07T07:40:37.068 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.065+0000 7fcdadffb640 1 -- 172.21.15.160:0/1158109503 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcda00035f0 con 0x7fcdb0100590 2024-03-07T07:40:37.068 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.065+0000 7fcdb6e87640 1 -- 172.21.15.160:0/1158109503 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdb019f800 con 0x7fcdb0100590 2024-03-07T07:40:37.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.065+0000 7fcdadffb640 1 -- 172.21.15.160:0/1158109503 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda0010690 con 0x7fcdb0100590 2024-03-07T07:40:37.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.065+0000 7fcdb6e87640 1 -- 172.21.15.160:0/1158109503 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdb019fd50 con 0x7fcdb0100590 2024-03-07T07:40:37.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.066+0000 7fcdb6e87640 1 -- 172.21.15.160:0/1158109503 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdb0107850 con 0x7fcdb0100590 2024-03-07T07:40:37.070 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.067+0000 7fcdadffb640 1 -- 172.21.15.160:0/1158109503 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) v1 ==== 136805+0+0 (secure 0 0 0) 0x7fcda00037a0 con 0x7fcdb0100590 2024-03-07T07:40:37.070 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.068+0000 7fcdadffb640 1 --2- 172.21.15.160:0/1158109503 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fcd980a1e10 0x7fcd980a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:37.070 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.068+0000 7fcdaffff640 1 --2- 172.21.15.160:0/1158109503 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fcd980a1e10 0x7fcd980a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:37.070 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.068+0000 7fcdadffb640 1 -- 172.21.15.160:0/1158109503 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fcda00ce2c0 con 0x7fcdb0100590 2024-03-07T07:40:37.070 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.068+0000 7fcdaffff640 1 --2- 172.21.15.160:0/1158109503 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fcd980a1e10 0x7fcd980a42d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcdb006a590 tx=0x7fcda4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:37.072 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.072+0000 7fcdadffb640 1 -- 172.21.15.160:0/1158109503 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fcda009b590 con 0x7fcdb0100590 2024-03-07T07:40:37.250 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.250+0000 7fcdb6e87640 1 -- 172.21.15.160:0/1158109503 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcdb002d250 con 0x7fcdb0100590 2024-03-07T07:40:37.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.251+0000 7fcdadffb640 1 -- 172.21.15.160:0/1158109503 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22721 (secure 0 0 0) 0x7fcda009ae90 con 0x7fcdb0100590 2024-03-07T07:40:37.252 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:40:37.252 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":87,"fsid":"826f6582-dc54-11ee-95c6-87774f69a715","created":"2024-03-07T07:31:14.229288+0000","modified":"2024-03-07T07:40:09.354265+0000","last_up_change":"2024-03-07T07:38:20.776225+0000","last_in_change":"2024-03-07T07:38:00.508917+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T07:34:34.529424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T07:39:12.075432+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T07:39:13.367820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bee66c67-fb6e-4717-9685-ca62587d5f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6803","nonce":1368482926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6805","nonce":1368482926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6809","nonce":1368482926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6807","nonce":1368482926}]},"public_addr":"172.21.15.160:6803/1368482926","cluster_addr":"172.21.15.160:6805/1368482926","heartbeat_back_addr":"172.21.15.160:6809/1368482926","heartbeat_front_addr":"172.21.15.160:6807/1368482926","state":["exists","up"]},{"osd":1,"uuid":"49236251-f750-424e-992b-bddf55762556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6801","nonce":2922391884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6803","nonce":2922391884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6807","nonce":2922391884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6805","nonce":2922391884}]},"public_addr":"172.21.15.177:6801/2922391884","cluster_addr":"172.21.15.177:6803/2922391884","heartbeat_back_addr":"172.21.15.177:6807/2922391884","heartbeat_front_addr":"172.21.15.177:6805/2922391884","state":["exists","up"]},{"osd":2,"uuid":"7a432b28-f7ee-43fe-9196-7aaeccfecf77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6801","nonce":1217901551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6803","nonce":1217901551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6807","nonce":1217901551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6805","nonce":1217901551}]},"public_addr":"172.21.15.178:6801/1217901551","cluster_addr":"172.21.15.178:6803/1217901551","heartbeat_back_addr":"172.21.15.178:6807/1217901551","heartbeat_front_addr":"172.21.15.178:6805/1217901551","state":["exists","up"]},{"osd":3,"uuid":"4ad22a20-f1be-4002-a48d-ecd3953e63f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6811","nonce":42543989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6813","nonce":42543989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6817","nonce":42543989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6815","nonce":42543989}]},"public_addr":"172.21.15.160:6811/42543989","cluster_addr":"172.21.15.160:6813/42543989","heartbeat_back_addr":"172.21.15.160:6817/42543989","heartbeat_front_addr":"172.21.15.160:6815/42543989","state":["exists","up"]},{"osd":4,"uuid":"2b65b104-ac82-4ce3-b6c5-11a399103045","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6809","nonce":88178553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6811","nonce":88178553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6815","nonce":88178553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6813","nonce":88178553}]},"public_addr":"172.21.15.177:6809/88178553","cluster_addr":"172.21.15.177:6811/88178553","heartbeat_back_addr":"172.21.15.177:6815/88178553","heartbeat_front_addr":"172.21.15.177:6813/88178553","state":["exists","up"]},{"osd":5,"uuid":"30a1e714-08ff-44e8-9264-9ef164604b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6809","nonce":964732891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6811","nonce":964732891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6815","nonce":964732891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6813","nonce":964732891}]},"public_addr":"172.21.15.178:6809/964732891","cluster_addr":"172.21.15.178:6811/964732891","heartbeat_back_addr":"172.21.15.178:6815/964732891","heartbeat_front_addr":"172.21.15.178:6813/964732891","state":["exists","up"]},{"osd":6,"uuid":"b1e1646e-f8a9-42b8-b62d-9d7b5d6a1ff4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6819","nonce":3251327001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6821","nonce":3251327001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6825","nonce":3251327001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6823","nonce":3251327001}]},"public_addr":"172.21.15.160:6819/3251327001","cluster_addr":"172.21.15.160:6821/3251327001","heartbeat_back_addr":"172.21.15.160:6825/3251327001","heartbeat_front_addr":"172.21.15.160:6823/3251327001","state":["exists","up"]},{"osd":7,"uuid":"e66b400b-9747-4f6d-b05b-db22468e2750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6817","nonce":1175500850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6819","nonce":1175500850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6823","nonce":1175500850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6821","nonce":1175500850}]},"public_addr":"172.21.15.177:6817/1175500850","cluster_addr":"172.21.15.177:6819/1175500850","heartbeat_back_addr":"172.21.15.177:6823/1175500850","heartbeat_front_addr":"172.21.15.177:6821/1175500850","state":["exists","up"]},{"osd":8,"uuid":"3ee0f2c4-fca4-49fc-a98b-07aa0fcba0e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6817","nonce":688015531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6819","nonce":688015531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6823","nonce":688015531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6821","nonce":688015531}]},"public_addr":"172.21.15.178:6817/688015531","cluster_addr":"172.21.15.178:6819/688015531","heartbeat_back_addr":"172.21.15.178:6823/688015531","heartbeat_front_addr":"172.21.15.178:6821/688015531","state":["exists","up"]},{"osd":9,"uuid":"856039d7-f7ee-4dbd-b019-0073e705ce0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6827","nonce":3265797851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6829","nonce":3265797851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6833","nonce":3265797851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6831","nonce":3265797851}]},"public_addr":"172.21.15.160:6827/3265797851","cluster_addr":"172.21.15.160:6829/3265797851","heartbeat_back_addr":"172.21.15.160:6833/3265797851","heartbeat_front_addr":"172.21.15.160:6831/3265797851","state":["exists","up"]},{"osd":10,"uuid":"bdc9899d-d720-4429-ad53-4c388a4dbe75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6825","nonce":2867473412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6827","nonce":2867473412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6831","nonce":2867473412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6829","nonce":2867473412}]},"public_addr":"172.21.15.177:6825/2867473412","cluster_addr":"172.21.15.177:6827/2867473412","heartbeat_back_addr":"172.21.15.177:6831/2867473412","heartbeat_front_addr":"172.21.15.177:6829/2867473412","state":["exists","up"]},{"osd":11,"uuid":"cc9adb54-ef49-47c6-90e3-fc03bc0f5cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6825","nonce":4068724731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6827","nonce":4068724731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6831","nonce":4068724731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6829","nonce":4068724731}]},"public_addr":"172.21.15.178:6825/4068724731","cluster_addr":"172.21.15.178:6827/4068724731","heartbeat_back_addr":"172.21.15.178:6831/4068724731","heartbeat_front_addr":"172.21.15.178:6829/4068724731","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:33:44.141585+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:07.685585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:31.044386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:56.308925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:35:21.582097+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:35:45.983540+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:36:11.530242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:36:36.844681+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:02.303437+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:27.629867+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:53.289495+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:38:18.536169+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/3027450612":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/874709722":"2024-03-08T07:32:08.190837+0000","172.21.15.160:6800/1130675225":"2024-03-08T07:31:41.166858+0000","172.21.15.160:0/3857193980":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/3861968297":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6800/3859174315":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/3013534743":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6801/3859174315":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/1310937667":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6801/1130675225":"2024-03-08T07:31:41.166858+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T07:40:37.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.253+0000 7fcdb6e87640 1 -- 172.21.15.160:0/1158109503 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fcd980a1e10 msgr2=0x7fcd980a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:37.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.253+0000 7fcdb6e87640 1 --2- 172.21.15.160:0/1158109503 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fcd980a1e10 0x7fcd980a42d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcdb006a590 tx=0x7fcda4005f90 comp rx=0 tx=0).stop 2024-03-07T07:40:37.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.253+0000 7fcdb6e87640 1 -- 172.21.15.160:0/1158109503 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdb0100590 msgr2=0x7fcdb006a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:37.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.253+0000 7fcdb6e87640 1 --2- 172.21.15.160:0/1158109503 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdb0100590 0x7fcdb006a820 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fcda000e9c0 tx=0x7fcda000ee80 comp rx=0 tx=0).stop 2024-03-07T07:40:37.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.253+0000 7fcdb6e87640 1 -- 172.21.15.160:0/1158109503 shutdown_connections 2024-03-07T07:40:37.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.253+0000 7fcdb6e87640 1 --2- 172.21.15.160:0/1158109503 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fcd980a1e10 0x7fcd980a42d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:37.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.253+0000 7fcdb6e87640 1 --2- 172.21.15.160:0/1158109503 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcdb0102140 0x7fcdb019ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:37.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.253+0000 7fcdb6e87640 1 --2- 172.21.15.160:0/1158109503 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcdb0101780 0x7fcdb019c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:37.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.253+0000 7fcdb6e87640 1 --2- 172.21.15.160:0/1158109503 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdb0100590 0x7fcdb006a820 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:37.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.253+0000 7fcdb6e87640 1 -- 172.21.15.160:0/1158109503 >> 172.21.15.160:0/1158109503 conn(0x7fcdb00fbb40 msgr2=0x7fcdb0104a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:37.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.254+0000 7fcdb6e87640 1 -- 172.21.15.160:0/1158109503 shutdown_connections 2024-03-07T07:40:37.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.254+0000 7fcdb6e87640 1 -- 172.21.15.160:0/1158109503 wait complete. 2024-03-07T07:40:37.266 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-07T07:34:34.529424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-07T07:39:12.075432+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-07T07:39:13.367820+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-07T07:40:37.266 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-07T07:40:37.374 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.371+0000 7f5f6bb1d640 1 -- 172.21.15.160:0/1907698597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5f64102100 msgr2=0x7f5f641044f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:37.374 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.371+0000 7f5f6bb1d640 1 --2- 172.21.15.160:0/1907698597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5f64102100 0x7f5f641044f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5f60009900 tx=0x7f5f6002bc30 comp rx=0 tx=0).stop 2024-03-07T07:40:37.374 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.371+0000 7f5f6bb1d640 1 -- 172.21.15.160:0/1907698597 shutdown_connections 2024-03-07T07:40:37.374 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.371+0000 7f5f6bb1d640 1 --2- 172.21.15.160:0/1907698597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f64107360 0x7f5f641097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:37.374 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.371+0000 7f5f6bb1d640 1 --2- 172.21.15.160:0/1907698597 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5f64104a30 0x7f5f64106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:37.374 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.371+0000 7f5f6bb1d640 1 --2- 172.21.15.160:0/1907698597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5f64102100 0x7f5f641044f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:37.374 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.371+0000 7f5f6bb1d640 1 -- 172.21.15.160:0/1907698597 >> 172.21.15.160:0/1907698597 conn(0x7f5f640fbb10 msgr2=0x7f5f640fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:37.374 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.371+0000 7f5f6bb1d640 1 -- 172.21.15.160:0/1907698597 shutdown_connections 2024-03-07T07:40:37.374 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.371+0000 7f5f6bb1d640 1 -- 172.21.15.160:0/1907698597 wait complete. 2024-03-07T07:40:37.374 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.372+0000 7f5f6bb1d640 1 Processor -- start 2024-03-07T07:40:37.374 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.372+0000 7f5f6bb1d640 1 -- start start 2024-03-07T07:40:37.375 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.372+0000 7f5f6bb1d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5f64102100 0x7f5f641979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:37.375 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.372+0000 7f5f6bb1d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f64104a30 0x7f5f64197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:37.375 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.372+0000 7f5f6bb1d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5f64107360 0x7f5f6419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:37.375 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.372+0000 7f5f6bb1d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6410d300 con 0x7f5f64102100 2024-03-07T07:40:37.375 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.372+0000 7f5f6bb1d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6410d180 con 0x7f5f64107360 2024-03-07T07:40:37.375 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.372+0000 7f5f6bb1d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6410d480 con 0x7f5f64104a30 2024-03-07T07:40:37.375 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.372+0000 7f5f698b9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5f64102100 0x7f5f641979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:37.375 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.372+0000 7f5f698b9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5f64102100 0x7f5f641979e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:38336/0 (socket says 172.21.15.160:38336) 2024-03-07T07:40:37.375 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.372+0000 7f5f698b9640 1 -- 172.21.15.160:0/1129548933 learned_addr learned my addr 172.21.15.160:0/1129548933 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:40:37.375 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.372+0000 7f5f6a0ba640 1 --2- 172.21.15.160:0/1129548933 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5f64107360 0x7f5f6419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:37.375 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.372+0000 7f5f690b8640 1 --2- 172.21.15.160:0/1129548933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f64104a30 0x7f5f64197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:37.375 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.372+0000 7f5f698b9640 1 -- 172.21.15.160:0/1129548933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f64104a30 msgr2=0x7f5f64197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:37.376 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.372+0000 7f5f698b9640 1 --2- 172.21.15.160:0/1129548933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f64104a30 0x7f5f64197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:37.376 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.372+0000 7f5f698b9640 1 -- 172.21.15.160:0/1129548933 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5f64107360 msgr2=0x7f5f6419ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:37.376 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.372+0000 7f5f698b9640 1 --2- 172.21.15.160:0/1129548933 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5f64107360 0x7f5f6419ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:37.376 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.372+0000 7f5f698b9640 1 -- 172.21.15.160:0/1129548933 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f6419f600 con 0x7f5f64102100 2024-03-07T07:40:37.376 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.372+0000 7f5f698b9640 1 --2- 172.21.15.160:0/1129548933 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5f64102100 0x7f5f641979e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5f60009230 tx=0x7f5f6002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:37.376 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.372+0000 7f5f5affd640 1 -- 172.21.15.160:0/1129548933 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f60041070 con 0x7f5f64102100 2024-03-07T07:40:37.376 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.372+0000 7f5f6bb1d640 1 -- 172.21.15.160:0/1129548933 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f6419f890 con 0x7f5f64102100 2024-03-07T07:40:37.376 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.372+0000 7f5f5affd640 1 -- 172.21.15.160:0/1129548933 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5f60004450 con 0x7f5f64102100 2024-03-07T07:40:37.376 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.372+0000 7f5f5affd640 1 -- 172.21.15.160:0/1129548933 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f60003e20 con 0x7f5f64102100 2024-03-07T07:40:37.376 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.373+0000 7f5f6bb1d640 1 -- 172.21.15.160:0/1129548933 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f6419fd80 con 0x7f5f64102100 2024-03-07T07:40:37.376 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.373+0000 7f5f6bb1d640 1 -- 172.21.15.160:0/1129548933 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f640fd600 con 0x7f5f64102100 2024-03-07T07:40:37.377 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.374+0000 7f5f5affd640 1 -- 172.21.15.160:0/1129548933 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) v1 ==== 136805+0+0 (secure 0 0 0) 0x7f5f60044070 con 0x7f5f64102100 2024-03-07T07:40:37.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.375+0000 7f5f5affd640 1 --2- 172.21.15.160:0/1129548933 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f5f500a1e10 0x7f5f500a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:37.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.375+0000 7f5f690b8640 1 --2- 172.21.15.160:0/1129548933 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f5f500a1e10 0x7f5f500a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:37.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.375+0000 7f5f5affd640 1 -- 172.21.15.160:0/1129548933 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5f600ef9a0 con 0x7f5f64102100 2024-03-07T07:40:37.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.375+0000 7f5f690b8640 1 --2- 172.21.15.160:0/1129548933 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f5f500a1e10 0x7f5f500a42d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5f64198ef0 tx=0x7f5f54005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:37.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.379+0000 7f5f5affd640 1 -- 172.21.15.160:0/1129548933 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5f600bc730 con 0x7f5f64102100 2024-03-07T07:40:37.553 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.553+0000 7f5f6bb1d640 1 -- 172.21.15.160:0/1129548933 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5f6402d250 con 0x7f5f64102100 2024-03-07T07:40:37.554 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.554+0000 7f5f5affd640 1 -- 172.21.15.160:0/1129548933 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5f600c07d0 con 0x7f5f64102100 2024-03-07T07:40:37.554 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 1 2024-03-07T07:40:37.556 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.556+0000 7f5f6bb1d640 1 -- 172.21.15.160:0/1129548933 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f5f500a1e10 msgr2=0x7f5f500a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:37.556 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.556+0000 7f5f6bb1d640 1 --2- 172.21.15.160:0/1129548933 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f5f500a1e10 0x7f5f500a42d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5f64198ef0 tx=0x7f5f54005f90 comp rx=0 tx=0).stop 2024-03-07T07:40:37.556 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.556+0000 7f5f6bb1d640 1 -- 172.21.15.160:0/1129548933 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5f64102100 msgr2=0x7f5f641979e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:37.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.556+0000 7f5f6bb1d640 1 --2- 172.21.15.160:0/1129548933 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5f64102100 0x7f5f641979e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5f60009230 tx=0x7f5f6002c650 comp rx=0 tx=0).stop 2024-03-07T07:40:37.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.556+0000 7f5f6bb1d640 1 -- 172.21.15.160:0/1129548933 shutdown_connections 2024-03-07T07:40:37.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.556+0000 7f5f6bb1d640 1 --2- 172.21.15.160:0/1129548933 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f5f500a1e10 0x7f5f500a42d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:37.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.556+0000 7f5f6bb1d640 1 --2- 172.21.15.160:0/1129548933 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5f64107360 0x7f5f6419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:37.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.556+0000 7f5f6bb1d640 1 --2- 172.21.15.160:0/1129548933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f64104a30 0x7f5f64197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:37.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.556+0000 7f5f6bb1d640 1 --2- 172.21.15.160:0/1129548933 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5f64102100 0x7f5f641979e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:37.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.556+0000 7f5f6bb1d640 1 -- 172.21.15.160:0/1129548933 >> 172.21.15.160:0/1129548933 conn(0x7f5f640fbb10 msgr2=0x7f5f64102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:37.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.556+0000 7f5f6bb1d640 1 -- 172.21.15.160:0/1129548933 shutdown_connections 2024-03-07T07:40:37.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.556+0000 7f5f6bb1d640 1 -- 172.21.15.160:0/1129548933 wait complete. 2024-03-07T07:40:37.566 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:37 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/1158109503' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:40:37.568 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-07T07:40:37.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.673+0000 7f40c18dd640 1 -- 172.21.15.160:0/2197649401 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40bc100580 msgr2=0x7f40bc1009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:37.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.673+0000 7f40c18dd640 1 --2- 172.21.15.160:0/2197649401 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40bc100580 0x7f40bc1009a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f40ac009900 tx=0x7f40ac02bc30 comp rx=0 tx=0).stop 2024-03-07T07:40:37.675 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.673+0000 7f40c18dd640 1 -- 172.21.15.160:0/2197649401 shutdown_connections 2024-03-07T07:40:37.676 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.673+0000 7f40c18dd640 1 --2- 172.21.15.160:0/2197649401 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40bc102130 0x7f40bc1089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:37.676 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.673+0000 7f40c18dd640 1 --2- 172.21.15.160:0/2197649401 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40bc101770 0x7f40bc101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:37.676 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.673+0000 7f40c18dd640 1 --2- 172.21.15.160:0/2197649401 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40bc100580 0x7f40bc1009a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:37.676 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.673+0000 7f40c18dd640 1 -- 172.21.15.160:0/2197649401 >> 172.21.15.160:0/2197649401 conn(0x7f40bc0fbb10 msgr2=0x7f40bc0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:37.676 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.673+0000 7f40c18dd640 1 -- 172.21.15.160:0/2197649401 shutdown_connections 2024-03-07T07:40:37.676 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.673+0000 7f40c18dd640 1 -- 172.21.15.160:0/2197649401 wait complete. 2024-03-07T07:40:37.676 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.673+0000 7f40c18dd640 1 Processor -- start 2024-03-07T07:40:37.676 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.673+0000 7f40c18dd640 1 -- start start 2024-03-07T07:40:37.676 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.673+0000 7f40c18dd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40bc100580 0x7f40bc06a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:37.676 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.673+0000 7f40c18dd640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40bc101770 0x7f40bc19c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:37.676 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.673+0000 7f40c18dd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40bc102130 0x7f40bc19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:37.676 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.673+0000 7f40c18dd640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc10c210 con 0x7f40bc100580 2024-03-07T07:40:37.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.673+0000 7f40c18dd640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc10c090 con 0x7f40bc101770 2024-03-07T07:40:37.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.673+0000 7f40c18dd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc10c390 con 0x7f40bc102130 2024-03-07T07:40:37.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.673+0000 7f40baffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40bc100580 0x7f40bc06a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:37.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.673+0000 7f40baffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40bc100580 0x7f40bc06a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:38350/0 (socket says 172.21.15.160:38350) 2024-03-07T07:40:37.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.673+0000 7f40baffd640 1 -- 172.21.15.160:0/3488891330 learned_addr learned my addr 172.21.15.160:0/3488891330 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:40:37.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.673+0000 7f40b3fff640 1 --2- 172.21.15.160:0/3488891330 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40bc101770 0x7f40bc19c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:37.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.674+0000 7f40bb7fe640 1 --2- 172.21.15.160:0/3488891330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40bc102130 0x7f40bc19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:37.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.674+0000 7f40baffd640 1 -- 172.21.15.160:0/3488891330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40bc102130 msgr2=0x7f40bc19ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:37.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.674+0000 7f40baffd640 1 --2- 172.21.15.160:0/3488891330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40bc102130 0x7f40bc19ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:37.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.674+0000 7f40baffd640 1 -- 172.21.15.160:0/3488891330 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40bc101770 msgr2=0x7f40bc19c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:37.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.674+0000 7f40baffd640 1 --2- 172.21.15.160:0/3488891330 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40bc101770 0x7f40bc19c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:37.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.674+0000 7f40baffd640 1 -- 172.21.15.160:0/3488891330 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40bc19f640 con 0x7f40bc100580 2024-03-07T07:40:37.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.674+0000 7f40b3fff640 1 --2- 172.21.15.160:0/3488891330 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40bc101770 0x7f40bc19c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:40:37.677 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.674+0000 7f40baffd640 1 --2- 172.21.15.160:0/3488891330 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40bc100580 0x7f40bc06a780 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f40ac000c00 tx=0x7f40ac02c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:37.678 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.674+0000 7f40c08db640 1 -- 172.21.15.160:0/3488891330 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ac041070 con 0x7f40bc100580 2024-03-07T07:40:37.678 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.674+0000 7f40c08db640 1 -- 172.21.15.160:0/3488891330 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f40ac004450 con 0x7f40bc100580 2024-03-07T07:40:37.678 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.674+0000 7f40c18dd640 1 -- 172.21.15.160:0/3488891330 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40bc19f870 con 0x7f40bc100580 2024-03-07T07:40:37.678 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.674+0000 7f40c08db640 1 -- 172.21.15.160:0/3488891330 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ac034440 con 0x7f40bc100580 2024-03-07T07:40:37.678 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.674+0000 7f40c18dd640 1 -- 172.21.15.160:0/3488891330 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40bc19fd60 con 0x7f40bc100580 2024-03-07T07:40:37.678 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.675+0000 7f40c18dd640 1 -- 172.21.15.160:0/3488891330 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40bc107920 con 0x7f40bc100580 2024-03-07T07:40:37.679 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.676+0000 7f40c08db640 1 -- 172.21.15.160:0/3488891330 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) v1 ==== 136805+0+0 (secure 0 0 0) 0x7f40ac044070 con 0x7f40bc100580 2024-03-07T07:40:37.679 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.677+0000 7f40c08db640 1 --2- 172.21.15.160:0/3488891330 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f40a80a1e10 0x7f40a80a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:37.679 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.677+0000 7f40b3fff640 1 --2- 172.21.15.160:0/3488891330 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f40a80a1e10 0x7f40a80a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:37.679 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.677+0000 7f40c08db640 1 -- 172.21.15.160:0/3488891330 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f40ac0ef8e0 con 0x7f40bc100580 2024-03-07T07:40:37.679 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.677+0000 7f40b3fff640 1 --2- 172.21.15.160:0/3488891330 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f40a80a1e10 0x7f40a80a42d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f40bc06a4f0 tx=0x7f40a4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:37.681 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.681+0000 7f40c08db640 1 -- 172.21.15.160:0/3488891330 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f40ac0bc700 con 0x7f40bc100580 2024-03-07T07:40:37.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:37 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/1158109503' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:40:37.856 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.855+0000 7f40c18dd640 1 -- 172.21.15.160:0/3488891330 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f40bc02d030 con 0x7f40bc100580 2024-03-07T07:40:37.856 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.856+0000 7f40c08db640 1 -- 172.21.15.160:0/3488891330 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f40ac0c07a0 con 0x7f40bc100580 2024-03-07T07:40:37.856 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 32 2024-03-07T07:40:37.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:37 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/1158109503' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:40:37.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.858+0000 7f40c18dd640 1 -- 172.21.15.160:0/3488891330 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f40a80a1e10 msgr2=0x7f40a80a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:37.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.858+0000 7f40c18dd640 1 --2- 172.21.15.160:0/3488891330 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f40a80a1e10 0x7f40a80a42d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f40bc06a4f0 tx=0x7f40a4005f90 comp rx=0 tx=0).stop 2024-03-07T07:40:37.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.858+0000 7f40c18dd640 1 -- 172.21.15.160:0/3488891330 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40bc100580 msgr2=0x7f40bc06a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:37.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.858+0000 7f40c18dd640 1 --2- 172.21.15.160:0/3488891330 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40bc100580 0x7f40bc06a780 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f40ac000c00 tx=0x7f40ac02c640 comp rx=0 tx=0).stop 2024-03-07T07:40:37.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.858+0000 7f40c18dd640 1 -- 172.21.15.160:0/3488891330 shutdown_connections 2024-03-07T07:40:37.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.858+0000 7f40c18dd640 1 --2- 172.21.15.160:0/3488891330 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f40a80a1e10 0x7f40a80a42d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:37.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.858+0000 7f40c18dd640 1 --2- 172.21.15.160:0/3488891330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40bc102130 0x7f40bc19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:37.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.858+0000 7f40c18dd640 1 --2- 172.21.15.160:0/3488891330 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40bc101770 0x7f40bc19c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:37.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.858+0000 7f40c18dd640 1 --2- 172.21.15.160:0/3488891330 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40bc100580 0x7f40bc06a780 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:37.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.858+0000 7f40c18dd640 1 -- 172.21.15.160:0/3488891330 >> 172.21.15.160:0/3488891330 conn(0x7f40bc0fbb10 msgr2=0x7f40bc104af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:37.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.858+0000 7f40c18dd640 1 -- 172.21.15.160:0/3488891330 shutdown_connections 2024-03-07T07:40:37.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.858+0000 7f40c18dd640 1 -- 172.21.15.160:0/3488891330 wait complete. 2024-03-07T07:40:37.869 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-07T07:40:37.976 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.974+0000 7f7c2955a640 1 -- 172.21.15.160:0/252027047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c24102110 msgr2=0x7f7c241089a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:37.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.974+0000 7f7c2955a640 1 --2- 172.21.15.160:0/252027047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c24102110 0x7f7c241089a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c009ca0 tx=0x7f7c1c02c0f0 comp rx=0 tx=0).stop 2024-03-07T07:40:37.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.974+0000 7f7c2955a640 1 -- 172.21.15.160:0/252027047 shutdown_connections 2024-03-07T07:40:37.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.974+0000 7f7c2955a640 1 --2- 172.21.15.160:0/252027047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c24102110 0x7f7c241089a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:37.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.974+0000 7f7c2955a640 1 --2- 172.21.15.160:0/252027047 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c24101750 0x7f7c24101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:37.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.974+0000 7f7c2955a640 1 --2- 172.21.15.160:0/252027047 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c24100560 0x7f7c24100980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:37.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.974+0000 7f7c2955a640 1 -- 172.21.15.160:0/252027047 >> 172.21.15.160:0/252027047 conn(0x7f7c240fbaf0 msgr2=0x7f7c240fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:37.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.974+0000 7f7c2955a640 1 -- 172.21.15.160:0/252027047 shutdown_connections 2024-03-07T07:40:37.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.974+0000 7f7c2955a640 1 -- 172.21.15.160:0/252027047 wait complete. 2024-03-07T07:40:37.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.974+0000 7f7c2955a640 1 Processor -- start 2024-03-07T07:40:37.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.974+0000 7f7c2955a640 1 -- start start 2024-03-07T07:40:37.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.975+0000 7f7c2955a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c24100560 0x7f7c24196840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:37.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.975+0000 7f7c2955a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c24101750 0x7f7c24196d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:37.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.975+0000 7f7c2955a640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c24102110 0x7f7c2419ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:37.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.975+0000 7f7c2955a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2410c1f0 con 0x7f7c24101750 2024-03-07T07:40:37.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.975+0000 7f7c2955a640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2410c070 con 0x7f7c24102110 2024-03-07T07:40:37.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.975+0000 7f7c2955a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2410c370 con 0x7f7c24100560 2024-03-07T07:40:37.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.975+0000 7f7c227fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c24101750 0x7f7c24196d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:37.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.975+0000 7f7c22ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c24100560 0x7f7c24196840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:37.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.975+0000 7f7c22ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c24100560 0x7f7c24196840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.160:49622/0 (socket says 172.21.15.160:49622) 2024-03-07T07:40:37.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.975+0000 7f7c227fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c24101750 0x7f7c24196d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:38368/0 (socket says 172.21.15.160:38368) 2024-03-07T07:40:37.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.975+0000 7f7c22ffd640 1 -- 172.21.15.160:0/2745308121 learned_addr learned my addr 172.21.15.160:0/2745308121 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:40:37.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.975+0000 7f7c237fe640 1 --2- 172.21.15.160:0/2745308121 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c24102110 0x7f7c2419ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:37.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.975+0000 7f7c22ffd640 1 -- 172.21.15.160:0/2745308121 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c24102110 msgr2=0x7f7c2419ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:37.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.975+0000 7f7c22ffd640 1 --2- 172.21.15.160:0/2745308121 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c24102110 0x7f7c2419ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:37.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.975+0000 7f7c22ffd640 1 -- 172.21.15.160:0/2745308121 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c24101750 msgr2=0x7f7c24196d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:37.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.975+0000 7f7c22ffd640 1 --2- 172.21.15.160:0/2745308121 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c24101750 0x7f7c24196d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:37.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.975+0000 7f7c22ffd640 1 -- 172.21.15.160:0/2745308121 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c2419e460 con 0x7f7c24100560 2024-03-07T07:40:37.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.975+0000 7f7c227fc640 1 --2- 172.21.15.160:0/2745308121 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c24101750 0x7f7c24196d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:40:37.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.975+0000 7f7c22ffd640 1 --2- 172.21.15.160:0/2745308121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c24100560 0x7f7c24196840 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7c1800e9c0 tx=0x7f7c1800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:37.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.975+0000 7f7c0bfff640 1 -- 172.21.15.160:0/2745308121 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1800cd80 con 0x7f7c24100560 2024-03-07T07:40:37.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.975+0000 7f7c2955a640 1 -- 172.21.15.160:0/2745308121 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c2419e750 con 0x7f7c24100560 2024-03-07T07:40:37.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.975+0000 7f7c0bfff640 1 -- 172.21.15.160:0/2745308121 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c180035f0 con 0x7f7c24100560 2024-03-07T07:40:37.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.975+0000 7f7c0bfff640 1 -- 172.21.15.160:0/2745308121 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c18010690 con 0x7f7c24100560 2024-03-07T07:40:37.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.975+0000 7f7c2955a640 1 -- 172.21.15.160:0/2745308121 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c2419ecf0 con 0x7f7c24100560 2024-03-07T07:40:37.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.976+0000 7f7c2955a640 1 -- 172.21.15.160:0/2745308121 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c24107880 con 0x7f7c24100560 2024-03-07T07:40:37.980 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.977+0000 7f7c0bfff640 1 -- 172.21.15.160:0/2745308121 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) v1 ==== 136805+0+0 (secure 0 0 0) 0x7f7c180037a0 con 0x7f7c24100560 2024-03-07T07:40:37.980 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.978+0000 7f7c0bfff640 1 --2- 172.21.15.160:0/2745308121 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f7c0c0a1e10 0x7f7c0c0a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:37.980 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.978+0000 7f7c227fc640 1 --2- 172.21.15.160:0/2745308121 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f7c0c0a1e10 0x7f7c0c0a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:37.980 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.978+0000 7f7c0bfff640 1 -- 172.21.15.160:0/2745308121 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7c180ce2d0 con 0x7f7c24100560 2024-03-07T07:40:37.980 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.978+0000 7f7c227fc640 1 --2- 172.21.15.160:0/2745308121 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f7c0c0a1e10 0x7f7c0c0a42d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7c24197d50 tx=0x7f7c14005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:37.981 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:37.981+0000 7f7c0bfff640 1 -- 172.21.15.160:0/2745308121 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7c1809b5a0 con 0x7f7c24100560 2024-03-07T07:40:38.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.154+0000 7f7c2955a640 1 -- 172.21.15.160:0/2745308121 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7c24062260 con 0x7f7c24100560 2024-03-07T07:40:38.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.155+0000 7f7c0bfff640 1 -- 172.21.15.160:0/2745308121 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7c1809aea0 con 0x7f7c24100560 2024-03-07T07:40:38.155 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 64 2024-03-07T07:40:38.157 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.157+0000 7f7c2955a640 1 -- 172.21.15.160:0/2745308121 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f7c0c0a1e10 msgr2=0x7f7c0c0a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:38.157 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.157+0000 7f7c2955a640 1 --2- 172.21.15.160:0/2745308121 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f7c0c0a1e10 0x7f7c0c0a42d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7c24197d50 tx=0x7f7c14005f90 comp rx=0 tx=0).stop 2024-03-07T07:40:38.157 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.157+0000 7f7c2955a640 1 -- 172.21.15.160:0/2745308121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c24100560 msgr2=0x7f7c24196840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:38.157 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.157+0000 7f7c2955a640 1 --2- 172.21.15.160:0/2745308121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c24100560 0x7f7c24196840 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7c1800e9c0 tx=0x7f7c1800ee80 comp rx=0 tx=0).stop 2024-03-07T07:40:38.157 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.157+0000 7f7c2955a640 1 -- 172.21.15.160:0/2745308121 shutdown_connections 2024-03-07T07:40:38.158 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.157+0000 7f7c2955a640 1 --2- 172.21.15.160:0/2745308121 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f7c0c0a1e10 0x7f7c0c0a42d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:38.158 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.157+0000 7f7c2955a640 1 --2- 172.21.15.160:0/2745308121 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c24102110 0x7f7c2419ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:38.158 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.157+0000 7f7c2955a640 1 --2- 172.21.15.160:0/2745308121 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c24101750 0x7f7c24196d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:38.158 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.157+0000 7f7c2955a640 1 --2- 172.21.15.160:0/2745308121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c24100560 0x7f7c24196840 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:38.158 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.157+0000 7f7c2955a640 1 -- 172.21.15.160:0/2745308121 >> 172.21.15.160:0/2745308121 conn(0x7f7c240fbaf0 msgr2=0x7f7c24104a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:38.158 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.157+0000 7f7c2955a640 1 -- 172.21.15.160:0/2745308121 shutdown_connections 2024-03-07T07:40:38.158 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.157+0000 7f7c2955a640 1 -- 172.21.15.160:0/2745308121 wait complete. 2024-03-07T07:40:38.168 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T07:40:38.271 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.270+0000 7fa140bc9640 1 -- 172.21.15.160:0/1254821610 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa13c101770 msgr2=0x7fa13c101bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:38.271 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.270+0000 7fa140bc9640 1 --2- 172.21.15.160:0/1254821610 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa13c101770 0x7fa13c101bf0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa130009900 tx=0x7fa13002bc30 comp rx=0 tx=0).stop 2024-03-07T07:40:38.271 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.271+0000 7fa140bc9640 1 -- 172.21.15.160:0/1254821610 shutdown_connections 2024-03-07T07:40:38.272 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.271+0000 7fa140bc9640 1 --2- 172.21.15.160:0/1254821610 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa13c102130 0x7fa13c1089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:38.272 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.271+0000 7fa140bc9640 1 --2- 172.21.15.160:0/1254821610 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa13c101770 0x7fa13c101bf0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:38.272 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.271+0000 7fa140bc9640 1 --2- 172.21.15.160:0/1254821610 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa13c100580 0x7fa13c1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:38.272 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.271+0000 7fa140bc9640 1 -- 172.21.15.160:0/1254821610 >> 172.21.15.160:0/1254821610 conn(0x7fa13c0fbaf0 msgr2=0x7fa13c0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:38.272 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.272+0000 7fa140bc9640 1 -- 172.21.15.160:0/1254821610 shutdown_connections 2024-03-07T07:40:38.272 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.272+0000 7fa140bc9640 1 -- 172.21.15.160:0/1254821610 wait complete. 2024-03-07T07:40:38.275 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.273+0000 7fa140bc9640 1 Processor -- start 2024-03-07T07:40:38.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.273+0000 7fa140bc9640 1 -- start start 2024-03-07T07:40:38.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.274+0000 7fa140bc9640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa13c100580 0x7fa13c19e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:38.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.274+0000 7fa140bc9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa13c101770 0x7fa13c19ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:38.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.274+0000 7fa140bc9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa13c102130 0x7fa13c1a5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:38.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.274+0000 7fa140bc9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c10b100 con 0x7fa13c101770 2024-03-07T07:40:38.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.274+0000 7fa140bc9640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c10af80 con 0x7fa13c100580 2024-03-07T07:40:38.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.274+0000 7fa140bc9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c10b280 con 0x7fa13c102130 2024-03-07T07:40:38.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.274+0000 7fa139d9b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa13c101770 0x7fa13c19ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:38.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.274+0000 7fa139d9b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa13c101770 0x7fa13c19ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:38392/0 (socket says 172.21.15.160:38392) 2024-03-07T07:40:38.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.274+0000 7fa139d9b640 1 -- 172.21.15.160:0/3413741858 learned_addr learned my addr 172.21.15.160:0/3413741858 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:40:38.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.275+0000 7fa13a59c640 1 --2- 172.21.15.160:0/3413741858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa13c100580 0x7fa13c19e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:38.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.275+0000 7fa13ad9d640 1 --2- 172.21.15.160:0/3413741858 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa13c102130 0x7fa13c1a5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:38.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.276+0000 7fa139d9b640 1 -- 172.21.15.160:0/3413741858 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa13c102130 msgr2=0x7fa13c1a5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:38.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.276+0000 7fa139d9b640 1 --2- 172.21.15.160:0/3413741858 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa13c102130 0x7fa13c1a5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:38.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.276+0000 7fa139d9b640 1 -- 172.21.15.160:0/3413741858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa13c100580 msgr2=0x7fa13c19e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:38.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.276+0000 7fa139d9b640 1 --2- 172.21.15.160:0/3413741858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa13c100580 0x7fa13c19e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:38.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.276+0000 7fa139d9b640 1 -- 172.21.15.160:0/3413741858 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa13c1a5c80 con 0x7fa13c101770 2024-03-07T07:40:38.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.276+0000 7fa13a59c640 1 --2- 172.21.15.160:0/3413741858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa13c100580 0x7fa13c19e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:40:38.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.278+0000 7fa139d9b640 1 --2- 172.21.15.160:0/3413741858 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa13c101770 0x7fa13c19ebb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa130009230 tx=0x7fa130005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:38.282 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.282+0000 7fa12b7fe640 1 -- 172.21.15.160:0/3413741858 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa130041070 con 0x7fa13c101770 2024-03-07T07:40:38.282 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.282+0000 7fa12b7fe640 1 -- 172.21.15.160:0/3413741858 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa1300044d0 con 0x7fa13c101770 2024-03-07T07:40:38.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.283+0000 7fa12b7fe640 1 -- 172.21.15.160:0/3413741858 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa130034440 con 0x7fa13c101770 2024-03-07T07:40:38.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.284+0000 7fa140bc9640 1 -- 172.21.15.160:0/3413741858 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa13c1a5f10 con 0x7fa13c101770 2024-03-07T07:40:38.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.284+0000 7fa140bc9640 1 -- 172.21.15.160:0/3413741858 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa13c1a62c0 con 0x7fa13c101770 2024-03-07T07:40:38.288 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.288+0000 7fa12b7fe640 1 -- 172.21.15.160:0/3413741858 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) v1 ==== 136805+0+0 (secure 0 0 0) 0x7fa130044070 con 0x7fa13c101770 2024-03-07T07:40:38.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.289+0000 7fa12b7fe640 1 --2- 172.21.15.160:0/3413741858 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa1240a1e40 0x7fa1240a4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:38.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.289+0000 7fa140bc9640 1 -- 172.21.15.160:0/3413741858 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa13c068450 con 0x7fa13c101770 2024-03-07T07:40:38.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.289+0000 7fa13a59c640 1 --2- 172.21.15.160:0/3413741858 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa1240a1e40 0x7fa1240a4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:38.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.289+0000 7fa12b7fe640 1 -- 172.21.15.160:0/3413741858 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa1300ef260 con 0x7fa13c101770 2024-03-07T07:40:38.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.293+0000 7fa13a59c640 1 --2- 172.21.15.160:0/3413741858 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa1240a1e40 0x7fa1240a4300 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa13c1015d0 tx=0x7fa12c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:38.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.299+0000 7fa12b7fe640 1 -- 172.21.15.160:0/3413741858 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa1300bc600 con 0x7fa13c101770 2024-03-07T07:40:38.427 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:38 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/1129548933' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-07T07:40:38.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:38 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/3488891330' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-07T07:40:38.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:38 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/2745308121' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-07T07:40:38.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:38 smithi160 ceph-mon[31963]: pgmap v332: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-07T07:40:38.526 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.526+0000 7fa140bc9640 1 -- 172.21.15.160:0/3413741858 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa13c02d030 con 0x7fa13c101770 2024-03-07T07:40:38.526 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.526+0000 7fa12b7fe640 1 -- 172.21.15.160:0/3413741858 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+11411 (secure 0 0 0) 0x7fa1300bbf00 con 0x7fa13c101770 2024-03-07T07:40:38.527 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:40:38.527 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/2615961083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2615961083},{"type":"v1","addr":"172.21.15.160:6835","nonce":2615961083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1759006033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1759006033},{"type":"v1","addr":"172.21.15.160:6837","nonce":1759006033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6839/3763944721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":3763944721},{"type":"v1","addr":"172.21.15.160:6839","nonce":3763944721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6841/3611426637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6840","nonce":3611426637},{"type":"v1","addr":"172.21.15.160:6841","nonce":3611426637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1457364761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1457364761},{"type":"v1","addr":"172.21.15.177:6833","nonce":1457364761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/170428292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":170428292},{"type":"v1","addr":"172.21.15.177:6835","nonce":170428292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-07T07:39:14.164243+0000","modified":"2024-03-07T07:39:18.861903+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24511,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/645365846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":645365846},{"type":"v1","addr":"172.21.15.178:6833","nonce":645365846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6837/4088036516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":4088036516},{"type":"v1","addr":"172.21.15.177:6837","nonce":4088036516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/647840547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":647840547},{"type":"v1","addr":"172.21.15.178:6835","nonce":647840547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.178:6837/1247829154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1247829154},{"type":"v1","addr":"172.21.15.178:6837","nonce":1247829154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24497":{"gid":24497,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/3329773838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3329773838},{"type":"v1","addr":"172.21.15.177:6839","nonce":3329773838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24511":{"gid":24511,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.178:6839/2947354024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":2947354024},{"type":"v1","addr":"172.21.15.178:6839","nonce":2947354024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24511,"qdb_cluster":[24487,24511,24497]},"id":1}]} 2024-03-07T07:40:38.527 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 21 2024-03-07T07:40:38.529 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.528+0000 7fa140bc9640 1 -- 172.21.15.160:0/3413741858 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa1240a1e40 msgr2=0x7fa1240a4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:38.529 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.528+0000 7fa140bc9640 1 --2- 172.21.15.160:0/3413741858 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa1240a1e40 0x7fa1240a4300 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa13c1015d0 tx=0x7fa12c005f90 comp rx=0 tx=0).stop 2024-03-07T07:40:38.529 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.528+0000 7fa140bc9640 1 -- 172.21.15.160:0/3413741858 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa13c101770 msgr2=0x7fa13c19ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:38.529 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.528+0000 7fa140bc9640 1 --2- 172.21.15.160:0/3413741858 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa13c101770 0x7fa13c19ebb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa130009230 tx=0x7fa130005d50 comp rx=0 tx=0).stop 2024-03-07T07:40:38.529 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.528+0000 7fa140bc9640 1 -- 172.21.15.160:0/3413741858 shutdown_connections 2024-03-07T07:40:38.529 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.528+0000 7fa140bc9640 1 --2- 172.21.15.160:0/3413741858 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa1240a1e40 0x7fa1240a4300 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:38.529 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.528+0000 7fa140bc9640 1 --2- 172.21.15.160:0/3413741858 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa13c102130 0x7fa13c1a5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:38.529 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.528+0000 7fa140bc9640 1 --2- 172.21.15.160:0/3413741858 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa13c101770 0x7fa13c19ebb0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:38.529 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.528+0000 7fa140bc9640 1 --2- 172.21.15.160:0/3413741858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa13c100580 0x7fa13c19e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:38.530 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.528+0000 7fa140bc9640 1 -- 172.21.15.160:0/3413741858 >> 172.21.15.160:0/3413741858 conn(0x7fa13c0fbaf0 msgr2=0x7fa13c10dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:38.530 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.529+0000 7fa140bc9640 1 -- 172.21.15.160:0/3413741858 shutdown_connections 2024-03-07T07:40:38.530 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.529+0000 7fa140bc9640 1 -- 172.21.15.160:0/3413741858 wait complete. 2024-03-07T07:40:38.540 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T07:40:38.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.642+0000 7efe4d1b1640 1 -- 172.21.15.160:0/3714244543 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe48101770 msgr2=0x7efe48101bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:38.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.642+0000 7efe4d1b1640 1 --2- 172.21.15.160:0/3714244543 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe48101770 0x7efe48101bf0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7efe30009900 tx=0x7efe3002bc30 comp rx=0 tx=0).stop 2024-03-07T07:40:38.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.642+0000 7efe4d1b1640 1 -- 172.21.15.160:0/3714244543 shutdown_connections 2024-03-07T07:40:38.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.642+0000 7efe4d1b1640 1 --2- 172.21.15.160:0/3714244543 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe48102130 0x7efe481089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:38.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.642+0000 7efe4d1b1640 1 --2- 172.21.15.160:0/3714244543 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe48101770 0x7efe48101bf0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:38.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.642+0000 7efe4d1b1640 1 --2- 172.21.15.160:0/3714244543 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe48100580 0x7efe481009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:38.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.642+0000 7efe4d1b1640 1 -- 172.21.15.160:0/3714244543 >> 172.21.15.160:0/3714244543 conn(0x7efe480fbaf0 msgr2=0x7efe480fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:38.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.642+0000 7efe4d1b1640 1 -- 172.21.15.160:0/3714244543 shutdown_connections 2024-03-07T07:40:38.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.642+0000 7efe4d1b1640 1 -- 172.21.15.160:0/3714244543 wait complete. 2024-03-07T07:40:38.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.643+0000 7efe4d1b1640 1 Processor -- start 2024-03-07T07:40:38.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.643+0000 7efe4d1b1640 1 -- start start 2024-03-07T07:40:38.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.643+0000 7efe4d1b1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe48100580 0x7efe48069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:38.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.643+0000 7efe4d1b1640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe48101770 0x7efe4819c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:38.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.643+0000 7efe4d1b1640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe48102130 0x7efe4819ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:38.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.643+0000 7efe4d1b1640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe4810c210 con 0x7efe48101770 2024-03-07T07:40:38.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.643+0000 7efe4d1b1640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe4810c090 con 0x7efe48102130 2024-03-07T07:40:38.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.643+0000 7efe4d1b1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe4810c390 con 0x7efe48100580 2024-03-07T07:40:38.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.643+0000 7efe3ffff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe48101770 0x7efe4819c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:38.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.643+0000 7efe3ffff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe48101770 0x7efe4819c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:38400/0 (socket says 172.21.15.160:38400) 2024-03-07T07:40:38.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.643+0000 7efe3ffff640 1 -- 172.21.15.160:0/195479034 learned_addr learned my addr 172.21.15.160:0/195479034 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:40:38.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.643+0000 7efe46d9d640 1 --2- 172.21.15.160:0/195479034 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe48100580 0x7efe48069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:38.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.643+0000 7efe4759e640 1 --2- 172.21.15.160:0/195479034 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe48102130 0x7efe4819ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:38.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.643+0000 7efe3ffff640 1 -- 172.21.15.160:0/195479034 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe48100580 msgr2=0x7efe48069330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:38.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.643+0000 7efe3ffff640 1 --2- 172.21.15.160:0/195479034 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe48100580 0x7efe48069330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:38.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.643+0000 7efe3ffff640 1 -- 172.21.15.160:0/195479034 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe48102130 msgr2=0x7efe4819ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:38.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.643+0000 7efe3ffff640 1 --2- 172.21.15.160:0/195479034 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe48102130 0x7efe4819ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:38.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.643+0000 7efe3ffff640 1 -- 172.21.15.160:0/195479034 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe4819f610 con 0x7efe48101770 2024-03-07T07:40:38.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.644+0000 7efe3ffff640 1 --2- 172.21.15.160:0/195479034 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe48101770 0x7efe4819c6e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7efe30009230 tx=0x7efe3002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:38.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.644+0000 7efe44d99640 1 -- 172.21.15.160:0/195479034 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe30041070 con 0x7efe48101770 2024-03-07T07:40:38.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.644+0000 7efe44d99640 1 -- 172.21.15.160:0/195479034 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efe30004450 con 0x7efe48101770 2024-03-07T07:40:38.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.644+0000 7efe4d1b1640 1 -- 172.21.15.160:0/195479034 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe4819f840 con 0x7efe48101770 2024-03-07T07:40:38.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.644+0000 7efe44d99640 1 -- 172.21.15.160:0/195479034 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe30003e20 con 0x7efe48101770 2024-03-07T07:40:38.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.644+0000 7efe4d1b1640 1 -- 172.21.15.160:0/195479034 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe4819fd30 con 0x7efe48101770 2024-03-07T07:40:38.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.645+0000 7efe4d1b1640 1 -- 172.21.15.160:0/195479034 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe481078a0 con 0x7efe48101770 2024-03-07T07:40:38.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.646+0000 7efe44d99640 1 -- 172.21.15.160:0/195479034 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) v1 ==== 136805+0+0 (secure 0 0 0) 0x7efe30044070 con 0x7efe48101770 2024-03-07T07:40:38.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.646+0000 7efe44d99640 1 --2- 172.21.15.160:0/195479034 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7efe340a1e10 0x7efe340a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:38.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.646+0000 7efe46d9d640 1 --2- 172.21.15.160:0/195479034 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7efe340a1e10 0x7efe340a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:38.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.646+0000 7efe44d99640 1 -- 172.21.15.160:0/195479034 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7efe300ef9a0 con 0x7efe48101770 2024-03-07T07:40:38.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.647+0000 7efe46d9d640 1 --2- 172.21.15.160:0/195479034 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7efe340a1e10 0x7efe340a42d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7efe481015d0 tx=0x7efe38005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:38.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.651+0000 7efe44d99640 1 -- 172.21.15.160:0/195479034 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7efe300bc730 con 0x7efe48101770 2024-03-07T07:40:38.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:38 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/1129548933' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-07T07:40:38.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:38 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/3488891330' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-07T07:40:38.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:38 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/2745308121' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-07T07:40:38.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:38 smithi178 ceph-mon[36650]: pgmap v332: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-07T07:40:38.827 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.827+0000 7efe4d1b1640 1 -- 172.21.15.160:0/195479034 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe4802d250 con 0x7efe48101770 2024-03-07T07:40:38.828 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.828+0000 7efe44d99640 1 -- 172.21.15.160:0/195479034 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22721 (secure 0 0 0) 0x7efe300c07d0 con 0x7efe48101770 2024-03-07T07:40:38.829 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:40:38.829 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":87,"fsid":"826f6582-dc54-11ee-95c6-87774f69a715","created":"2024-03-07T07:31:14.229288+0000","modified":"2024-03-07T07:40:09.354265+0000","last_up_change":"2024-03-07T07:38:20.776225+0000","last_in_change":"2024-03-07T07:38:00.508917+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T07:34:34.529424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T07:39:12.075432+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T07:39:13.367820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bee66c67-fb6e-4717-9685-ca62587d5f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6803","nonce":1368482926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6805","nonce":1368482926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6809","nonce":1368482926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6807","nonce":1368482926}]},"public_addr":"172.21.15.160:6803/1368482926","cluster_addr":"172.21.15.160:6805/1368482926","heartbeat_back_addr":"172.21.15.160:6809/1368482926","heartbeat_front_addr":"172.21.15.160:6807/1368482926","state":["exists","up"]},{"osd":1,"uuid":"49236251-f750-424e-992b-bddf55762556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6801","nonce":2922391884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6803","nonce":2922391884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6807","nonce":2922391884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6805","nonce":2922391884}]},"public_addr":"172.21.15.177:6801/2922391884","cluster_addr":"172.21.15.177:6803/2922391884","heartbeat_back_addr":"172.21.15.177:6807/2922391884","heartbeat_front_addr":"172.21.15.177:6805/2922391884","state":["exists","up"]},{"osd":2,"uuid":"7a432b28-f7ee-43fe-9196-7aaeccfecf77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6801","nonce":1217901551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6803","nonce":1217901551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6807","nonce":1217901551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6805","nonce":1217901551}]},"public_addr":"172.21.15.178:6801/1217901551","cluster_addr":"172.21.15.178:6803/1217901551","heartbeat_back_addr":"172.21.15.178:6807/1217901551","heartbeat_front_addr":"172.21.15.178:6805/1217901551","state":["exists","up"]},{"osd":3,"uuid":"4ad22a20-f1be-4002-a48d-ecd3953e63f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6811","nonce":42543989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6813","nonce":42543989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6817","nonce":42543989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6815","nonce":42543989}]},"public_addr":"172.21.15.160:6811/42543989","cluster_addr":"172.21.15.160:6813/42543989","heartbeat_back_addr":"172.21.15.160:6817/42543989","heartbeat_front_addr":"172.21.15.160:6815/42543989","state":["exists","up"]},{"osd":4,"uuid":"2b65b104-ac82-4ce3-b6c5-11a399103045","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6809","nonce":88178553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6811","nonce":88178553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6815","nonce":88178553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6813","nonce":88178553}]},"public_addr":"172.21.15.177:6809/88178553","cluster_addr":"172.21.15.177:6811/88178553","heartbeat_back_addr":"172.21.15.177:6815/88178553","heartbeat_front_addr":"172.21.15.177:6813/88178553","state":["exists","up"]},{"osd":5,"uuid":"30a1e714-08ff-44e8-9264-9ef164604b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6809","nonce":964732891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6811","nonce":964732891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6815","nonce":964732891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6813","nonce":964732891}]},"public_addr":"172.21.15.178:6809/964732891","cluster_addr":"172.21.15.178:6811/964732891","heartbeat_back_addr":"172.21.15.178:6815/964732891","heartbeat_front_addr":"172.21.15.178:6813/964732891","state":["exists","up"]},{"osd":6,"uuid":"b1e1646e-f8a9-42b8-b62d-9d7b5d6a1ff4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6819","nonce":3251327001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6821","nonce":3251327001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6825","nonce":3251327001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6823","nonce":3251327001}]},"public_addr":"172.21.15.160:6819/3251327001","cluster_addr":"172.21.15.160:6821/3251327001","heartbeat_back_addr":"172.21.15.160:6825/3251327001","heartbeat_front_addr":"172.21.15.160:6823/3251327001","state":["exists","up"]},{"osd":7,"uuid":"e66b400b-9747-4f6d-b05b-db22468e2750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6817","nonce":1175500850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6819","nonce":1175500850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6823","nonce":1175500850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6821","nonce":1175500850}]},"public_addr":"172.21.15.177:6817/1175500850","cluster_addr":"172.21.15.177:6819/1175500850","heartbeat_back_addr":"172.21.15.177:6823/1175500850","heartbeat_front_addr":"172.21.15.177:6821/1175500850","state":["exists","up"]},{"osd":8,"uuid":"3ee0f2c4-fca4-49fc-a98b-07aa0fcba0e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6817","nonce":688015531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6819","nonce":688015531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6823","nonce":688015531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6821","nonce":688015531}]},"public_addr":"172.21.15.178:6817/688015531","cluster_addr":"172.21.15.178:6819/688015531","heartbeat_back_addr":"172.21.15.178:6823/688015531","heartbeat_front_addr":"172.21.15.178:6821/688015531","state":["exists","up"]},{"osd":9,"uuid":"856039d7-f7ee-4dbd-b019-0073e705ce0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6827","nonce":3265797851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6829","nonce":3265797851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6833","nonce":3265797851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6831","nonce":3265797851}]},"public_addr":"172.21.15.160:6827/3265797851","cluster_addr":"172.21.15.160:6829/3265797851","heartbeat_back_addr":"172.21.15.160:6833/3265797851","heartbeat_front_addr":"172.21.15.160:6831/3265797851","state":["exists","up"]},{"osd":10,"uuid":"bdc9899d-d720-4429-ad53-4c388a4dbe75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6825","nonce":2867473412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6827","nonce":2867473412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6831","nonce":2867473412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6829","nonce":2867473412}]},"public_addr":"172.21.15.177:6825/2867473412","cluster_addr":"172.21.15.177:6827/2867473412","heartbeat_back_addr":"172.21.15.177:6831/2867473412","heartbeat_front_addr":"172.21.15.177:6829/2867473412","state":["exists","up"]},{"osd":11,"uuid":"cc9adb54-ef49-47c6-90e3-fc03bc0f5cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6825","nonce":4068724731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6827","nonce":4068724731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6831","nonce":4068724731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6829","nonce":4068724731}]},"public_addr":"172.21.15.178:6825/4068724731","cluster_addr":"172.21.15.178:6827/4068724731","heartbeat_back_addr":"172.21.15.178:6831/4068724731","heartbeat_front_addr":"172.21.15.178:6829/4068724731","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:33:44.141585+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:07.685585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:31.044386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:56.308925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:35:21.582097+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:35:45.983540+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:36:11.530242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:36:36.844681+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:02.303437+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:27.629867+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:53.289495+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:38:18.536169+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/3027450612":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/874709722":"2024-03-08T07:32:08.190837+0000","172.21.15.160:6800/1130675225":"2024-03-08T07:31:41.166858+0000","172.21.15.160:0/3857193980":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/3861968297":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6800/3859174315":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/3013534743":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6801/3859174315":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/1310937667":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6801/1130675225":"2024-03-08T07:31:41.166858+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T07:40:38.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.830+0000 7efe4d1b1640 1 -- 172.21.15.160:0/195479034 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7efe340a1e10 msgr2=0x7efe340a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:38.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.830+0000 7efe4d1b1640 1 --2- 172.21.15.160:0/195479034 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7efe340a1e10 0x7efe340a42d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7efe481015d0 tx=0x7efe38005f90 comp rx=0 tx=0).stop 2024-03-07T07:40:38.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.830+0000 7efe4d1b1640 1 -- 172.21.15.160:0/195479034 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe48101770 msgr2=0x7efe4819c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:38.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.830+0000 7efe4d1b1640 1 --2- 172.21.15.160:0/195479034 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe48101770 0x7efe4819c6e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7efe30009230 tx=0x7efe3002c650 comp rx=0 tx=0).stop 2024-03-07T07:40:38.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.830+0000 7efe4d1b1640 1 -- 172.21.15.160:0/195479034 shutdown_connections 2024-03-07T07:40:38.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.830+0000 7efe4d1b1640 1 --2- 172.21.15.160:0/195479034 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7efe340a1e10 0x7efe340a42d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:38.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.830+0000 7efe4d1b1640 1 --2- 172.21.15.160:0/195479034 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe48102130 0x7efe4819ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:38.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.830+0000 7efe4d1b1640 1 --2- 172.21.15.160:0/195479034 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe48101770 0x7efe4819c6e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:38.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.830+0000 7efe4d1b1640 1 --2- 172.21.15.160:0/195479034 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe48100580 0x7efe48069330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:38.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.830+0000 7efe4d1b1640 1 -- 172.21.15.160:0/195479034 >> 172.21.15.160:0/195479034 conn(0x7efe480fbaf0 msgr2=0x7efe48104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:38.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.830+0000 7efe4d1b1640 1 -- 172.21.15.160:0/195479034 shutdown_connections 2024-03-07T07:40:38.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.830+0000 7efe4d1b1640 1 -- 172.21.15.160:0/195479034 wait complete. 2024-03-07T07:40:38.843 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-07T07:34:34.529424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-07T07:39:12.075432+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-07T07:39:13.367820+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-07T07:40:38.844 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-07T07:40:38.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:38 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/1129548933' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-07T07:40:38.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:38 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/3488891330' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-07T07:40:38.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:38 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/2745308121' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-07T07:40:38.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:38 smithi177 ceph-mon[36578]: pgmap v332: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-07T07:40:38.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.948+0000 7f5dad01a640 1 -- 172.21.15.160:0/420288204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5da8104a70 msgr2=0x7f5da8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:38.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.948+0000 7f5dad01a640 1 --2- 172.21.15.160:0/420288204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5da8104a70 0x7f5da8106e60 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c009900 tx=0x7f5d9c02bc30 comp rx=0 tx=0).stop 2024-03-07T07:40:38.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.948+0000 7f5dad01a640 1 -- 172.21.15.160:0/420288204 shutdown_connections 2024-03-07T07:40:38.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.948+0000 7f5dad01a640 1 --2- 172.21.15.160:0/420288204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5da81073a0 0x7f5da8109790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:38.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.948+0000 7f5dad01a640 1 --2- 172.21.15.160:0/420288204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5da8104a70 0x7f5da8106e60 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:38.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.948+0000 7f5dad01a640 1 --2- 172.21.15.160:0/420288204 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5da8102140 0x7f5da8104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:38.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.948+0000 7f5dad01a640 1 -- 172.21.15.160:0/420288204 >> 172.21.15.160:0/420288204 conn(0x7f5da80fbaf0 msgr2=0x7f5da80fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:38.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.948+0000 7f5dad01a640 1 -- 172.21.15.160:0/420288204 shutdown_connections 2024-03-07T07:40:38.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.948+0000 7f5dad01a640 1 -- 172.21.15.160:0/420288204 wait complete. 2024-03-07T07:40:38.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.949+0000 7f5dad01a640 1 Processor -- start 2024-03-07T07:40:38.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.949+0000 7f5dad01a640 1 -- start start 2024-03-07T07:40:38.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.949+0000 7f5dad01a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5da8102140 0x7f5da8197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:38.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.949+0000 7f5dad01a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5da8104a70 0x7f5da8197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:38.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.949+0000 7f5dad01a640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5da81073a0 0x7f5da819efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:38.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.949+0000 7f5dad01a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da810d2e0 con 0x7f5da8104a70 2024-03-07T07:40:38.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.949+0000 7f5dad01a640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da810d160 con 0x7f5da81073a0 2024-03-07T07:40:38.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.949+0000 7f5dad01a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da810d460 con 0x7f5da8102140 2024-03-07T07:40:38.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.949+0000 7f5da659c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5da8104a70 0x7f5da8197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:38.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.949+0000 7f5da659c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5da8104a70 0x7f5da8197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:38422/0 (socket says 172.21.15.160:38422) 2024-03-07T07:40:38.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.949+0000 7f5da659c640 1 -- 172.21.15.160:0/91453891 learned_addr learned my addr 172.21.15.160:0/91453891 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:40:38.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.949+0000 7f5da6d9d640 1 --2- 172.21.15.160:0/91453891 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5da8102140 0x7f5da8197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:38.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.949+0000 7f5da759e640 1 --2- 172.21.15.160:0/91453891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5da81073a0 0x7f5da819efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:38.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.950+0000 7f5da659c640 1 -- 172.21.15.160:0/91453891 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5da8102140 msgr2=0x7f5da8197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:38.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.950+0000 7f5da659c640 1 --2- 172.21.15.160:0/91453891 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5da8102140 0x7f5da8197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:38.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.950+0000 7f5da659c640 1 -- 172.21.15.160:0/91453891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5da81073a0 msgr2=0x7f5da819efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:38.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.950+0000 7f5da659c640 1 --2- 172.21.15.160:0/91453891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5da81073a0 0x7f5da819efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:38.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.950+0000 7f5da659c640 1 -- 172.21.15.160:0/91453891 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5da819f650 con 0x7f5da8104a70 2024-03-07T07:40:38.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.950+0000 7f5da659c640 1 --2- 172.21.15.160:0/91453891 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5da8104a70 0x7f5da8197fc0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c000c00 tx=0x7f5d9c02c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:38.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.950+0000 7f5d8ffff640 1 -- 172.21.15.160:0/91453891 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c041070 con 0x7f5da8104a70 2024-03-07T07:40:38.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.950+0000 7f5dad01a640 1 -- 172.21.15.160:0/91453891 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5da819f8e0 con 0x7f5da8104a70 2024-03-07T07:40:38.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.950+0000 7f5d8ffff640 1 -- 172.21.15.160:0/91453891 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d9c004450 con 0x7f5da8104a70 2024-03-07T07:40:38.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.950+0000 7f5d8ffff640 1 -- 172.21.15.160:0/91453891 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c034440 con 0x7f5da8104a70 2024-03-07T07:40:38.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.950+0000 7f5dad01a640 1 -- 172.21.15.160:0/91453891 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5da819fdd0 con 0x7f5da8104a70 2024-03-07T07:40:38.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.951+0000 7f5dad01a640 1 -- 172.21.15.160:0/91453891 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5da80fd5e0 con 0x7f5da8104a70 2024-03-07T07:40:38.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.952+0000 7f5d8ffff640 1 -- 172.21.15.160:0/91453891 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) v1 ==== 136805+0+0 (secure 0 0 0) 0x7f5d9c044070 con 0x7f5da8104a70 2024-03-07T07:40:38.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.953+0000 7f5d8ffff640 1 --2- 172.21.15.160:0/91453891 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f5d900a1e10 0x7f5d900a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:38.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.953+0000 7f5da6d9d640 1 --2- 172.21.15.160:0/91453891 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f5d900a1e10 0x7f5d900a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:38.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.953+0000 7f5d8ffff640 1 -- 172.21.15.160:0/91453891 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5d9c0ef8a0 con 0x7f5da8104a70 2024-03-07T07:40:38.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.953+0000 7f5da6d9d640 1 --2- 172.21.15.160:0/91453891 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f5d900a1e10 0x7f5d900a42d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5d980099a0 tx=0x7f5d98009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:38.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:38.957+0000 7f5d8ffff640 1 -- 172.21.15.160:0/91453891 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5d9c0bc6c0 con 0x7f5da8104a70 2024-03-07T07:40:39.132 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.130+0000 7f5dad01a640 1 -- 172.21.15.160:0/91453891 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5da802d250 con 0x7f5da8104a70 2024-03-07T07:40:39.132 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.131+0000 7f5d8ffff640 1 -- 172.21.15.160:0/91453891 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5d9c0c0760 con 0x7f5da8104a70 2024-03-07T07:40:39.133 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 1 2024-03-07T07:40:39.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.133+0000 7f5dad01a640 1 -- 172.21.15.160:0/91453891 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f5d900a1e10 msgr2=0x7f5d900a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:39.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.133+0000 7f5dad01a640 1 --2- 172.21.15.160:0/91453891 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f5d900a1e10 0x7f5d900a42d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5d980099a0 tx=0x7f5d98009500 comp rx=0 tx=0).stop 2024-03-07T07:40:39.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.133+0000 7f5dad01a640 1 -- 172.21.15.160:0/91453891 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5da8104a70 msgr2=0x7f5da8197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:39.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.133+0000 7f5dad01a640 1 --2- 172.21.15.160:0/91453891 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5da8104a70 0x7f5da8197fc0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c000c00 tx=0x7f5d9c02c640 comp rx=0 tx=0).stop 2024-03-07T07:40:39.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.133+0000 7f5dad01a640 1 -- 172.21.15.160:0/91453891 shutdown_connections 2024-03-07T07:40:39.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.133+0000 7f5dad01a640 1 --2- 172.21.15.160:0/91453891 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f5d900a1e10 0x7f5d900a42d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:39.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.133+0000 7f5dad01a640 1 --2- 172.21.15.160:0/91453891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5da81073a0 0x7f5da819efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:39.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.133+0000 7f5dad01a640 1 --2- 172.21.15.160:0/91453891 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5da8104a70 0x7f5da8197fc0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:39.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.133+0000 7f5dad01a640 1 --2- 172.21.15.160:0/91453891 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5da8102140 0x7f5da8197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:39.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.133+0000 7f5dad01a640 1 -- 172.21.15.160:0/91453891 >> 172.21.15.160:0/91453891 conn(0x7f5da80fbaf0 msgr2=0x7f5da8102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:39.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.133+0000 7f5dad01a640 1 -- 172.21.15.160:0/91453891 shutdown_connections 2024-03-07T07:40:39.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.133+0000 7f5dad01a640 1 -- 172.21.15.160:0/91453891 wait complete. 2024-03-07T07:40:39.145 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-07T07:40:39.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.250+0000 7f58e0edd640 1 -- 172.21.15.160:0/453705985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58dc101750 msgr2=0x7f58dc101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:39.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.250+0000 7f58e0edd640 1 --2- 172.21.15.160:0/453705985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58dc101750 0x7f58dc101bd0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f58c4009900 tx=0x7f58c402bc30 comp rx=0 tx=0).stop 2024-03-07T07:40:39.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.250+0000 7f58e0edd640 1 -- 172.21.15.160:0/453705985 shutdown_connections 2024-03-07T07:40:39.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.250+0000 7f58e0edd640 1 --2- 172.21.15.160:0/453705985 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58dc102110 0x7f58dc1089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:39.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.250+0000 7f58e0edd640 1 --2- 172.21.15.160:0/453705985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58dc101750 0x7f58dc101bd0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:39.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.250+0000 7f58e0edd640 1 --2- 172.21.15.160:0/453705985 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58dc100560 0x7f58dc100980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:39.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.250+0000 7f58e0edd640 1 -- 172.21.15.160:0/453705985 >> 172.21.15.160:0/453705985 conn(0x7f58dc0fbad0 msgr2=0x7f58dc0fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:39.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.250+0000 7f58e0edd640 1 -- 172.21.15.160:0/453705985 shutdown_connections 2024-03-07T07:40:39.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.250+0000 7f58e0edd640 1 -- 172.21.15.160:0/453705985 wait complete. 2024-03-07T07:40:39.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.250+0000 7f58e0edd640 1 Processor -- start 2024-03-07T07:40:39.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.250+0000 7f58e0edd640 1 -- start start 2024-03-07T07:40:39.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.250+0000 7f58e0edd640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58dc100560 0x7f58dc197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:39.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.250+0000 7f58e0edd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58dc101750 0x7f58dc197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:39.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.251+0000 7f58e0edd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58dc102110 0x7f58dc19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:39.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.251+0000 7f58e0edd640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc10c1f0 con 0x7f58dc101750 2024-03-07T07:40:39.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.251+0000 7f58e0edd640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc10c070 con 0x7f58dc100560 2024-03-07T07:40:39.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.251+0000 7f58e0edd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc10c370 con 0x7f58dc102110 2024-03-07T07:40:39.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.251+0000 7f58d3fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58dc101750 0x7f58dc197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:39.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.251+0000 7f58d3fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58dc101750 0x7f58dc197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:38434/0 (socket says 172.21.15.160:38434) 2024-03-07T07:40:39.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.251+0000 7f58d3fff640 1 -- 172.21.15.160:0/3674181198 learned_addr learned my addr 172.21.15.160:0/3674181198 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:40:39.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.251+0000 7f58da59c640 1 --2- 172.21.15.160:0/3674181198 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58dc100560 0x7f58dc197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:39.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.251+0000 7f58dad9d640 1 --2- 172.21.15.160:0/3674181198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58dc102110 0x7f58dc19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:39.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.251+0000 7f58da59c640 1 -- 172.21.15.160:0/3674181198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58dc102110 msgr2=0x7f58dc19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:39.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.251+0000 7f58da59c640 1 --2- 172.21.15.160:0/3674181198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58dc102110 0x7f58dc19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:39.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.251+0000 7f58da59c640 1 -- 172.21.15.160:0/3674181198 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58dc101750 msgr2=0x7f58dc197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:39.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.251+0000 7f58da59c640 1 --2- 172.21.15.160:0/3674181198 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58dc101750 0x7f58dc197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:39.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.251+0000 7f58da59c640 1 -- 172.21.15.160:0/3674181198 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58dc19f5b0 con 0x7f58dc100560 2024-03-07T07:40:39.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.251+0000 7f58d3fff640 1 --2- 172.21.15.160:0/3674181198 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58dc101750 0x7f58dc197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:40:39.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.251+0000 7f58da59c640 1 --2- 172.21.15.160:0/3674181198 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58dc100560 0x7f58dc197990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f58cc00e9c0 tx=0x7f58cc00edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:39.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.251+0000 7f58d37fe640 1 -- 172.21.15.160:0/3674181198 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58cc00ccc0 con 0x7f58dc100560 2024-03-07T07:40:39.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.251+0000 7f58d37fe640 1 -- 172.21.15.160:0/3674181198 <== mon.2 v2:172.21.15.177:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f58cc0035f0 con 0x7f58dc100560 2024-03-07T07:40:39.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.251+0000 7f58e0edd640 1 -- 172.21.15.160:0/3674181198 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58dc19f8a0 con 0x7f58dc100560 2024-03-07T07:40:39.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.251+0000 7f58d37fe640 1 -- 172.21.15.160:0/3674181198 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58cc00ce60 con 0x7f58dc100560 2024-03-07T07:40:39.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.251+0000 7f58e0edd640 1 -- 172.21.15.160:0/3674181198 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58dc19fdc0 con 0x7f58dc100560 2024-03-07T07:40:39.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.252+0000 7f58e0edd640 1 -- 172.21.15.160:0/3674181198 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58dc1a0020 con 0x7f58dc100560 2024-03-07T07:40:39.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.253+0000 7f58d37fe640 1 -- 172.21.15.160:0/3674181198 <== mon.2 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 17) v1 ==== 136805+0+0 (secure 0 0 0) 0x7f58cc0037a0 con 0x7f58dc100560 2024-03-07T07:40:39.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.254+0000 7f58d37fe640 1 --2- 172.21.15.160:0/3674181198 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f58c80a1e10 0x7f58c80a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:39.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.254+0000 7f58d3fff640 1 --2- 172.21.15.160:0/3674181198 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f58c80a1e10 0x7f58c80a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:39.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.254+0000 7f58d37fe640 1 -- 172.21.15.160:0/3674181198 <== mon.2 v2:172.21.15.177:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f58cc0ce250 con 0x7f58dc100560 2024-03-07T07:40:39.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.254+0000 7f58d3fff640 1 --2- 172.21.15.160:0/3674181198 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f58c80a1e10 0x7f58c80a42d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f58c4000c00 tx=0x7f58c402c880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:39.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.257+0000 7f58d37fe640 1 -- 172.21.15.160:0/3674181198 <== mon.2 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f58cc09b520 con 0x7f58dc100560 2024-03-07T07:40:39.399 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.399+0000 7f58d37fe640 1 -- 172.21.15.160:0/3674181198 <== mon.2 v2:172.21.15.177:3300/0 7 ==== mgrmap(e 18) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f58cc097490 con 0x7f58dc100560 2024-03-07T07:40:39.427 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:39 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/3413741858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:40:39.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:39 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/195479034' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:40:39.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:39 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/91453891' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-07T07:40:39.436 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.436+0000 7f58e0edd640 1 -- 172.21.15.160:0/3674181198 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f58dc1a01f0 con 0x7f58dc100560 2024-03-07T07:40:39.437 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.437+0000 7f58d37fe640 1 -- 172.21.15.160:0/3674181198 <== mon.2 v2:172.21.15.177:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f58cc06ce70 con 0x7f58dc100560 2024-03-07T07:40:39.437 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 32 2024-03-07T07:40:39.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.439+0000 7f58e0edd640 1 -- 172.21.15.160:0/3674181198 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f58c80a1e10 msgr2=0x7f58c80a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:39.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.439+0000 7f58e0edd640 1 --2- 172.21.15.160:0/3674181198 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f58c80a1e10 0x7f58c80a42d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f58c4000c00 tx=0x7f58c402c880 comp rx=0 tx=0).stop 2024-03-07T07:40:39.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.439+0000 7f58e0edd640 1 -- 172.21.15.160:0/3674181198 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58dc100560 msgr2=0x7f58dc197990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:39.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.439+0000 7f58e0edd640 1 --2- 172.21.15.160:0/3674181198 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58dc100560 0x7f58dc197990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f58cc00e9c0 tx=0x7f58cc00edc0 comp rx=0 tx=0).stop 2024-03-07T07:40:39.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.439+0000 7f58e0edd640 1 -- 172.21.15.160:0/3674181198 shutdown_connections 2024-03-07T07:40:39.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.439+0000 7f58e0edd640 1 --2- 172.21.15.160:0/3674181198 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f58c80a1e10 0x7f58c80a42d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:39.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.439+0000 7f58e0edd640 1 --2- 172.21.15.160:0/3674181198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58dc102110 0x7f58dc19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:39.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.439+0000 7f58e0edd640 1 --2- 172.21.15.160:0/3674181198 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58dc101750 0x7f58dc197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:39.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.439+0000 7f58e0edd640 1 --2- 172.21.15.160:0/3674181198 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58dc100560 0x7f58dc197990 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:39.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.439+0000 7f58e0edd640 1 -- 172.21.15.160:0/3674181198 >> 172.21.15.160:0/3674181198 conn(0x7f58dc0fbad0 msgr2=0x7f58dc0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:39.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.439+0000 7f58e0edd640 1 -- 172.21.15.160:0/3674181198 shutdown_connections 2024-03-07T07:40:39.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.439+0000 7f58e0edd640 1 -- 172.21.15.160:0/3674181198 wait complete. 2024-03-07T07:40:39.450 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-07T07:40:39.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.554+0000 7fc042bc9640 1 -- 172.21.15.160:0/3856099971 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc03c0ffd00 msgr2=0x7fc03c100120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:39.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.554+0000 7fc042bc9640 1 --2- 172.21.15.160:0/3856099971 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc03c0ffd00 0x7fc03c100120 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc03000aa10 tx=0x7fc03002cf70 comp rx=0 tx=0).stop 2024-03-07T07:40:39.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.554+0000 7fc042bc9640 1 -- 172.21.15.160:0/3856099971 shutdown_connections 2024-03-07T07:40:39.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.554+0000 7fc042bc9640 1 --2- 172.21.15.160:0/3856099971 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc03c1018b0 0x7fc03c108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:39.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.554+0000 7fc042bc9640 1 --2- 172.21.15.160:0/3856099971 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc03c100ef0 0x7fc03c101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:39.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.554+0000 7fc042bc9640 1 --2- 172.21.15.160:0/3856099971 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc03c0ffd00 0x7fc03c100120 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:39.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.554+0000 7fc042bc9640 1 -- 172.21.15.160:0/3856099971 >> 172.21.15.160:0/3856099971 conn(0x7fc03c0fb290 msgr2=0x7fc03c0fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:39.558 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.554+0000 7fc042bc9640 1 -- 172.21.15.160:0/3856099971 shutdown_connections 2024-03-07T07:40:39.558 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.554+0000 7fc042bc9640 1 -- 172.21.15.160:0/3856099971 wait complete. 2024-03-07T07:40:39.558 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.555+0000 7fc042bc9640 1 Processor -- start 2024-03-07T07:40:39.558 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.555+0000 7fc042bc9640 1 -- start start 2024-03-07T07:40:39.558 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.555+0000 7fc042bc9640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc03c0ffd00 0x7fc03c1979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:39.558 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.555+0000 7fc042bc9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc03c100ef0 0x7fc03c197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:39.558 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.555+0000 7fc042bc9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc03c1018b0 0x7fc03c19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:39.558 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.555+0000 7fc042bc9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c10b110 con 0x7fc03c1018b0 2024-03-07T07:40:39.558 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.555+0000 7fc042bc9640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c10af90 con 0x7fc03c0ffd00 2024-03-07T07:40:39.558 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.555+0000 7fc042bc9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c10b290 con 0x7fc03c100ef0 2024-03-07T07:40:39.559 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.555+0000 7fc041166640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc03c1018b0 0x7fc03c19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:39.559 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.555+0000 7fc041166640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc03c1018b0 0x7fc03c19ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59038/0 (socket says 172.21.15.160:59038) 2024-03-07T07:40:39.559 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.555+0000 7fc041166640 1 -- 172.21.15.160:0/1438026892 learned_addr learned my addr 172.21.15.160:0/1438026892 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:40:39.559 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.555+0000 7fc040965640 1 --2- 172.21.15.160:0/1438026892 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc03c0ffd00 0x7fc03c1979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:39.559 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.555+0000 7fc03bfff640 1 --2- 172.21.15.160:0/1438026892 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc03c100ef0 0x7fc03c197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:39.559 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.555+0000 7fc041166640 1 -- 172.21.15.160:0/1438026892 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc03c100ef0 msgr2=0x7fc03c197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:39.559 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.555+0000 7fc041166640 1 --2- 172.21.15.160:0/1438026892 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc03c100ef0 0x7fc03c197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:39.559 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.555+0000 7fc041166640 1 -- 172.21.15.160:0/1438026892 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc03c0ffd00 msgr2=0x7fc03c1979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:39.559 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.555+0000 7fc041166640 1 --2- 172.21.15.160:0/1438026892 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc03c0ffd00 0x7fc03c1979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:39.559 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.555+0000 7fc041166640 1 -- 172.21.15.160:0/1438026892 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc03c19f5c0 con 0x7fc03c1018b0 2024-03-07T07:40:39.559 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.555+0000 7fc040965640 1 --2- 172.21.15.160:0/1438026892 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc03c0ffd00 0x7fc03c1979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:40:39.559 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.555+0000 7fc041166640 1 --2- 172.21.15.160:0/1438026892 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc03c1018b0 0x7fc03c19ef50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc03400eb10 tx=0x7fc03400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:39.559 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.555+0000 7fc039ffb640 1 -- 172.21.15.160:0/1438026892 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc034004b20 con 0x7fc03c1018b0 2024-03-07T07:40:39.560 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.556+0000 7fc042bc9640 1 -- 172.21.15.160:0/1438026892 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc03c19f8b0 con 0x7fc03c1018b0 2024-03-07T07:40:39.560 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.556+0000 7fc039ffb640 1 -- 172.21.15.160:0/1438026892 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc034004cc0 con 0x7fc03c1018b0 2024-03-07T07:40:39.560 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.556+0000 7fc039ffb640 1 -- 172.21.15.160:0/1438026892 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc034010760 con 0x7fc03c1018b0 2024-03-07T07:40:39.560 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.556+0000 7fc042bc9640 1 -- 172.21.15.160:0/1438026892 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc03c19fe50 con 0x7fc03c1018b0 2024-03-07T07:40:39.560 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.557+0000 7fc042bc9640 1 -- 172.21.15.160:0/1438026892 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc03c107020 con 0x7fc03c1018b0 2024-03-07T07:40:39.561 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.558+0000 7fc039ffb640 1 -- 172.21.15.160:0/1438026892 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 136754+0+0 (secure 0 0 0) 0x7fc034005020 con 0x7fc03c1018b0 2024-03-07T07:40:39.561 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.558+0000 7fc039ffb640 1 --2- 172.21.15.160:0/1438026892 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fc02c0a1d80 0x7fc02c0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:39.561 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.558+0000 7fc039ffb640 1 -- 172.21.15.160:0/1438026892 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc0340ce4b0 con 0x7fc03c1018b0 2024-03-07T07:40:39.561 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.558+0000 7fc040965640 1 --2- 172.21.15.160:0/1438026892 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fc02c0a1d80 0x7fc02c0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:39.561 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.559+0000 7fc040965640 1 --2- 172.21.15.160:0/1438026892 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fc02c0a1d80 0x7fc02c0a4240 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc030000c00 tx=0x7fc030005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:39.563 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.563+0000 7fc039ffb640 1 -- 172.21.15.160:0/1438026892 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc03409b8e0 con 0x7fc03c1018b0 2024-03-07T07:40:39.738 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.738+0000 7fc042bc9640 1 -- 172.21.15.160:0/1438026892 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc03c062260 con 0x7fc03c1018b0 2024-03-07T07:40:39.739 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.739+0000 7fc039ffb640 1 -- 172.21.15.160:0/1438026892 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc03409b1e0 con 0x7fc03c1018b0 2024-03-07T07:40:39.739 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 64 2024-03-07T07:40:39.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.741+0000 7fc042bc9640 1 -- 172.21.15.160:0/1438026892 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fc02c0a1d80 msgr2=0x7fc02c0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:39.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.741+0000 7fc042bc9640 1 --2- 172.21.15.160:0/1438026892 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fc02c0a1d80 0x7fc02c0a4240 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc030000c00 tx=0x7fc030005fd0 comp rx=0 tx=0).stop 2024-03-07T07:40:39.742 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.741+0000 7fc042bc9640 1 -- 172.21.15.160:0/1438026892 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc03c1018b0 msgr2=0x7fc03c19ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:39.742 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.741+0000 7fc042bc9640 1 --2- 172.21.15.160:0/1438026892 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc03c1018b0 0x7fc03c19ef50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc03400eb10 tx=0x7fc03400c2d0 comp rx=0 tx=0).stop 2024-03-07T07:40:39.742 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.741+0000 7fc042bc9640 1 -- 172.21.15.160:0/1438026892 shutdown_connections 2024-03-07T07:40:39.742 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.741+0000 7fc042bc9640 1 --2- 172.21.15.160:0/1438026892 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fc02c0a1d80 0x7fc02c0a4240 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:39.742 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.741+0000 7fc042bc9640 1 --2- 172.21.15.160:0/1438026892 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc03c1018b0 0x7fc03c19ef50 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:39.742 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.741+0000 7fc042bc9640 1 --2- 172.21.15.160:0/1438026892 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc03c100ef0 0x7fc03c197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:39.742 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.741+0000 7fc042bc9640 1 --2- 172.21.15.160:0/1438026892 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc03c0ffd00 0x7fc03c1979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:39.742 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.741+0000 7fc042bc9640 1 -- 172.21.15.160:0/1438026892 >> 172.21.15.160:0/1438026892 conn(0x7fc03c0fb290 msgr2=0x7fc03c1041f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:39.742 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.741+0000 7fc042bc9640 1 -- 172.21.15.160:0/1438026892 shutdown_connections 2024-03-07T07:40:39.742 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.741+0000 7fc042bc9640 1 -- 172.21.15.160:0/1438026892 wait complete. 2024-03-07T07:40:39.752 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T07:40:39.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:39 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/3413741858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:40:39.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:39 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/195479034' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:40:39.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:39 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/91453891' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-07T07:40:39.856 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.853+0000 7f84dbd89640 1 -- 172.21.15.160:0/552001877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84d41016b0 msgr2=0x7f84d4101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:39.856 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.853+0000 7f84dbd89640 1 --2- 172.21.15.160:0/552001877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84d41016b0 0x7f84d4101ad0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f84cc009ca0 tx=0x7f84cc02c0f0 comp rx=0 tx=0).stop 2024-03-07T07:40:39.856 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.853+0000 7f84dbd89640 1 -- 172.21.15.160:0/552001877 shutdown_connections 2024-03-07T07:40:39.856 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.853+0000 7f84dbd89640 1 --2- 172.21.15.160:0/552001877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84d4103260 0x7f84d4109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:39.856 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.853+0000 7f84dbd89640 1 --2- 172.21.15.160:0/552001877 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f84d41028a0 0x7f84d4102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:39.856 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.853+0000 7f84dbd89640 1 --2- 172.21.15.160:0/552001877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84d41016b0 0x7f84d4101ad0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:39.856 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.853+0000 7f84dbd89640 1 -- 172.21.15.160:0/552001877 >> 172.21.15.160:0/552001877 conn(0x7f84d40fcc00 msgr2=0x7f84d40ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:39.856 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.854+0000 7f84dbd89640 1 -- 172.21.15.160:0/552001877 shutdown_connections 2024-03-07T07:40:39.856 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.854+0000 7f84dbd89640 1 -- 172.21.15.160:0/552001877 wait complete. 2024-03-07T07:40:39.856 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.854+0000 7f84dbd89640 1 Processor -- start 2024-03-07T07:40:39.856 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.854+0000 7f84dbd89640 1 -- start start 2024-03-07T07:40:39.857 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.854+0000 7f84dbd89640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84d41016b0 0x7f84d41979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:39.857 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.854+0000 7f84dbd89640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f84d41028a0 0x7f84d4197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:39.857 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.854+0000 7f84dbd89640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84d4103260 0x7f84d419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:39.857 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.854+0000 7f84dbd89640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d410d300 con 0x7f84d41016b0 2024-03-07T07:40:39.857 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.854+0000 7f84dbd89640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d410d180 con 0x7f84d41028a0 2024-03-07T07:40:39.857 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.854+0000 7f84dbd89640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d410d480 con 0x7f84d4103260 2024-03-07T07:40:39.857 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.854+0000 7f84d9b25640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84d41016b0 0x7f84d41979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:39.857 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.854+0000 7f84d9b25640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84d41016b0 0x7f84d41979e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59064/0 (socket says 172.21.15.160:59064) 2024-03-07T07:40:39.857 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.854+0000 7f84d9b25640 1 -- 172.21.15.160:0/692349172 learned_addr learned my addr 172.21.15.160:0/692349172 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:40:39.857 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.854+0000 7f84da326640 1 --2- 172.21.15.160:0/692349172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84d4103260 0x7f84d419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:39.857 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.854+0000 7f84d9324640 1 --2- 172.21.15.160:0/692349172 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f84d41028a0 0x7f84d4197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:39.857 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.855+0000 7f84d9b25640 1 -- 172.21.15.160:0/692349172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84d4103260 msgr2=0x7f84d419ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:39.857 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.855+0000 7f84d9b25640 1 --2- 172.21.15.160:0/692349172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84d4103260 0x7f84d419ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:39.857 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.855+0000 7f84d9b25640 1 -- 172.21.15.160:0/692349172 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f84d41028a0 msgr2=0x7f84d4197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:39.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.855+0000 7f84d9b25640 1 --2- 172.21.15.160:0/692349172 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f84d41028a0 0x7f84d4197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:39.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.855+0000 7f84d9b25640 1 -- 172.21.15.160:0/692349172 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d419f600 con 0x7f84d41016b0 2024-03-07T07:40:39.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.855+0000 7f84d9b25640 1 --2- 172.21.15.160:0/692349172 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84d41016b0 0x7f84d41979e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f84cc0324c0 tx=0x7f84cc032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:39.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.855+0000 7f84caffd640 1 -- 172.21.15.160:0/692349172 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84cc041020 con 0x7f84d41016b0 2024-03-07T07:40:39.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.855+0000 7f84caffd640 1 -- 172.21.15.160:0/692349172 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84cc034cd0 con 0x7f84d41016b0 2024-03-07T07:40:39.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.855+0000 7f84dbd89640 1 -- 172.21.15.160:0/692349172 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84d419f890 con 0x7f84d41016b0 2024-03-07T07:40:39.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.855+0000 7f84caffd640 1 -- 172.21.15.160:0/692349172 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84cc039040 con 0x7f84d41016b0 2024-03-07T07:40:39.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.855+0000 7f84dbd89640 1 -- 172.21.15.160:0/692349172 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84d419fd80 con 0x7f84d41016b0 2024-03-07T07:40:39.859 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:39 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/3413741858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:40:39.859 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:39 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/195479034' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:40:39.859 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:39 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/91453891' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-07T07:40:39.860 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.856+0000 7f84dbd89640 1 -- 172.21.15.160:0/692349172 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84d4108a50 con 0x7f84d41016b0 2024-03-07T07:40:39.860 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.857+0000 7f84caffd640 1 -- 172.21.15.160:0/692349172 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f84cc044020 con 0x7f84d41016b0 2024-03-07T07:40:39.860 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.857+0000 7f84caffd640 1 --2- 172.21.15.160:0/692349172 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f84c00a1d80 0x7f84c00a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:39.860 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.858+0000 7f84d9324640 1 --2- 172.21.15.160:0/692349172 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f84c00a1d80 0x7f84c00a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:39.860 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.858+0000 7f84caffd640 1 -- 172.21.15.160:0/692349172 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f84cc0efc70 con 0x7f84d41016b0 2024-03-07T07:40:39.860 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.858+0000 7f84d9324640 1 --2- 172.21.15.160:0/692349172 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f84c00a1d80 0x7f84c00a4240 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f84d4198ef0 tx=0x7f84c4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:39.862 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:39.862+0000 7f84caffd640 1 -- 172.21.15.160:0/692349172 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f84cc04b020 con 0x7f84d41016b0 2024-03-07T07:40:40.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.074+0000 7f84dbd89640 1 -- 172.21.15.160:0/692349172 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f84d4062260 con 0x7f84d41016b0 2024-03-07T07:40:40.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.075+0000 7f84caffd640 1 -- 172.21.15.160:0/692349172 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+11411 (secure 0 0 0) 0x7f84cc0bd130 con 0x7f84d41016b0 2024-03-07T07:40:40.075 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:40:40.075 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/2615961083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2615961083},{"type":"v1","addr":"172.21.15.160:6835","nonce":2615961083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1759006033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1759006033},{"type":"v1","addr":"172.21.15.160:6837","nonce":1759006033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6839/3763944721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":3763944721},{"type":"v1","addr":"172.21.15.160:6839","nonce":3763944721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6841/3611426637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6840","nonce":3611426637},{"type":"v1","addr":"172.21.15.160:6841","nonce":3611426637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1457364761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1457364761},{"type":"v1","addr":"172.21.15.177:6833","nonce":1457364761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/170428292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":170428292},{"type":"v1","addr":"172.21.15.177:6835","nonce":170428292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-07T07:39:14.164243+0000","modified":"2024-03-07T07:39:18.861903+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24511,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/645365846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":645365846},{"type":"v1","addr":"172.21.15.178:6833","nonce":645365846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6837/4088036516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":4088036516},{"type":"v1","addr":"172.21.15.177:6837","nonce":4088036516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/647840547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":647840547},{"type":"v1","addr":"172.21.15.178:6835","nonce":647840547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.178:6837/1247829154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1247829154},{"type":"v1","addr":"172.21.15.178:6837","nonce":1247829154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24497":{"gid":24497,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/3329773838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3329773838},{"type":"v1","addr":"172.21.15.177:6839","nonce":3329773838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24511":{"gid":24511,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.178:6839/2947354024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":2947354024},{"type":"v1","addr":"172.21.15.178:6839","nonce":2947354024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24511,"qdb_cluster":[24487,24511,24497]},"id":1}]} 2024-03-07T07:40:40.076 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 21 2024-03-07T07:40:40.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.077+0000 7f84dbd89640 1 -- 172.21.15.160:0/692349172 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f84c00a1d80 msgr2=0x7f84c00a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:40.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.077+0000 7f84dbd89640 1 --2- 172.21.15.160:0/692349172 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f84c00a1d80 0x7f84c00a4240 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f84d4198ef0 tx=0x7f84c4005f90 comp rx=0 tx=0).stop 2024-03-07T07:40:40.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.077+0000 7f84dbd89640 1 -- 172.21.15.160:0/692349172 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84d41016b0 msgr2=0x7f84d41979e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:40.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.077+0000 7f84dbd89640 1 --2- 172.21.15.160:0/692349172 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84d41016b0 0x7f84d41979e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f84cc0324c0 tx=0x7f84cc032f40 comp rx=0 tx=0).stop 2024-03-07T07:40:40.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.077+0000 7f84dbd89640 1 -- 172.21.15.160:0/692349172 shutdown_connections 2024-03-07T07:40:40.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.077+0000 7f84dbd89640 1 --2- 172.21.15.160:0/692349172 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f84c00a1d80 0x7f84c00a4240 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.077+0000 7f84dbd89640 1 --2- 172.21.15.160:0/692349172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84d4103260 0x7f84d419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.077+0000 7f84dbd89640 1 --2- 172.21.15.160:0/692349172 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f84d41028a0 0x7f84d4197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.077+0000 7f84dbd89640 1 --2- 172.21.15.160:0/692349172 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84d41016b0 0x7f84d41979e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.077+0000 7f84dbd89640 1 -- 172.21.15.160:0/692349172 >> 172.21.15.160:0/692349172 conn(0x7f84d40fcc00 msgr2=0x7f84d4105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:40.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.077+0000 7f84dbd89640 1 -- 172.21.15.160:0/692349172 shutdown_connections 2024-03-07T07:40:40.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.077+0000 7f84dbd89640 1 -- 172.21.15.160:0/692349172 wait complete. 2024-03-07T07:40:40.088 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T07:40:40.191 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.189+0000 7fcfb4e34640 1 -- 172.21.15.160:0/3807813089 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfb0101790 msgr2=0x7fcfb0101c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:40.191 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.189+0000 7fcfb4e34640 1 --2- 172.21.15.160:0/3807813089 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfb0101790 0x7fcfb0101c10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcf98009900 tx=0x7fcf9802bc30 comp rx=0 tx=0).stop 2024-03-07T07:40:40.191 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.189+0000 7fcfb4e34640 1 -- 172.21.15.160:0/3807813089 shutdown_connections 2024-03-07T07:40:40.191 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.189+0000 7fcfb4e34640 1 --2- 172.21.15.160:0/3807813089 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcfb0102150 0x7fcfb01089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.189+0000 7fcfb4e34640 1 --2- 172.21.15.160:0/3807813089 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfb0101790 0x7fcfb0101c10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.189+0000 7fcfb4e34640 1 --2- 172.21.15.160:0/3807813089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcfb01005a0 0x7fcfb01009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.189+0000 7fcfb4e34640 1 -- 172.21.15.160:0/3807813089 >> 172.21.15.160:0/3807813089 conn(0x7fcfb00fbb10 msgr2=0x7fcfb00fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:40.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.189+0000 7fcfb4e34640 1 -- 172.21.15.160:0/3807813089 shutdown_connections 2024-03-07T07:40:40.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.189+0000 7fcfb4e34640 1 -- 172.21.15.160:0/3807813089 wait complete. 2024-03-07T07:40:40.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.190+0000 7fcfb4e34640 1 Processor -- start 2024-03-07T07:40:40.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.190+0000 7fcfb4e34640 1 -- start start 2024-03-07T07:40:40.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.190+0000 7fcfb4e34640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcfb01005a0 0x7fcfb0197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:40.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.190+0000 7fcfb4e34640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfb0101790 0x7fcfb0197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:40.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.190+0000 7fcfb4e34640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcfb0102150 0x7fcfb019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:40.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.190+0000 7fcfb4e34640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb010c230 con 0x7fcfb0102150 2024-03-07T07:40:40.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.190+0000 7fcfb4e34640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb010c0b0 con 0x7fcfb0101790 2024-03-07T07:40:40.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.190+0000 7fcfb4e34640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb010c3b0 con 0x7fcfb01005a0 2024-03-07T07:40:40.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.190+0000 7fcfaed9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcfb0102150 0x7fcfb019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:40.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.190+0000 7fcfaed9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcfb0102150 0x7fcfb019ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59078/0 (socket says 172.21.15.160:59078) 2024-03-07T07:40:40.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.190+0000 7fcfaed9d640 1 -- 172.21.15.160:0/1466886991 learned_addr learned my addr 172.21.15.160:0/1466886991 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:40:40.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.190+0000 7fcfae59c640 1 --2- 172.21.15.160:0/1466886991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcfb01005a0 0x7fcfb0197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:40.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.190+0000 7fcfa7fff640 1 --2- 172.21.15.160:0/1466886991 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfb0101790 0x7fcfb0197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:40.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.190+0000 7fcfae59c640 1 -- 172.21.15.160:0/1466886991 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfb0101790 msgr2=0x7fcfb0197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:40.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.190+0000 7fcfae59c640 1 --2- 172.21.15.160:0/1466886991 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfb0101790 0x7fcfb0197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.190+0000 7fcfae59c640 1 -- 172.21.15.160:0/1466886991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcfb0102150 msgr2=0x7fcfb019ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:40.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.190+0000 7fcfae59c640 1 --2- 172.21.15.160:0/1466886991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcfb0102150 0x7fcfb019ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.190+0000 7fcfae59c640 1 -- 172.21.15.160:0/1466886991 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb019f580 con 0x7fcfb01005a0 2024-03-07T07:40:40.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.190+0000 7fcfaed9d640 1 --2- 172.21.15.160:0/1466886991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcfb0102150 0x7fcfb019ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:40:40.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.191+0000 7fcfae59c640 1 --2- 172.21.15.160:0/1466886991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcfb01005a0 0x7fcfb0197960 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcfa000e9c0 tx=0x7fcfa000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:40.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.191+0000 7fcfa77fe640 1 -- 172.21.15.160:0/1466886991 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa000ccc0 con 0x7fcfb01005a0 2024-03-07T07:40:40.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.191+0000 7fcfb4e34640 1 -- 172.21.15.160:0/1466886991 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfb019f870 con 0x7fcfb01005a0 2024-03-07T07:40:40.194 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.191+0000 7fcfa77fe640 1 -- 172.21.15.160:0/1466886991 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcfa00035f0 con 0x7fcfb01005a0 2024-03-07T07:40:40.194 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.191+0000 7fcfa77fe640 1 -- 172.21.15.160:0/1466886991 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa000ce60 con 0x7fcfb01005a0 2024-03-07T07:40:40.194 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.191+0000 7fcfb4e34640 1 -- 172.21.15.160:0/1466886991 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfb019fde0 con 0x7fcfb01005a0 2024-03-07T07:40:40.194 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.192+0000 7fcfb4e34640 1 -- 172.21.15.160:0/1466886991 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfb01a0040 con 0x7fcfb01005a0 2024-03-07T07:40:40.195 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.193+0000 7fcfa77fe640 1 -- 172.21.15.160:0/1466886991 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136754+0+0 (secure 0 0 0) 0x7fcfa00037a0 con 0x7fcfb01005a0 2024-03-07T07:40:40.195 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.193+0000 7fcfa77fe640 1 --2- 172.21.15.160:0/1466886991 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fcf9c0a1d80 0x7fcf9c0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:40.195 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.193+0000 7fcfa7fff640 1 --2- 172.21.15.160:0/1466886991 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fcf9c0a1d80 0x7fcf9c0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:40.195 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.193+0000 7fcfa77fe640 1 -- 172.21.15.160:0/1466886991 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fcfa00ce1d0 con 0x7fcfb01005a0 2024-03-07T07:40:40.195 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.194+0000 7fcfa7fff640 1 --2- 172.21.15.160:0/1466886991 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fcf9c0a1d80 0x7fcf9c0a4240 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcf9802c7b0 tx=0x7fcf98005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:40.197 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.197+0000 7fcfa77fe640 1 -- 172.21.15.160:0/1466886991 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fcfa009b600 con 0x7fcfb01005a0 2024-03-07T07:40:40.374 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.374+0000 7fcfb4e34640 1 -- 172.21.15.160:0/1466886991 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcfb01a0210 con 0x7fcfb01005a0 2024-03-07T07:40:40.375 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.376+0000 7fcfa77fe640 1 -- 172.21.15.160:0/1466886991 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22721 (secure 0 0 0) 0x7fcfa009af00 con 0x7fcfb01005a0 2024-03-07T07:40:40.376 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:40:40.376 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":87,"fsid":"826f6582-dc54-11ee-95c6-87774f69a715","created":"2024-03-07T07:31:14.229288+0000","modified":"2024-03-07T07:40:09.354265+0000","last_up_change":"2024-03-07T07:38:20.776225+0000","last_in_change":"2024-03-07T07:38:00.508917+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T07:34:34.529424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T07:39:12.075432+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T07:39:13.367820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bee66c67-fb6e-4717-9685-ca62587d5f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6803","nonce":1368482926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6805","nonce":1368482926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6809","nonce":1368482926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6807","nonce":1368482926}]},"public_addr":"172.21.15.160:6803/1368482926","cluster_addr":"172.21.15.160:6805/1368482926","heartbeat_back_addr":"172.21.15.160:6809/1368482926","heartbeat_front_addr":"172.21.15.160:6807/1368482926","state":["exists","up"]},{"osd":1,"uuid":"49236251-f750-424e-992b-bddf55762556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6801","nonce":2922391884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6803","nonce":2922391884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6807","nonce":2922391884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6805","nonce":2922391884}]},"public_addr":"172.21.15.177:6801/2922391884","cluster_addr":"172.21.15.177:6803/2922391884","heartbeat_back_addr":"172.21.15.177:6807/2922391884","heartbeat_front_addr":"172.21.15.177:6805/2922391884","state":["exists","up"]},{"osd":2,"uuid":"7a432b28-f7ee-43fe-9196-7aaeccfecf77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6801","nonce":1217901551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6803","nonce":1217901551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6807","nonce":1217901551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6805","nonce":1217901551}]},"public_addr":"172.21.15.178:6801/1217901551","cluster_addr":"172.21.15.178:6803/1217901551","heartbeat_back_addr":"172.21.15.178:6807/1217901551","heartbeat_front_addr":"172.21.15.178:6805/1217901551","state":["exists","up"]},{"osd":3,"uuid":"4ad22a20-f1be-4002-a48d-ecd3953e63f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6811","nonce":42543989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6813","nonce":42543989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6817","nonce":42543989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6815","nonce":42543989}]},"public_addr":"172.21.15.160:6811/42543989","cluster_addr":"172.21.15.160:6813/42543989","heartbeat_back_addr":"172.21.15.160:6817/42543989","heartbeat_front_addr":"172.21.15.160:6815/42543989","state":["exists","up"]},{"osd":4,"uuid":"2b65b104-ac82-4ce3-b6c5-11a399103045","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6809","nonce":88178553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6811","nonce":88178553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6815","nonce":88178553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6813","nonce":88178553}]},"public_addr":"172.21.15.177:6809/88178553","cluster_addr":"172.21.15.177:6811/88178553","heartbeat_back_addr":"172.21.15.177:6815/88178553","heartbeat_front_addr":"172.21.15.177:6813/88178553","state":["exists","up"]},{"osd":5,"uuid":"30a1e714-08ff-44e8-9264-9ef164604b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6809","nonce":964732891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6811","nonce":964732891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6815","nonce":964732891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6813","nonce":964732891}]},"public_addr":"172.21.15.178:6809/964732891","cluster_addr":"172.21.15.178:6811/964732891","heartbeat_back_addr":"172.21.15.178:6815/964732891","heartbeat_front_addr":"172.21.15.178:6813/964732891","state":["exists","up"]},{"osd":6,"uuid":"b1e1646e-f8a9-42b8-b62d-9d7b5d6a1ff4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6819","nonce":3251327001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6821","nonce":3251327001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6825","nonce":3251327001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6823","nonce":3251327001}]},"public_addr":"172.21.15.160:6819/3251327001","cluster_addr":"172.21.15.160:6821/3251327001","heartbeat_back_addr":"172.21.15.160:6825/3251327001","heartbeat_front_addr":"172.21.15.160:6823/3251327001","state":["exists","up"]},{"osd":7,"uuid":"e66b400b-9747-4f6d-b05b-db22468e2750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6817","nonce":1175500850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6819","nonce":1175500850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6823","nonce":1175500850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6821","nonce":1175500850}]},"public_addr":"172.21.15.177:6817/1175500850","cluster_addr":"172.21.15.177:6819/1175500850","heartbeat_back_addr":"172.21.15.177:6823/1175500850","heartbeat_front_addr":"172.21.15.177:6821/1175500850","state":["exists","up"]},{"osd":8,"uuid":"3ee0f2c4-fca4-49fc-a98b-07aa0fcba0e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6817","nonce":688015531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6819","nonce":688015531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6823","nonce":688015531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6821","nonce":688015531}]},"public_addr":"172.21.15.178:6817/688015531","cluster_addr":"172.21.15.178:6819/688015531","heartbeat_back_addr":"172.21.15.178:6823/688015531","heartbeat_front_addr":"172.21.15.178:6821/688015531","state":["exists","up"]},{"osd":9,"uuid":"856039d7-f7ee-4dbd-b019-0073e705ce0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6827","nonce":3265797851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6829","nonce":3265797851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6833","nonce":3265797851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6831","nonce":3265797851}]},"public_addr":"172.21.15.160:6827/3265797851","cluster_addr":"172.21.15.160:6829/3265797851","heartbeat_back_addr":"172.21.15.160:6833/3265797851","heartbeat_front_addr":"172.21.15.160:6831/3265797851","state":["exists","up"]},{"osd":10,"uuid":"bdc9899d-d720-4429-ad53-4c388a4dbe75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6825","nonce":2867473412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6827","nonce":2867473412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6831","nonce":2867473412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6829","nonce":2867473412}]},"public_addr":"172.21.15.177:6825/2867473412","cluster_addr":"172.21.15.177:6827/2867473412","heartbeat_back_addr":"172.21.15.177:6831/2867473412","heartbeat_front_addr":"172.21.15.177:6829/2867473412","state":["exists","up"]},{"osd":11,"uuid":"cc9adb54-ef49-47c6-90e3-fc03bc0f5cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6825","nonce":4068724731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6827","nonce":4068724731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6831","nonce":4068724731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6829","nonce":4068724731}]},"public_addr":"172.21.15.178:6825/4068724731","cluster_addr":"172.21.15.178:6827/4068724731","heartbeat_back_addr":"172.21.15.178:6831/4068724731","heartbeat_front_addr":"172.21.15.178:6829/4068724731","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:33:44.141585+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:07.685585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:31.044386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:56.308925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:35:21.582097+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:35:45.983540+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:36:11.530242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:36:36.844681+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:02.303437+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:27.629867+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:53.289495+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:38:18.536169+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/3027450612":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/874709722":"2024-03-08T07:32:08.190837+0000","172.21.15.160:6800/1130675225":"2024-03-08T07:31:41.166858+0000","172.21.15.160:0/3857193980":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/3861968297":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6800/3859174315":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/3013534743":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6801/3859174315":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/1310937667":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6801/1130675225":"2024-03-08T07:31:41.166858+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T07:40:40.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.377+0000 7fcfb4e34640 1 -- 172.21.15.160:0/1466886991 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fcf9c0a1d80 msgr2=0x7fcf9c0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:40.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.377+0000 7fcfb4e34640 1 --2- 172.21.15.160:0/1466886991 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fcf9c0a1d80 0x7fcf9c0a4240 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcf9802c7b0 tx=0x7fcf98005fd0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.377+0000 7fcfb4e34640 1 -- 172.21.15.160:0/1466886991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcfb01005a0 msgr2=0x7fcfb0197960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:40.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.377+0000 7fcfb4e34640 1 --2- 172.21.15.160:0/1466886991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcfb01005a0 0x7fcfb0197960 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcfa000e9c0 tx=0x7fcfa000ee80 comp rx=0 tx=0).stop 2024-03-07T07:40:40.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.378+0000 7fcfb4e34640 1 -- 172.21.15.160:0/1466886991 shutdown_connections 2024-03-07T07:40:40.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.378+0000 7fcfb4e34640 1 --2- 172.21.15.160:0/1466886991 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fcf9c0a1d80 0x7fcf9c0a4240 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.378+0000 7fcfb4e34640 1 --2- 172.21.15.160:0/1466886991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcfb0102150 0x7fcfb019ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.378+0000 7fcfb4e34640 1 --2- 172.21.15.160:0/1466886991 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfb0101790 0x7fcfb0197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.378+0000 7fcfb4e34640 1 --2- 172.21.15.160:0/1466886991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcfb01005a0 0x7fcfb0197960 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.378+0000 7fcfb4e34640 1 -- 172.21.15.160:0/1466886991 >> 172.21.15.160:0/1466886991 conn(0x7fcfb00fbb10 msgr2=0x7fcfb0069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:40.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.378+0000 7fcfb4e34640 1 -- 172.21.15.160:0/1466886991 shutdown_connections 2024-03-07T07:40:40.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.378+0000 7fcfb4e34640 1 -- 172.21.15.160:0/1466886991 wait complete. 2024-03-07T07:40:40.390 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T07:40:40.493 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.491+0000 7f66c3ba7640 1 -- 172.21.15.160:0/1999833814 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66bc1017f0 msgr2=0x7f66bc101c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:40.493 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.491+0000 7f66c3ba7640 1 --2- 172.21.15.160:0/1999833814 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66bc1017f0 0x7f66bc101c70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f66b8009900 tx=0x7f66b802bc30 comp rx=0 tx=0).stop 2024-03-07T07:40:40.494 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.491+0000 7f66c3ba7640 1 -- 172.21.15.160:0/1999833814 shutdown_connections 2024-03-07T07:40:40.494 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.491+0000 7f66c3ba7640 1 --2- 172.21.15.160:0/1999833814 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66bc1021b0 0x7f66bc108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.494 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.491+0000 7f66c3ba7640 1 --2- 172.21.15.160:0/1999833814 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66bc1017f0 0x7f66bc101c70 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.494 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.491+0000 7f66c3ba7640 1 --2- 172.21.15.160:0/1999833814 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66bc100600 0x7f66bc100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.494 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.491+0000 7f66c3ba7640 1 -- 172.21.15.160:0/1999833814 >> 172.21.15.160:0/1999833814 conn(0x7f66bc0fbb90 msgr2=0x7f66bc0fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:40.494 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.491+0000 7f66c3ba7640 1 -- 172.21.15.160:0/1999833814 shutdown_connections 2024-03-07T07:40:40.495 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.491+0000 7f66c3ba7640 1 -- 172.21.15.160:0/1999833814 wait complete. 2024-03-07T07:40:40.495 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.491+0000 7f66c3ba7640 1 Processor -- start 2024-03-07T07:40:40.495 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.491+0000 7f66c3ba7640 1 -- start start 2024-03-07T07:40:40.495 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.491+0000 7f66c3ba7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66bc100600 0x7f66bc1979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:40.495 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.491+0000 7f66c3ba7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66bc1017f0 0x7f66bc197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:40.495 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.491+0000 7f66c3ba7640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66bc1021b0 0x7f66bc19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:40.495 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.491+0000 7f66c3ba7640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66bc10c220 con 0x7f66bc1021b0 2024-03-07T07:40:40.495 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.491+0000 7f66c3ba7640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66bc10c0a0 con 0x7f66bc100600 2024-03-07T07:40:40.495 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.491+0000 7f66c3ba7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66bc10c3a0 con 0x7f66bc1017f0 2024-03-07T07:40:40.495 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.492+0000 7f66c2144640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66bc1021b0 0x7f66bc19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:40.495 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.492+0000 7f66c2144640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66bc1021b0 0x7f66bc19ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59106/0 (socket says 172.21.15.160:59106) 2024-03-07T07:40:40.496 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.492+0000 7f66c2144640 1 -- 172.21.15.160:0/650016228 learned_addr learned my addr 172.21.15.160:0/650016228 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:40:40.496 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.492+0000 7f66c1142640 1 --2- 172.21.15.160:0/650016228 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66bc1017f0 0x7f66bc197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:40.496 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.492+0000 7f66c1943640 1 --2- 172.21.15.160:0/650016228 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66bc100600 0x7f66bc1979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:40.496 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.492+0000 7f66c2144640 1 -- 172.21.15.160:0/650016228 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66bc1017f0 msgr2=0x7f66bc197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:40.496 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.492+0000 7f66c2144640 1 --2- 172.21.15.160:0/650016228 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66bc1017f0 0x7f66bc197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.496 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.492+0000 7f66c2144640 1 -- 172.21.15.160:0/650016228 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66bc100600 msgr2=0x7f66bc1979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:40.496 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.492+0000 7f66c2144640 1 --2- 172.21.15.160:0/650016228 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66bc100600 0x7f66bc1979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.496 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.492+0000 7f66c2144640 1 -- 172.21.15.160:0/650016228 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66bc19f5a0 con 0x7f66bc1021b0 2024-03-07T07:40:40.496 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.492+0000 7f66c2144640 1 --2- 172.21.15.160:0/650016228 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66bc1021b0 0x7f66bc19ef30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f66b400eb10 tx=0x7f66b400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:40.496 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.492+0000 7f66b2ffd640 1 -- 172.21.15.160:0/650016228 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b4004b20 con 0x7f66bc1021b0 2024-03-07T07:40:40.496 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.492+0000 7f66b2ffd640 1 -- 172.21.15.160:0/650016228 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f66b4004cc0 con 0x7f66bc1021b0 2024-03-07T07:40:40.496 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.492+0000 7f66b2ffd640 1 -- 172.21.15.160:0/650016228 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b4010760 con 0x7f66bc1021b0 2024-03-07T07:40:40.496 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.492+0000 7f66c3ba7640 1 -- 172.21.15.160:0/650016228 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66bc19f890 con 0x7f66bc1021b0 2024-03-07T07:40:40.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.492+0000 7f66c3ba7640 1 -- 172.21.15.160:0/650016228 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66bc19fdb0 con 0x7f66bc1021b0 2024-03-07T07:40:40.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.493+0000 7f66c3ba7640 1 -- 172.21.15.160:0/650016228 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66bc1a0010 con 0x7f66bc1021b0 2024-03-07T07:40:40.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.494+0000 7f66b2ffd640 1 -- 172.21.15.160:0/650016228 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f66b4005020 con 0x7f66bc1021b0 2024-03-07T07:40:40.498 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.495+0000 7f66b2ffd640 1 --2- 172.21.15.160:0/650016228 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f66a80a1d80 0x7f66a80a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:40.498 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.495+0000 7f66c1943640 1 --2- 172.21.15.160:0/650016228 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f66a80a1d80 0x7f66a80a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:40.498 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.495+0000 7f66b2ffd640 1 -- 172.21.15.160:0/650016228 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f66b40ce4b0 con 0x7f66bc1021b0 2024-03-07T07:40:40.498 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.495+0000 7f66c1943640 1 --2- 172.21.15.160:0/650016228 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f66a80a1d80 0x7f66a80a4240 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f66bc101650 tx=0x7f66ac005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:40.499 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.499+0000 7f66b2ffd640 1 -- 172.21.15.160:0/650016228 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f66b409b8e0 con 0x7f66bc1021b0 2024-03-07T07:40:40.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:40 smithi160 ceph-mon[31963]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-07T07:40:40.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:40 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/3674181198' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-07T07:40:40.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:40 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/1438026892' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-07T07:40:40.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:40 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/692349172' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:40:40.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:40 smithi160 ceph-mon[31963]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-03-07T07:40:40.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:40 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/1466886991' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:40:40.711 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.711+0000 7f66c3ba7640 1 -- 172.21.15.160:0/650016228 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66bc1a01e0 con 0x7f66bc1021b0 2024-03-07T07:40:40.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.712+0000 7f66b2ffd640 1 -- 172.21.15.160:0/650016228 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+11411 (secure 0 0 0) 0x7f66b409b1e0 con 0x7f66bc1021b0 2024-03-07T07:40:40.712 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:40:40.713 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/2615961083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2615961083},{"type":"v1","addr":"172.21.15.160:6835","nonce":2615961083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1759006033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1759006033},{"type":"v1","addr":"172.21.15.160:6837","nonce":1759006033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6839/3763944721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":3763944721},{"type":"v1","addr":"172.21.15.160:6839","nonce":3763944721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6841/3611426637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6840","nonce":3611426637},{"type":"v1","addr":"172.21.15.160:6841","nonce":3611426637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1457364761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1457364761},{"type":"v1","addr":"172.21.15.177:6833","nonce":1457364761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/170428292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":170428292},{"type":"v1","addr":"172.21.15.177:6835","nonce":170428292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-07T07:39:14.164243+0000","modified":"2024-03-07T07:39:18.861903+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24511,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/645365846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":645365846},{"type":"v1","addr":"172.21.15.178:6833","nonce":645365846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6837/4088036516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":4088036516},{"type":"v1","addr":"172.21.15.177:6837","nonce":4088036516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/647840547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":647840547},{"type":"v1","addr":"172.21.15.178:6835","nonce":647840547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.178:6837/1247829154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1247829154},{"type":"v1","addr":"172.21.15.178:6837","nonce":1247829154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24497":{"gid":24497,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/3329773838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3329773838},{"type":"v1","addr":"172.21.15.177:6839","nonce":3329773838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24511":{"gid":24511,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.178:6839/2947354024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":2947354024},{"type":"v1","addr":"172.21.15.178:6839","nonce":2947354024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24511,"qdb_cluster":[24487,24511,24497]},"id":1}]} 2024-03-07T07:40:40.713 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 21 2024-03-07T07:40:40.714 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.714+0000 7f66c3ba7640 1 -- 172.21.15.160:0/650016228 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f66a80a1d80 msgr2=0x7f66a80a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:40.714 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.714+0000 7f66c3ba7640 1 --2- 172.21.15.160:0/650016228 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f66a80a1d80 0x7f66a80a4240 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f66bc101650 tx=0x7f66ac005f90 comp rx=0 tx=0).stop 2024-03-07T07:40:40.715 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.714+0000 7f66c3ba7640 1 -- 172.21.15.160:0/650016228 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66bc1021b0 msgr2=0x7f66bc19ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:40.715 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.714+0000 7f66c3ba7640 1 --2- 172.21.15.160:0/650016228 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66bc1021b0 0x7f66bc19ef30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f66b400eb10 tx=0x7f66b400c2d0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.715 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.714+0000 7f66c3ba7640 1 -- 172.21.15.160:0/650016228 shutdown_connections 2024-03-07T07:40:40.715 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.714+0000 7f66c3ba7640 1 --2- 172.21.15.160:0/650016228 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f66a80a1d80 0x7f66a80a4240 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.715 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.714+0000 7f66c3ba7640 1 --2- 172.21.15.160:0/650016228 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66bc1021b0 0x7f66bc19ef30 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.715 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.714+0000 7f66c3ba7640 1 --2- 172.21.15.160:0/650016228 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66bc1017f0 0x7f66bc197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.715 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.714+0000 7f66c3ba7640 1 --2- 172.21.15.160:0/650016228 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66bc100600 0x7f66bc1979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.715 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.714+0000 7f66c3ba7640 1 -- 172.21.15.160:0/650016228 >> 172.21.15.160:0/650016228 conn(0x7f66bc0fbb90 msgr2=0x7f66bc0fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:40.715 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.714+0000 7f66c3ba7640 1 -- 172.21.15.160:0/650016228 shutdown_connections 2024-03-07T07:40:40.715 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.714+0000 7f66c3ba7640 1 -- 172.21.15.160:0/650016228 wait complete. 2024-03-07T07:40:40.726 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-07T07:39:14.164243+0000', 'modified': '2024-03-07T07:39:18.861903+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24511, 'mds_1': 24497, 'mds_2': 24487}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24436': {'gid': 24436, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.178:6833/645365846', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6832', 'nonce': 645365846}, {'type': 'v1', 'addr': '172.21.15.178:6833', 'nonce': 645365846}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24458': {'gid': 24458, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.177:6837/4088036516', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6836', 'nonce': 4088036516}, {'type': 'v1', 'addr': '172.21.15.177:6837', 'nonce': 4088036516}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24475': {'gid': 24475, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.178:6835/647840547', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6834', 'nonce': 647840547}, {'type': 'v1', 'addr': '172.21.15.178:6835', 'nonce': 647840547}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24487': {'gid': 24487, 'name': 'c', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.178:6837/1247829154', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6836', 'nonce': 1247829154}, {'type': 'v1', 'addr': '172.21.15.178:6837', 'nonce': 1247829154}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24497': {'gid': 24497, 'name': 'b', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.177:6839/3329773838', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6838', 'nonce': 3329773838}, {'type': 'v1', 'addr': '172.21.15.177:6839', 'nonce': 3329773838}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24511': {'gid': 24511, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.178:6839/2947354024', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6838', 'nonce': 2947354024}, {'type': 'v1', 'addr': '172.21.15.178:6839', 'nonce': 2947354024}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24511, 'qdb_cluster': [24487, 24511, 24497]} 2024-03-07T07:40:40.726 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-03-07T07:40:40.726 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-07T07:40:40.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:40 smithi178 ceph-mon[36650]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-07T07:40:40.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:40 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/3674181198' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-07T07:40:40.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:40 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/1438026892' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-07T07:40:40.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:40 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/692349172' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:40:40.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:40 smithi178 ceph-mon[36650]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-03-07T07:40:40.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:40 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/1466886991' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:40:40.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.829+0000 7fd7a8ad0640 1 -- 172.21.15.160:0/4004931547 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7a40ff940 msgr2=0x7fd7a40ffd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:40.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.829+0000 7fd7a8ad0640 1 --2- 172.21.15.160:0/4004931547 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7a40ff940 0x7fd7a40ffd60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd794009930 tx=0x7fd79402bc30 comp rx=0 tx=0).stop 2024-03-07T07:40:40.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.829+0000 7fd7a8ad0640 1 -- 172.21.15.160:0/4004931547 shutdown_connections 2024-03-07T07:40:40.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.829+0000 7fd7a8ad0640 1 --2- 172.21.15.160:0/4004931547 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7a41032d0 0x7fd7a4109b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.829+0000 7fd7a8ad0640 1 --2- 172.21.15.160:0/4004931547 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7a4102910 0x7fd7a4102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.829+0000 7fd7a8ad0640 1 --2- 172.21.15.160:0/4004931547 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7a40ff940 0x7fd7a40ffd60 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.829+0000 7fd7a8ad0640 1 -- 172.21.15.160:0/4004931547 >> 172.21.15.160:0/4004931547 conn(0x7fd7a40fb250 msgr2=0x7fd7a40fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:40.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.829+0000 7fd7a8ad0640 1 -- 172.21.15.160:0/4004931547 shutdown_connections 2024-03-07T07:40:40.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.829+0000 7fd7a8ad0640 1 -- 172.21.15.160:0/4004931547 wait complete. 2024-03-07T07:40:40.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.830+0000 7fd7a8ad0640 1 Processor -- start 2024-03-07T07:40:40.832 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.830+0000 7fd7a8ad0640 1 -- start start 2024-03-07T07:40:40.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.830+0000 7fd7a8ad0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7a40ff940 0x7fd7a41979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:40.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.830+0000 7fd7a8ad0640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7a4102910 0x7fd7a4197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:40.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.830+0000 7fd7a8ad0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7a41032d0 0x7fd7a419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:40.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.830+0000 7fd7a8ad0640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a410d300 con 0x7fd7a41032d0 2024-03-07T07:40:40.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.830+0000 7fd7a8ad0640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a410d180 con 0x7fd7a4102910 2024-03-07T07:40:40.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.830+0000 7fd7a8ad0640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a410d480 con 0x7fd7a40ff940 2024-03-07T07:40:40.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.830+0000 7fd7a2d9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7a41032d0 0x7fd7a419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:40.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.830+0000 7fd7a2d9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7a41032d0 0x7fd7a419efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59114/0 (socket says 172.21.15.160:59114) 2024-03-07T07:40:40.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.830+0000 7fd7a2d9d640 1 -- 172.21.15.160:0/3448661233 learned_addr learned my addr 172.21.15.160:0/3448661233 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:40:40.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.830+0000 7fd7a259c640 1 --2- 172.21.15.160:0/3448661233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7a40ff940 0x7fd7a41979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:40.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.830+0000 7fd7a1d9b640 1 --2- 172.21.15.160:0/3448661233 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7a4102910 0x7fd7a4197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:40.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.830+0000 7fd7a2d9d640 1 -- 172.21.15.160:0/3448661233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7a40ff940 msgr2=0x7fd7a41979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:40.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.830+0000 7fd7a2d9d640 1 --2- 172.21.15.160:0/3448661233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7a40ff940 0x7fd7a41979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.830+0000 7fd7a2d9d640 1 -- 172.21.15.160:0/3448661233 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7a4102910 msgr2=0x7fd7a4197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:40.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.830+0000 7fd7a2d9d640 1 --2- 172.21.15.160:0/3448661233 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7a4102910 0x7fd7a4197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.830+0000 7fd7a2d9d640 1 -- 172.21.15.160:0/3448661233 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a419f610 con 0x7fd7a41032d0 2024-03-07T07:40:40.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.831+0000 7fd7a2d9d640 1 --2- 172.21.15.160:0/3448661233 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7a41032d0 0x7fd7a419efa0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd79c00eb10 tx=0x7fd79c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:40.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.831+0000 7fd7937fe640 1 -- 172.21.15.160:0/3448661233 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79c004b20 con 0x7fd7a41032d0 2024-03-07T07:40:40.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.831+0000 7fd7a8ad0640 1 -- 172.21.15.160:0/3448661233 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7a419f900 con 0x7fd7a41032d0 2024-03-07T07:40:40.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.831+0000 7fd7937fe640 1 -- 172.21.15.160:0/3448661233 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd79c004cc0 con 0x7fd7a41032d0 2024-03-07T07:40:40.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.831+0000 7fd7937fe640 1 -- 172.21.15.160:0/3448661233 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79c010760 con 0x7fd7a41032d0 2024-03-07T07:40:40.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.831+0000 7fd7a8ad0640 1 -- 172.21.15.160:0/3448661233 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7a419fe50 con 0x7fd7a41032d0 2024-03-07T07:40:40.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.833+0000 7fd7937fe640 1 -- 172.21.15.160:0/3448661233 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 136754+0+0 (secure 0 0 0) 0x7fd79c005020 con 0x7fd7a41032d0 2024-03-07T07:40:40.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.833+0000 7fd7ab98d740 1 Processor -- start 2024-03-07T07:40:40.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.833+0000 7fd7ab98d740 1 -- start start 2024-03-07T07:40:40.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.834+0000 7fd7ab98d740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561d4ca640b0 0x561d4ca644d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:40.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.834+0000 7fd7937fe640 1 --2- 172.21.15.160:0/3448661233 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fd78c0a1d80 0x7fd78c0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:40.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.834+0000 7fd7ab98d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561d4ca64aa0 0x561d4ca695a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:40.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.834+0000 7fd7ab98d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d4ca69c10 0x561d4ca6c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:40.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.834+0000 7fd7ab98d740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d4c9c9790 con 0x561d4ca640b0 2024-03-07T07:40:40.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.834+0000 7fd7ab98d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d4c972b20 con 0x561d4ca69c10 2024-03-07T07:40:40.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.834+0000 7fd7ab98d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d4c9c5830 con 0x561d4ca64aa0 2024-03-07T07:40:40.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.834+0000 7fd7a1d9b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561d4ca640b0 0x561d4ca644d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:40.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.834+0000 7fd7a259c640 1 --2- 172.21.15.160:0/3448661233 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fd78c0a1d80 0x7fd78c0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:40.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.834+0000 7fd7a1d9b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561d4ca640b0 0x561d4ca644d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59118/0 (socket says 172.21.15.160:59118) 2024-03-07T07:40:40.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.834+0000 7fd7a1d9b640 1 -- 172.21.15.160:0/1109206964 learned_addr learned my addr 172.21.15.160:0/1109206964 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:40:40.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.834+0000 7fd7937fe640 1 -- 172.21.15.160:0/3448661233 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd79c0ce360 con 0x7fd7a41032d0 2024-03-07T07:40:40.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.834+0000 7fd7a2d9d640 1 --2- 172.21.15.160:0/1109206964 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d4ca69c10 0x561d4ca6c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:40.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.834+0000 7fd7a1d9b640 1 --2- 172.21.15.160:0/1109206964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561d4ca64aa0 0x561d4ca695a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:40.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.834+0000 7fd7a2d9d640 1 -- 172.21.15.160:0/1109206964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561d4ca64aa0 msgr2=0x561d4ca695a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:40:40.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.834+0000 7fd7a2d9d640 1 --2- 172.21.15.160:0/1109206964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561d4ca64aa0 0x561d4ca695a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.834+0000 7fd7a2d9d640 1 -- 172.21.15.160:0/1109206964 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561d4ca640b0 msgr2=0x561d4ca644d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:40:40.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.834+0000 7fd7a2d9d640 1 --2- 172.21.15.160:0/1109206964 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561d4ca640b0 0x561d4ca644d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.834+0000 7fd7a2d9d640 1 -- 172.21.15.160:0/1109206964 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d4c986970 con 0x561d4ca69c10 2024-03-07T07:40:40.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.834+0000 7fd7a1d9b640 1 --2- 172.21.15.160:0/1109206964 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561d4ca640b0 0x561d4ca644d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:40:40.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.834+0000 7fd7a259c640 1 --2- 172.21.15.160:0/3448661233 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fd78c0a1d80 0x7fd78c0a4240 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd79402c7b0 tx=0x7fd794000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:40.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.834+0000 7fd7a2d9d640 1 --2- 172.21.15.160:0/1109206964 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d4ca69c10 0x561d4ca6c0a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd79c0049d0 tx=0x7fd79c09abd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb70562d8a95f54a server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:40.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.834+0000 7fd790ff9640 1 -- 172.21.15.160:0/1109206964 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79c0b3dc0 con 0x561d4ca69c10 2024-03-07T07:40:40.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.834+0000 7fd790ff9640 1 -- 172.21.15.160:0/1109206964 <== mon.2 v2:172.21.15.177:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd79c09b790 con 0x561d4ca69c10 2024-03-07T07:40:40.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.834+0000 7fd790ff9640 1 -- 172.21.15.160:0/1109206964 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79c0b4180 con 0x561d4ca69c10 2024-03-07T07:40:40.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.835+0000 7fd7ab98d740 1 -- 172.21.15.160:0/1109206964 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d4ca69c10 msgr2=0x561d4ca6c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:40.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.835+0000 7fd7ab98d740 1 --2- 172.21.15.160:0/1109206964 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d4ca69c10 0x561d4ca6c0a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd79c0049d0 tx=0x7fd79c09abd0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.835+0000 7fd7ab98d740 1 -- 172.21.15.160:0/1109206964 shutdown_connections 2024-03-07T07:40:40.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.835+0000 7fd7ab98d740 1 --2- 172.21.15.160:0/1109206964 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d4ca69c10 0x561d4ca6c0a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.835+0000 7fd7ab98d740 1 --2- 172.21.15.160:0/1109206964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561d4ca64aa0 0x561d4ca695a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.835+0000 7fd7ab98d740 1 --2- 172.21.15.160:0/1109206964 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561d4ca640b0 0x561d4ca644d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.835+0000 7fd7ab98d740 1 -- 172.21.15.160:0/1109206964 >> 172.21.15.160:0/1109206964 conn(0x561d4ca2c3d0 msgr2=0x561d4ca5fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:40.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.835+0000 7fd7ab98d740 1 -- 172.21.15.160:0/1109206964 shutdown_connections 2024-03-07T07:40:40.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.835+0000 7fd7ab98d740 1 -- 172.21.15.160:0/1109206964 wait complete. 2024-03-07T07:40:40.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.835+0000 7fd7ab98d740 1 -- start start 2024-03-07T07:40:40.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.836+0000 7fd7ab98d740 1 Processor -- start 2024-03-07T07:40:40.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.836+0000 7fd7ab98d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561d4ca640b0 0x561d4ca6e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:40.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.836+0000 7fd7ab98d740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561d4ca64aa0 0x561d4ca6eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:40.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.836+0000 7fd7ab98d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d4ca69c10 0x561d4ca75ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:40.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.836+0000 7fd7ab98d740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d4ca6cb60 con 0x561d4ca64aa0 2024-03-07T07:40:40.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.836+0000 7fd7ab98d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d4c985f60 con 0x561d4ca69c10 2024-03-07T07:40:40.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.836+0000 7fd7ab98d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d4ca6cce0 con 0x561d4ca640b0 2024-03-07T07:40:40.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.836+0000 7fd7ab98d740 10 client.0 _ll_register_callbacks cb 0x561d4c9c13d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-07T07:40:40.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.836+0000 7fd7a1d9b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561d4ca64aa0 0x561d4ca6eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:40.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.836+0000 7fd7a1d9b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561d4ca64aa0 0x561d4ca6eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59130/0 (socket says 172.21.15.160:59130) 2024-03-07T07:40:40.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.836+0000 7fd7a1d9b640 1 -- 172.21.15.160:0/3990328286 learned_addr learned my addr 172.21.15.160:0/3990328286 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:40:40.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.836+0000 7fd7a259c640 1 --2- 172.21.15.160:0/3990328286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561d4ca640b0 0x561d4ca6e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:40.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.836+0000 7fd7a2d9d640 1 --2- 172.21.15.160:0/3990328286 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d4ca69c10 0x561d4ca75ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:40.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.836+0000 7fd7a1d9b640 1 -- 172.21.15.160:0/3990328286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561d4ca640b0 msgr2=0x561d4ca6e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:40:40.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.836+0000 7fd7a1d9b640 1 --2- 172.21.15.160:0/3990328286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561d4ca640b0 0x561d4ca6e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.836+0000 7fd7a1d9b640 1 -- 172.21.15.160:0/3990328286 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d4ca69c10 msgr2=0x561d4ca75ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:40:40.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.836+0000 7fd7a1d9b640 1 --2- 172.21.15.160:0/3990328286 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d4ca69c10 0x561d4ca75ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.836+0000 7fd7a1d9b640 1 -- 172.21.15.160:0/3990328286 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561d4c6de2d0 con 0x561d4ca64aa0 2024-03-07T07:40:40.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.836+0000 7fd7a1d9b640 1 --2- 172.21.15.160:0/3990328286 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561d4ca64aa0 0x561d4ca6eaa0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd798013a20 tx=0x7fd798013ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0d48ed0e28a0495 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:40.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.836+0000 7fd789ffb640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-03-07T07:40:40.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.836+0000 7fd789ffb640 1 -- 172.21.15.160:0/3990328286 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd798004300 con 0x561d4ca64aa0 2024-03-07T07:40:40.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.836+0000 7fd7ab98d740 1 -- 172.21.15.160:0/3990328286 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd79c09b790 con 0x561d4ca64aa0 2024-03-07T07:40:40.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.836+0000 7fd789ffb640 1 -- 172.21.15.160:0/3990328286 <== mon.0 v2:172.21.15.160:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd7980109b0 con 0x561d4ca64aa0 2024-03-07T07:40:40.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.837+0000 7fd789ffb640 1 -- 172.21.15.160:0/3990328286 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd798027050 con 0x561d4ca64aa0 2024-03-07T07:40:40.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.837+0000 7fd7ab98d740 10 client.15045 fetch_fsmap learned FSMap version 21 2024-03-07T07:40:40.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.837+0000 7fd7ab98d740 1 -- 172.21.15.160:0/3990328286 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x561d4c7a86b0 con 0x561d4ca64aa0 2024-03-07T07:40:40.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.837+0000 7fd789ffb640 1 -- 172.21.15.160:0/3990328286 <== mon.0 v2:172.21.15.160:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd79c09b790 con 0x561d4ca64aa0 2024-03-07T07:40:40.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.837+0000 7fd789ffb640 1 -- 172.21.15.160:0/3990328286 <== mon.0 v2:172.21.15.160:3300/0 5 ==== fsmap(e 21) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fd7980047a0 con 0x561d4ca64aa0 2024-03-07T07:40:40.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.837+0000 7fd7ab98d740 10 client.15045 fetch_fsmap finished waiting for FSMap version 21 2024-03-07T07:40:40.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.837+0000 7fd7ab98d740 10 client.15045 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-07T07:40:40.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.837+0000 7fd7ab98d740 20 client.15045 populate_metadata read hostname 'smithi160' 2024-03-07T07:40:40.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.837+0000 7fd7ab98d740 1 --2- 172.21.15.160:0/3990328286 >> [v2:172.21.15.178:6836/1247829154,v1:172.21.15.178:6837/1247829154] conn(0x561d4ca7b1a0 0x561d4ca7a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:40.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.837+0000 7fd7ab98d740 4 client.15045 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-07T07:40:40.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.837+0000 7fd7ab98d740 1 -- 172.21.15.160:0/3990328286 --> [v2:172.21.15.178:6836/1247829154,v1:172.21.15.178:6837/1247829154] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561d4c9d6cd0 con 0x561d4ca7b1a0 2024-03-07T07:40:40.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.837+0000 7fd7a259c640 1 --2- 172.21.15.160:0/3990328286 >> [v2:172.21.15.178:6836/1247829154,v1:172.21.15.178:6837/1247829154] conn(0x561d4ca7b1a0 0x561d4ca7a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:40.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.838+0000 7fd789ffb640 10 client.15045 ms_handle_connect on v2:172.21.15.178:6836/1247829154 2024-03-07T07:40:40.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.838+0000 7fd7a259c640 1 --2- 172.21.15.160:0/3990328286 >> [v2:172.21.15.178:6836/1247829154,v1:172.21.15.178:6837/1247829154] conn(0x561d4ca7b1a0 0x561d4ca7a160 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=26e64b1c1701b54b server_cookie=375ea650f0784282 in_seq=0 out_seq=0 2024-03-07T07:40:40.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.838+0000 7fd789ffb640 1 -- 172.21.15.160:0/3990328286 <== mds.2 v2:172.21.15.178:6836/1247829154 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561d4c9d6cd0 con 0x561d4ca7b1a0 2024-03-07T07:40:40.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.838+0000 7fd789ffb640 10 client.15045 handle_command_reply: tid=0 2024-03-07T07:40:40.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.838+0000 7fd7ab98d740 1 client.15045 shutdown 2024-03-07T07:40:40.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.839+0000 7fd7ab98d740 1 -- 172.21.15.160:0/3990328286 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561d4ca64aa0 msgr2=0x561d4ca6eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:40.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.839+0000 7fd7ab98d740 1 --2- 172.21.15.160:0/3990328286 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561d4ca64aa0 0x561d4ca6eaa0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd798013a20 tx=0x7fd798013ee0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.839+0000 7fd7ab98d740 1 -- 172.21.15.160:0/3990328286 shutdown_connections 2024-03-07T07:40:40.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.839+0000 7fd7ab98d740 1 --2- 172.21.15.160:0/3990328286 >> [v2:172.21.15.178:6836/1247829154,v1:172.21.15.178:6837/1247829154] conn(0x561d4ca7b1a0 0x561d4ca7a160 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.839+0000 7fd7ab98d740 1 --2- 172.21.15.160:0/3990328286 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d4ca69c10 0x561d4ca75ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.839+0000 7fd789ffb640 0 client.15045 ms_handle_reset on v2:172.21.15.178:6836/1247829154 2024-03-07T07:40:40.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.839+0000 7fd7ab98d740 1 --2- 172.21.15.160:0/3990328286 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561d4ca64aa0 0x561d4ca6eaa0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.839+0000 7fd7ab98d740 1 --2- 172.21.15.160:0/3990328286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561d4ca640b0 0x561d4ca6e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.839+0000 7fd7ab98d740 1 -- 172.21.15.160:0/3990328286 >> 172.21.15.160:0/3990328286 conn(0x561d4ca2c3d0 msgr2=0x561d4ca5ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:40.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.839+0000 7fd7ab98d740 1 -- 172.21.15.160:0/3990328286 shutdown_connections 2024-03-07T07:40:40.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.839+0000 7fd7ab98d740 1 -- 172.21.15.160:0/3990328286 wait complete. 2024-03-07T07:40:40.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.839+0000 7fd7ab98d740 20 client.15045 trim_cache size 0 max 16384 2024-03-07T07:40:40.860 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:40 smithi177 ceph-mon[36578]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-07T07:40:40.860 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:40 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/3674181198' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-07T07:40:40.860 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:40 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/1438026892' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-07T07:40:40.860 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:40 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/692349172' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:40:40.861 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:40 smithi177 ceph-mon[36578]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-03-07T07:40:40.861 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:40 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/1466886991' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:40:40.861 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.855+0000 7fd7ab98d740 1 Processor -- start 2024-03-07T07:40:40.861 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.855+0000 7fd7ab98d740 1 -- start start 2024-03-07T07:40:40.862 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.855+0000 7fd7ab98d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561d4ca2c3d0 0x561d4ca6d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:40.862 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.855+0000 7fd7ab98d740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561d4ca640b0 0x561d4ca75aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:40.862 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.855+0000 7fd7ab98d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d4ca64aa0 0x561d4ca6dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:40.862 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.855+0000 7fd7ab98d740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d4ca7a9c0 con 0x561d4ca640b0 2024-03-07T07:40:40.862 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.855+0000 7fd7ab98d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d4ca7b020 con 0x561d4ca64aa0 2024-03-07T07:40:40.862 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.855+0000 7fd7ab98d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d4ca7b680 con 0x561d4ca2c3d0 2024-03-07T07:40:40.862 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.855+0000 7fd7a1d9b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561d4ca640b0 0x561d4ca75aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:40.862 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.855+0000 7fd7a1d9b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561d4ca640b0 0x561d4ca75aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59134/0 (socket says 172.21.15.160:59134) 2024-03-07T07:40:40.862 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.855+0000 7fd7a1d9b640 1 -- 172.21.15.160:0/251898846 learned_addr learned my addr 172.21.15.160:0/251898846 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:40:40.862 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.855+0000 7fd7a259c640 1 --2- 172.21.15.160:0/251898846 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561d4ca2c3d0 0x561d4ca6d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:40.862 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.855+0000 7fd7a2d9d640 1 --2- 172.21.15.160:0/251898846 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d4ca64aa0 0x561d4ca6dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:40.863 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.856+0000 7fd7a1d9b640 1 -- 172.21.15.160:0/251898846 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561d4ca2c3d0 msgr2=0x561d4ca6d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:40:40.863 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.856+0000 7fd7a1d9b640 1 --2- 172.21.15.160:0/251898846 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561d4ca2c3d0 0x561d4ca6d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.863 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.856+0000 7fd7a1d9b640 1 -- 172.21.15.160:0/251898846 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d4ca64aa0 msgr2=0x561d4ca6dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:40:40.863 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.856+0000 7fd7a1d9b640 1 --2- 172.21.15.160:0/251898846 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d4ca64aa0 0x561d4ca6dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.863 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.856+0000 7fd7a1d9b640 1 -- 172.21.15.160:0/251898846 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d4ca6caa0 con 0x561d4ca640b0 2024-03-07T07:40:40.863 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.856+0000 7fd7a1d9b640 1 --2- 172.21.15.160:0/251898846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561d4ca640b0 0x561d4ca75aa0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd7980108c0 tx=0x7fd798011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b74a569405e41d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:40.863 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.856+0000 7fd78bfff640 1 -- 172.21.15.160:0/251898846 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd798011d40 con 0x561d4ca640b0 2024-03-07T07:40:40.863 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.856+0000 7fd78bfff640 1 -- 172.21.15.160:0/251898846 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd7980047e0 con 0x561d4ca640b0 2024-03-07T07:40:40.863 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.856+0000 7fd78bfff640 1 -- 172.21.15.160:0/251898846 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd798003c30 con 0x561d4ca640b0 2024-03-07T07:40:40.863 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.856+0000 7fd7ab98d740 1 -- 172.21.15.160:0/251898846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561d4ca640b0 msgr2=0x561d4ca75aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:40.863 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.856+0000 7fd7ab98d740 1 --2- 172.21.15.160:0/251898846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561d4ca640b0 0x561d4ca75aa0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd7980108c0 tx=0x7fd798011b50 comp rx=0 tx=0).stop 2024-03-07T07:40:40.863 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.856+0000 7fd7ab98d740 1 -- 172.21.15.160:0/251898846 shutdown_connections 2024-03-07T07:40:40.863 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.856+0000 7fd7ab98d740 1 --2- 172.21.15.160:0/251898846 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d4ca64aa0 0x561d4ca6dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.863 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.856+0000 7fd7ab98d740 1 --2- 172.21.15.160:0/251898846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561d4ca640b0 0x561d4ca75aa0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.864 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.856+0000 7fd7ab98d740 1 --2- 172.21.15.160:0/251898846 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561d4ca2c3d0 0x561d4ca6d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.864 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.856+0000 7fd7ab98d740 1 -- 172.21.15.160:0/251898846 >> 172.21.15.160:0/251898846 conn(0x561d4ca7a4d0 msgr2=0x561d4ca65f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:40.864 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.857+0000 7fd7ab98d740 1 -- 172.21.15.160:0/251898846 shutdown_connections 2024-03-07T07:40:40.864 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.857+0000 7fd7ab98d740 1 -- 172.21.15.160:0/251898846 wait complete. 2024-03-07T07:40:40.864 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.857+0000 7fd7ab98d740 1 -- start start 2024-03-07T07:40:40.864 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.857+0000 7fd7ab98d740 1 Processor -- start 2024-03-07T07:40:40.864 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.857+0000 7fd7ab98d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d4ca2c3d0 0x561d4ca75af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:40.864 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.857+0000 7fd7ab98d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561d4ca640b0 0x561d4ca72650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:40.864 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.857+0000 7fd7ab98d740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561d4ca64aa0 0x561d4ca72c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:40.864 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.857+0000 7fd7ab98d740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d4ca7c340 con 0x561d4ca64aa0 2024-03-07T07:40:40.864 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.857+0000 7fd7ab98d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d4ca7bce0 con 0x561d4ca2c3d0 2024-03-07T07:40:40.864 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.857+0000 7fd7ab98d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d4ca7c9a0 con 0x561d4ca640b0 2024-03-07T07:40:40.864 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.857+0000 7fd7ab98d740 10 client.0 _ll_register_callbacks cb 0x7fd798013660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-07T07:40:40.864 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.857+0000 7fd7a2d9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561d4ca64aa0 0x561d4ca72c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:40.865 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.857+0000 7fd7a2d9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561d4ca64aa0 0x561d4ca72c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59146/0 (socket says 172.21.15.160:59146) 2024-03-07T07:40:40.865 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.857+0000 7fd7a2d9d640 1 -- 172.21.15.160:0/2631094785 learned_addr learned my addr 172.21.15.160:0/2631094785 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:40:40.865 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.857+0000 7fd7a1d9b640 1 --2- 172.21.15.160:0/2631094785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561d4ca640b0 0x561d4ca72650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:40.865 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.857+0000 7fd7a259c640 1 --2- 172.21.15.160:0/2631094785 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d4ca2c3d0 0x561d4ca75af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:40.865 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.857+0000 7fd7a2d9d640 1 -- 172.21.15.160:0/2631094785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561d4ca640b0 msgr2=0x561d4ca72650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:40:40.865 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.857+0000 7fd7a2d9d640 1 --2- 172.21.15.160:0/2631094785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561d4ca640b0 0x561d4ca72650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.865 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.857+0000 7fd7a2d9d640 1 -- 172.21.15.160:0/2631094785 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d4ca2c3d0 msgr2=0x561d4ca75af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:40:40.865 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.857+0000 7fd7a2d9d640 1 --2- 172.21.15.160:0/2631094785 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d4ca2c3d0 0x561d4ca75af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.865 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.857+0000 7fd7a2d9d640 1 -- 172.21.15.160:0/2631094785 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561d4cadeac0 con 0x561d4ca64aa0 2024-03-07T07:40:40.865 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.858+0000 7fd7a2d9d640 1 --2- 172.21.15.160:0/2631094785 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561d4ca64aa0 0x561d4ca72c80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd79c009270 tx=0x7fd79c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a49177ed026da781 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:40.865 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.858+0000 7fd789ffb640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-03-07T07:40:40.865 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.858+0000 7fd789ffb640 1 -- 172.21.15.160:0/2631094785 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79c064020 con 0x561d4ca64aa0 2024-03-07T07:40:40.865 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.858+0000 7fd789ffb640 1 -- 172.21.15.160:0/2631094785 <== mon.0 v2:172.21.15.160:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd79c0a35b0 con 0x561d4ca64aa0 2024-03-07T07:40:40.866 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.858+0000 7fd7ab98d740 1 -- 172.21.15.160:0/2631094785 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd7980047e0 con 0x561d4ca64aa0 2024-03-07T07:40:40.866 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.858+0000 7fd789ffb640 1 -- 172.21.15.160:0/2631094785 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd79c07e050 con 0x561d4ca64aa0 2024-03-07T07:40:40.866 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.858+0000 7fd7ab98d740 10 client.15051 fetch_fsmap learned FSMap version 21 2024-03-07T07:40:40.866 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.858+0000 7fd7ab98d740 1 -- 172.21.15.160:0/2631094785 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x561d4cadfdc0 con 0x561d4ca64aa0 2024-03-07T07:40:40.866 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.858+0000 7fd789ffb640 1 -- 172.21.15.160:0/2631094785 <== mon.0 v2:172.21.15.160:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd7980047e0 con 0x561d4ca64aa0 2024-03-07T07:40:40.866 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.858+0000 7fd789ffb640 1 -- 172.21.15.160:0/2631094785 <== mon.0 v2:172.21.15.160:3300/0 5 ==== fsmap(e 21) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fd79c005310 con 0x561d4ca64aa0 2024-03-07T07:40:40.866 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.858+0000 7fd7ab98d740 10 client.15051 fetch_fsmap finished waiting for FSMap version 21 2024-03-07T07:40:40.866 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.858+0000 7fd7ab98d740 10 client.15051 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-07T07:40:40.866 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.858+0000 7fd7ab98d740 20 client.15051 populate_metadata read hostname 'smithi160' 2024-03-07T07:40:40.866 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.858+0000 7fd7ab98d740 1 --2- 172.21.15.160:0/2631094785 >> [v2:172.21.15.178:6836/1247829154,v1:172.21.15.178:6837/1247829154] conn(0x561d4ca69c10 0x561d4cadff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:40.866 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.858+0000 7fd7ab98d740 4 client.15051 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-07T07:40:40.866 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.858+0000 7fd7ab98d740 1 -- 172.21.15.160:0/2631094785 --> [v2:172.21.15.178:6836/1247829154,v1:172.21.15.178:6837/1247829154] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561d4c9d1020 con 0x561d4ca69c10 2024-03-07T07:40:40.866 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.858+0000 7fd7a259c640 1 --2- 172.21.15.160:0/2631094785 >> [v2:172.21.15.178:6836/1247829154,v1:172.21.15.178:6837/1247829154] conn(0x561d4ca69c10 0x561d4cadff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:40.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.859+0000 7fd789ffb640 10 client.15051 ms_handle_connect on v2:172.21.15.178:6836/1247829154 2024-03-07T07:40:40.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.859+0000 7fd7a259c640 1 --2- 172.21.15.160:0/2631094785 >> [v2:172.21.15.178:6836/1247829154,v1:172.21.15.178:6837/1247829154] conn(0x561d4ca69c10 0x561d4cadff90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=25bed7ec91a919e1 server_cookie=156e8455bfaa9a7f in_seq=0 out_seq=0 2024-03-07T07:40:40.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.859+0000 7fd789ffb640 1 -- 172.21.15.160:0/2631094785 <== mds.2 v2:172.21.15.178:6836/1247829154 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x561d4c9d1020 con 0x561d4ca69c10 2024-03-07T07:40:40.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.859+0000 7fd789ffb640 10 client.15051 handle_command_reply: tid=0 2024-03-07T07:40:40.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.859+0000 7fd7ab98d740 1 client.15051 shutdown 2024-03-07T07:40:40.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.859+0000 7fd7ab98d740 1 -- 172.21.15.160:0/2631094785 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561d4ca64aa0 msgr2=0x561d4ca72c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:40.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.859+0000 7fd7ab98d740 1 --2- 172.21.15.160:0/2631094785 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561d4ca64aa0 0x561d4ca72c80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd79c009270 tx=0x7fd79c005840 comp rx=0 tx=0).stop 2024-03-07T07:40:40.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.859+0000 7fd7ab98d740 1 -- 172.21.15.160:0/2631094785 shutdown_connections 2024-03-07T07:40:40.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.859+0000 7fd7ab98d740 1 --2- 172.21.15.160:0/2631094785 >> [v2:172.21.15.178:6836/1247829154,v1:172.21.15.178:6837/1247829154] conn(0x561d4ca69c10 0x561d4cadff90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.859+0000 7fd7ab98d740 1 --2- 172.21.15.160:0/2631094785 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561d4ca64aa0 0x561d4ca72c80 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.859+0000 7fd7ab98d740 1 --2- 172.21.15.160:0/2631094785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561d4ca640b0 0x561d4ca72650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.859+0000 7fd7ab98d740 1 --2- 172.21.15.160:0/2631094785 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d4ca2c3d0 0x561d4ca75af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.859+0000 7fd789ffb640 0 client.15051 ms_handle_reset on v2:172.21.15.178:6836/1247829154 2024-03-07T07:40:40.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.859+0000 7fd7ab98d740 1 -- 172.21.15.160:0/2631094785 >> 172.21.15.160:0/2631094785 conn(0x561d4ca7a4d0 msgr2=0x561d4ca6d830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:40.868 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.859+0000 7fd7ab98d740 1 -- 172.21.15.160:0/2631094785 shutdown_connections 2024-03-07T07:40:40.868 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.859+0000 7fd7ab98d740 1 -- 172.21.15.160:0/2631094785 wait complete. 2024-03-07T07:40:40.868 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.859+0000 7fd7ab98d740 20 client.15051 trim_cache size 0 max 16384 2024-03-07T07:40:40.868 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-03-07T07:40:40.868 INFO:teuthology.orchestra.run.smithi160.stdout: "cluster_fsid": "826f6582-dc54-11ee-95c6-87774f69a715", 2024-03-07T07:40:40.868 INFO:teuthology.orchestra.run.smithi160.stdout: "whoami": 2, 2024-03-07T07:40:40.868 INFO:teuthology.orchestra.run.smithi160.stdout: "id": 24487, 2024-03-07T07:40:40.868 INFO:teuthology.orchestra.run.smithi160.stdout: "want_state": "up:active", 2024-03-07T07:40:40.868 INFO:teuthology.orchestra.run.smithi160.stdout: "state": "up:active", 2024-03-07T07:40:40.868 INFO:teuthology.orchestra.run.smithi160.stdout: "fs_name": "cephfs", 2024-03-07T07:40:40.869 INFO:teuthology.orchestra.run.smithi160.stdout: "rank_uptime": 83.006578332000004, 2024-03-07T07:40:40.869 INFO:teuthology.orchestra.run.smithi160.stdout: "mdsmap_epoch": 21, 2024-03-07T07:40:40.869 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_epoch": 87, 2024-03-07T07:40:40.869 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_epoch_barrier": 85, 2024-03-07T07:40:40.869 INFO:teuthology.orchestra.run.smithi160.stdout: "uptime": 107.176865975 2024-03-07T07:40:40.869 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-03-07T07:40:40.870 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.860+0000 7fd7a8ad0640 1 -- 172.21.15.160:0/3448661233 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fd78c0a1d80 msgr2=0x7fd78c0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:40.870 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.860+0000 7fd7a8ad0640 1 --2- 172.21.15.160:0/3448661233 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fd78c0a1d80 0x7fd78c0a4240 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd79402c7b0 tx=0x7fd794000bc0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.870 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.860+0000 7fd7a8ad0640 1 -- 172.21.15.160:0/3448661233 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7a41032d0 msgr2=0x7fd7a419efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:40.870 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.860+0000 7fd7a8ad0640 1 --2- 172.21.15.160:0/3448661233 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7a41032d0 0x7fd7a419efa0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd79c00eb10 tx=0x7fd79c00c2d0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.870 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.860+0000 7fd7a8ad0640 1 -- 172.21.15.160:0/3448661233 shutdown_connections 2024-03-07T07:40:40.870 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.860+0000 7fd7a8ad0640 1 --2- 172.21.15.160:0/3448661233 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fd78c0a1d80 0x7fd78c0a4240 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.870 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.860+0000 7fd7a8ad0640 1 --2- 172.21.15.160:0/3448661233 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7a41032d0 0x7fd7a419efa0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.870 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.860+0000 7fd7a8ad0640 1 --2- 172.21.15.160:0/3448661233 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7a4102910 0x7fd7a4197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.870 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.860+0000 7fd7a8ad0640 1 --2- 172.21.15.160:0/3448661233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7a40ff940 0x7fd7a41979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.870 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.860+0000 7fd7a8ad0640 1 -- 172.21.15.160:0/3448661233 >> 172.21.15.160:0/3448661233 conn(0x7fd7a40fb250 msgr2=0x7fd7a4105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:40.870 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.860+0000 7fd7a8ad0640 1 -- 172.21.15.160:0/3448661233 shutdown_connections 2024-03-07T07:40:40.870 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.860+0000 7fd7a8ad0640 1 -- 172.21.15.160:0/3448661233 wait complete. 2024-03-07T07:40:40.872 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-07T07:40:40.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.975+0000 7f44f7beb640 1 -- 172.21.15.160:0/473380040 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44f0101770 msgr2=0x7f44f0101bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:40.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.975+0000 7f44f7beb640 1 --2- 172.21.15.160:0/473380040 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44f0101770 0x7f44f0101bf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f44ec009900 tx=0x7f44ec02bc30 comp rx=0 tx=0).stop 2024-03-07T07:40:40.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.975+0000 7f44f7beb640 1 -- 172.21.15.160:0/473380040 shutdown_connections 2024-03-07T07:40:40.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.975+0000 7f44f7beb640 1 --2- 172.21.15.160:0/473380040 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f44f0102130 0x7f44f01089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.975+0000 7f44f7beb640 1 --2- 172.21.15.160:0/473380040 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44f0101770 0x7f44f0101bf0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.975+0000 7f44f7beb640 1 --2- 172.21.15.160:0/473380040 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f44f0100580 0x7f44f01009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.975+0000 7f44f7beb640 1 -- 172.21.15.160:0/473380040 >> 172.21.15.160:0/473380040 conn(0x7f44f00fbb10 msgr2=0x7f44f00fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:40.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.975+0000 7f44f7beb640 1 -- 172.21.15.160:0/473380040 shutdown_connections 2024-03-07T07:40:40.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.975+0000 7f44f7beb640 1 -- 172.21.15.160:0/473380040 wait complete. 2024-03-07T07:40:40.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.975+0000 7f44f7beb640 1 Processor -- start 2024-03-07T07:40:40.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.975+0000 7f44f7beb640 1 -- start start 2024-03-07T07:40:40.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.975+0000 7f44f7beb640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44f0100580 0x7f44f0197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:40.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.975+0000 7f44f7beb640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f44f0101770 0x7f44f0197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:40.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.975+0000 7f44f7beb640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f44f0102130 0x7f44f019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:40.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.975+0000 7f44f7beb640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f010c210 con 0x7f44f0101770 2024-03-07T07:40:40.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.975+0000 7f44f7beb640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f010c090 con 0x7f44f0102130 2024-03-07T07:40:40.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.975+0000 7f44f7beb640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f010c390 con 0x7f44f0100580 2024-03-07T07:40:40.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.976+0000 7f44f5186640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f44f0101770 0x7f44f0197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:40.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.976+0000 7f44f5186640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f44f0101770 0x7f44f0197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59168/0 (socket says 172.21.15.160:59168) 2024-03-07T07:40:40.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.976+0000 7f44f5186640 1 -- 172.21.15.160:0/777296806 learned_addr learned my addr 172.21.15.160:0/777296806 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:40:40.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.976+0000 7f44f5987640 1 --2- 172.21.15.160:0/777296806 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44f0100580 0x7f44f0197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:40.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.976+0000 7f44f6188640 1 --2- 172.21.15.160:0/777296806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f44f0102130 0x7f44f019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:40.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.976+0000 7f44f5186640 1 -- 172.21.15.160:0/777296806 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44f0100580 msgr2=0x7f44f0197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:40.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.976+0000 7f44f5186640 1 --2- 172.21.15.160:0/777296806 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44f0100580 0x7f44f0197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.976+0000 7f44f5186640 1 -- 172.21.15.160:0/777296806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f44f0102130 msgr2=0x7f44f019ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:40.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.976+0000 7f44f5186640 1 --2- 172.21.15.160:0/777296806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f44f0102130 0x7f44f019ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.980 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.976+0000 7f44f5186640 1 -- 172.21.15.160:0/777296806 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44f019f5c0 con 0x7f44f0101770 2024-03-07T07:40:40.980 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.976+0000 7f44f5987640 1 --2- 172.21.15.160:0/777296806 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44f0100580 0x7f44f0197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:40:40.980 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.976+0000 7f44f5186640 1 --2- 172.21.15.160:0/777296806 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f44f0101770 0x7f44f0197f50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f44ec009230 tx=0x7f44ec02c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:40.980 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.976+0000 7f44e6ffd640 1 -- 172.21.15.160:0/777296806 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ec041070 con 0x7f44f0101770 2024-03-07T07:40:40.980 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.976+0000 7f44f7beb640 1 -- 172.21.15.160:0/777296806 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44f019f850 con 0x7f44f0101770 2024-03-07T07:40:40.980 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.976+0000 7f44e6ffd640 1 -- 172.21.15.160:0/777296806 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f44ec004450 con 0x7f44f0101770 2024-03-07T07:40:40.980 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.976+0000 7f44e6ffd640 1 -- 172.21.15.160:0/777296806 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ec003e20 con 0x7f44f0101770 2024-03-07T07:40:40.980 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.976+0000 7f44f7beb640 1 -- 172.21.15.160:0/777296806 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44f019fd90 con 0x7f44f0101770 2024-03-07T07:40:40.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.978+0000 7f44e6ffd640 1 -- 172.21.15.160:0/777296806 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f44ec044070 con 0x7f44f0101770 2024-03-07T07:40:40.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.979+0000 7f44fa9d3740 1 Processor -- start 2024-03-07T07:40:40.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.979+0000 7f44fa9d3740 1 -- start start 2024-03-07T07:40:40.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.979+0000 7f44fa9d3740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56365c0320b0 0x56365c0324d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:40.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.979+0000 7f44fa9d3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56365c032aa0 0x56365c0375a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:40.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.979+0000 7f44fa9d3740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56365c037c10 0x56365c03a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:40.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.979+0000 7f44fa9d3740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56365bf97790 con 0x56365c037c10 2024-03-07T07:40:40.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.979+0000 7f44fa9d3740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x56365bf40b20 con 0x56365c0320b0 2024-03-07T07:40:40.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.979+0000 7f44fa9d3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x56365bf93830 con 0x56365c032aa0 2024-03-07T07:40:40.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.979+0000 7f44e6ffd640 1 --2- 172.21.15.160:0/777296806 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f44dc0a1d80 0x7f44dc0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:40.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.979+0000 7f44f5186640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56365c037c10 0x56365c03a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:40.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.979+0000 7f44f5186640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56365c037c10 0x56365c03a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59182/0 (socket says 172.21.15.160:59182) 2024-03-07T07:40:40.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.979+0000 7f44f5186640 1 -- 172.21.15.160:0/3153824198 learned_addr learned my addr 172.21.15.160:0/3153824198 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:40:40.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.979+0000 7f44f5987640 1 --2- 172.21.15.160:0/3153824198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56365c032aa0 0x56365c0375a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:40.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.979+0000 7f44f6188640 1 --2- 172.21.15.160:0/3153824198 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56365c0320b0 0x56365c0324d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:40.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.979+0000 7f44f6188640 1 --2- 172.21.15.160:0/777296806 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f44dc0a1d80 0x7f44dc0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:40.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.979+0000 7f44f5987640 1 -- 172.21.15.160:0/3153824198 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56365c0320b0 msgr2=0x56365c0324d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:40:40.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.979+0000 7f44f5987640 1 --2- 172.21.15.160:0/3153824198 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56365c0320b0 0x56365c0324d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.979+0000 7f44f5987640 1 -- 172.21.15.160:0/3153824198 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56365c037c10 msgr2=0x56365c03a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:40:40.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.979+0000 7f44f5987640 1 --2- 172.21.15.160:0/3153824198 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56365c037c10 0x56365c03a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.979+0000 7f44f5987640 1 -- 172.21.15.160:0/3153824198 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56365bf54970 con 0x56365c032aa0 2024-03-07T07:40:40.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.979+0000 7f44e6ffd640 1 -- 172.21.15.160:0/777296806 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f44ec0ef960 con 0x7f44f0101770 2024-03-07T07:40:40.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.980+0000 7f44f5987640 1 --2- 172.21.15.160:0/3153824198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56365c032aa0 0x56365c0375a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f44e0009fa0 tx=0x7f44e0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9be295f8027a66a server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:40.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.980+0000 7f44f6188640 1 --2- 172.21.15.160:0/777296806 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f44dc0a1d80 0x7f44dc0a4240 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x56365c033300 tx=0x7f44e8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:40.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.980+0000 7f44dbfff640 1 -- 172.21.15.160:0/3153824198 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e0004310 con 0x56365c032aa0 2024-03-07T07:40:40.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.980+0000 7f44dbfff640 1 -- 172.21.15.160:0/3153824198 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f44e00044b0 con 0x56365c032aa0 2024-03-07T07:40:40.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.980+0000 7f44dbfff640 1 -- 172.21.15.160:0/3153824198 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e0003850 con 0x56365c032aa0 2024-03-07T07:40:40.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.981+0000 7f44fa9d3740 1 -- 172.21.15.160:0/3153824198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56365c032aa0 msgr2=0x56365c0375a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:40.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.981+0000 7f44fa9d3740 1 --2- 172.21.15.160:0/3153824198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56365c032aa0 0x56365c0375a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f44e0009fa0 tx=0x7f44e0011810 comp rx=0 tx=0).stop 2024-03-07T07:40:40.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.981+0000 7f44fa9d3740 1 -- 172.21.15.160:0/3153824198 shutdown_connections 2024-03-07T07:40:40.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.981+0000 7f44fa9d3740 1 --2- 172.21.15.160:0/3153824198 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56365c037c10 0x56365c03a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.981+0000 7f44fa9d3740 1 --2- 172.21.15.160:0/3153824198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56365c032aa0 0x56365c0375a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.981+0000 7f44fa9d3740 1 --2- 172.21.15.160:0/3153824198 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56365c0320b0 0x56365c0324d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.981+0000 7f44fa9d3740 1 -- 172.21.15.160:0/3153824198 >> 172.21.15.160:0/3153824198 conn(0x56365bffa3d0 msgr2=0x56365c02da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:40.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.981+0000 7f44fa9d3740 1 -- 172.21.15.160:0/3153824198 shutdown_connections 2024-03-07T07:40:40.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.981+0000 7f44fa9d3740 1 -- 172.21.15.160:0/3153824198 wait complete. 2024-03-07T07:40:40.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.981+0000 7f44fa9d3740 1 -- start start 2024-03-07T07:40:40.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.981+0000 7f44fa9d3740 1 Processor -- start 2024-03-07T07:40:40.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.981+0000 7f44fa9d3740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56365c0320b0 0x56365c03c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:40.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.981+0000 7f44fa9d3740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56365c032aa0 0x56365c03cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:40.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.981+0000 7f44fa9d3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56365c037c10 0x56365c043b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:40.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.981+0000 7f44fa9d3740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56365c03abc0 con 0x56365c032aa0 2024-03-07T07:40:40.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.981+0000 7f44fa9d3740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x56365bf53f60 con 0x56365c0320b0 2024-03-07T07:40:40.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.981+0000 7f44fa9d3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x56365c03ad40 con 0x56365c037c10 2024-03-07T07:40:40.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.982+0000 7f44f5186640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56365c032aa0 0x56365c03cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:40.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.982+0000 7f44fa9d3740 10 client.0 _ll_register_callbacks cb 0x56365bf8f3d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-07T07:40:40.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.982+0000 7f44f5186640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56365c032aa0 0x56365c03cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59196/0 (socket says 172.21.15.160:59196) 2024-03-07T07:40:40.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.982+0000 7f44f6188640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56365c037c10 0x56365c043b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:40.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.982+0000 7f44f5987640 1 --2- 172.21.15.160:0/3694163746 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56365c0320b0 0x56365c03c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:40.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.982+0000 7f44f5186640 1 -- 172.21.15.160:0/3694163746 learned_addr learned my addr 172.21.15.160:0/3694163746 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:40:40.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.982+0000 7f44f5186640 1 -- 172.21.15.160:0/3694163746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56365c037c10 msgr2=0x56365c043b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:40:40.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.982+0000 7f44f5186640 1 --2- 172.21.15.160:0/3694163746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56365c037c10 0x56365c043b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.982+0000 7f44f5186640 1 -- 172.21.15.160:0/3694163746 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56365c0320b0 msgr2=0x56365c03c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:40:40.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.982+0000 7f44f5186640 1 --2- 172.21.15.160:0/3694163746 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56365c0320b0 0x56365c03c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.982+0000 7f44f5186640 1 -- 172.21.15.160:0/3694163746 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56365bcac2d0 con 0x56365c032aa0 2024-03-07T07:40:40.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.982+0000 7f44f6188640 1 --2- 172.21.15.160:0/3694163746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56365c037c10 0x56365c043b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:40:40.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.982+0000 7f44f5987640 1 --2- 172.21.15.160:0/3694163746 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56365c0320b0 0x56365c03c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:40:40.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.982+0000 7f44f5186640 1 --2- 172.21.15.160:0/3694163746 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56365c032aa0 0x56365c03cae0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f44ec0b8b50 tx=0x7f44ec0efc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f07ab71e4f5539f server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:40.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.982+0000 7f44d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-03-07T07:40:40.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.982+0000 7f44d97fa640 1 -- 172.21.15.160:0/3694163746 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ec04b070 con 0x56365c032aa0 2024-03-07T07:40:40.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.982+0000 7f44fa9d3740 1 -- 172.21.15.160:0/3694163746 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f44e00044b0 con 0x56365c032aa0 2024-03-07T07:40:40.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.982+0000 7f44d97fa640 1 -- 172.21.15.160:0/3694163746 <== mon.0 v2:172.21.15.160:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f44ec0ed570 con 0x56365c032aa0 2024-03-07T07:40:40.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.982+0000 7f44d97fa640 1 -- 172.21.15.160:0/3694163746 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44ec004c60 con 0x56365c032aa0 2024-03-07T07:40:40.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.982+0000 7f44fa9d3740 10 client.15063 fetch_fsmap learned FSMap version 21 2024-03-07T07:40:40.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.982+0000 7f44fa9d3740 1 -- 172.21.15.160:0/3694163746 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x56365bd766b0 con 0x56365c032aa0 2024-03-07T07:40:40.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.982+0000 7f44d97fa640 1 -- 172.21.15.160:0/3694163746 <== mon.0 v2:172.21.15.160:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f44e00044b0 con 0x56365c032aa0 2024-03-07T07:40:40.992 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.983+0000 7f44d97fa640 1 -- 172.21.15.160:0/3694163746 <== mon.0 v2:172.21.15.160:3300/0 5 ==== fsmap(e 21) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f44ec0b8db0 con 0x56365c032aa0 2024-03-07T07:40:40.992 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.983+0000 7f44fa9d3740 10 client.15063 fetch_fsmap finished waiting for FSMap version 21 2024-03-07T07:40:40.992 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.983+0000 7f44fa9d3740 10 client.15063 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-07T07:40:40.992 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.983+0000 7f44fa9d3740 20 client.15063 populate_metadata read hostname 'smithi160' 2024-03-07T07:40:40.992 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.983+0000 7f44fa9d3740 1 --2- 172.21.15.160:0/3694163746 >> [v2:172.21.15.177:6838/3329773838,v1:172.21.15.177:6839/3329773838] conn(0x56365c049270 0x56365c048010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:40.992 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.983+0000 7f44fa9d3740 4 client.15063 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-07T07:40:40.992 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.983+0000 7f44fa9d3740 1 -- 172.21.15.160:0/3694163746 --> [v2:172.21.15.177:6838/3329773838,v1:172.21.15.177:6839/3329773838] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56365bfa4cd0 con 0x56365c049270 2024-03-07T07:40:40.992 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.983+0000 7f44f5987640 1 --2- 172.21.15.160:0/3694163746 >> [v2:172.21.15.177:6838/3329773838,v1:172.21.15.177:6839/3329773838] conn(0x56365c049270 0x56365c048010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:40.992 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.983+0000 7f44f5987640 1 --2- 172.21.15.160:0/3694163746 >> [v2:172.21.15.177:6838/3329773838,v1:172.21.15.177:6839/3329773838] conn(0x56365c049270 0x56365c048010 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f7fad5e089f784e5 server_cookie=f0d94792ce72b2ae in_seq=0 out_seq=0 2024-03-07T07:40:40.992 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.983+0000 7f44d97fa640 10 client.15063 ms_handle_connect on v2:172.21.15.177:6838/3329773838 2024-03-07T07:40:40.992 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.984+0000 7f44d97fa640 1 -- 172.21.15.160:0/3694163746 <== mds.1 v2:172.21.15.177:6838/3329773838 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56365bfa4cd0 con 0x56365c049270 2024-03-07T07:40:40.992 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.984+0000 7f44d97fa640 10 client.15063 handle_command_reply: tid=0 2024-03-07T07:40:40.992 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.984+0000 7f44fa9d3740 1 client.15063 shutdown 2024-03-07T07:40:40.993 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.984+0000 7f44fa9d3740 1 -- 172.21.15.160:0/3694163746 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56365c032aa0 msgr2=0x56365c03cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:40.993 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.984+0000 7f44fa9d3740 1 --2- 172.21.15.160:0/3694163746 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56365c032aa0 0x56365c03cae0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f44ec0b8b50 tx=0x7f44ec0efc00 comp rx=0 tx=0).stop 2024-03-07T07:40:40.993 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.984+0000 7f44fa9d3740 1 -- 172.21.15.160:0/3694163746 shutdown_connections 2024-03-07T07:40:40.993 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.984+0000 7f44fa9d3740 1 --2- 172.21.15.160:0/3694163746 >> [v2:172.21.15.177:6838/3329773838,v1:172.21.15.177:6839/3329773838] conn(0x56365c049270 0x56365c048010 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.993 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.984+0000 7f44fa9d3740 1 --2- 172.21.15.160:0/3694163746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56365c037c10 0x56365c043b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.993 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.984+0000 7f44fa9d3740 1 --2- 172.21.15.160:0/3694163746 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56365c032aa0 0x56365c03cae0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.993 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.984+0000 7f44fa9d3740 1 --2- 172.21.15.160:0/3694163746 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56365c0320b0 0x56365c03c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:40.993 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.984+0000 7f44fa9d3740 1 -- 172.21.15.160:0/3694163746 >> 172.21.15.160:0/3694163746 conn(0x56365bffa3d0 msgr2=0x56365c02cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:40.993 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.984+0000 7f44d97fa640 0 client.15063 ms_handle_reset on v2:172.21.15.177:6838/3329773838 2024-03-07T07:40:40.993 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.984+0000 7f44fa9d3740 1 -- 172.21.15.160:0/3694163746 shutdown_connections 2024-03-07T07:40:40.993 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.984+0000 7f44fa9d3740 1 -- 172.21.15.160:0/3694163746 wait complete. 2024-03-07T07:40:40.993 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:40.984+0000 7f44fa9d3740 20 client.15063 trim_cache size 0 max 16384 2024-03-07T07:40:41.006 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.001+0000 7f44fa9d3740 1 Processor -- start 2024-03-07T07:40:41.006 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.001+0000 7f44fa9d3740 1 -- start start 2024-03-07T07:40:41.007 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.001+0000 7f44fa9d3740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56365c0320b0 0x56365c03bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:41.007 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.001+0000 7f44fa9d3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56365c032aa0 0x56365c043ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:41.007 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.001+0000 7f44fa9d3740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56365c037c10 0x56365c03a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:41.007 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.001+0000 7f44fa9d3740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56365c048a90 con 0x56365c0320b0 2024-03-07T07:40:41.007 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.001+0000 7f44fa9d3740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x56365c0490f0 con 0x56365c037c10 2024-03-07T07:40:41.007 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.001+0000 7f44fa9d3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x56365c049750 con 0x56365c032aa0 2024-03-07T07:40:41.007 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.001+0000 7f44f5987640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56365c0320b0 0x56365c03bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:41.007 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.001+0000 7f44f5987640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56365c0320b0 0x56365c03bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59200/0 (socket says 172.21.15.160:59200) 2024-03-07T07:40:41.007 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.001+0000 7f44f5987640 1 -- 172.21.15.160:0/56543783 learned_addr learned my addr 172.21.15.160:0/56543783 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:40:41.007 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.001+0000 7f44f6188640 1 --2- 172.21.15.160:0/56543783 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56365c037c10 0x56365c03a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:41.007 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.001+0000 7f44f5186640 1 --2- 172.21.15.160:0/56543783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56365c032aa0 0x56365c043ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:41.007 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.001+0000 7f44f5987640 1 -- 172.21.15.160:0/56543783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56365c032aa0 msgr2=0x56365c043ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:40:41.007 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.001+0000 7f44f5987640 1 --2- 172.21.15.160:0/56543783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56365c032aa0 0x56365c043ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:41.008 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.001+0000 7f44f5987640 1 -- 172.21.15.160:0/56543783 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56365c037c10 msgr2=0x56365c03a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:40:41.008 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.001+0000 7f44f5987640 1 --2- 172.21.15.160:0/56543783 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56365c037c10 0x56365c03a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:41.008 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.001+0000 7f44f5987640 1 -- 172.21.15.160:0/56543783 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56365c0488c0 con 0x56365c0320b0 2024-03-07T07:40:41.008 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.001+0000 7f44f6188640 1 --2- 172.21.15.160:0/56543783 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56365c037c10 0x56365c03a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:40:41.008 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.001+0000 7f44f5987640 1 --2- 172.21.15.160:0/56543783 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56365c0320b0 0x56365c03bbc0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f44e000a010 tx=0x7f44e0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37cd1e1617b0b0e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:41.008 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.001+0000 7f44db7fe640 1 -- 172.21.15.160:0/56543783 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e0032070 con 0x56365c0320b0 2024-03-07T07:40:41.008 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.002+0000 7f44db7fe640 1 -- 172.21.15.160:0/56543783 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f44e001f040 con 0x56365c0320b0 2024-03-07T07:40:41.008 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.002+0000 7f44db7fe640 1 -- 172.21.15.160:0/56543783 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e001e040 con 0x56365c0320b0 2024-03-07T07:40:41.008 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.002+0000 7f44fa9d3740 1 -- 172.21.15.160:0/56543783 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56365c0320b0 msgr2=0x56365c03bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:41.008 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.002+0000 7f44fa9d3740 1 --2- 172.21.15.160:0/56543783 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56365c0320b0 0x56365c03bbc0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f44e000a010 tx=0x7f44e0011810 comp rx=0 tx=0).stop 2024-03-07T07:40:41.008 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.002+0000 7f44fa9d3740 1 -- 172.21.15.160:0/56543783 shutdown_connections 2024-03-07T07:40:41.008 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.002+0000 7f44fa9d3740 1 --2- 172.21.15.160:0/56543783 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56365c037c10 0x56365c03a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:41.008 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.002+0000 7f44fa9d3740 1 --2- 172.21.15.160:0/56543783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56365c032aa0 0x56365c043ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:41.008 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.002+0000 7f44fa9d3740 1 --2- 172.21.15.160:0/56543783 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56365c0320b0 0x56365c03bbc0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:41.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.002+0000 7f44fa9d3740 1 -- 172.21.15.160:0/56543783 >> 172.21.15.160:0/56543783 conn(0x56365bffa3d0 msgr2=0x56365c03b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:41.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.002+0000 7f44fa9d3740 1 -- 172.21.15.160:0/56543783 shutdown_connections 2024-03-07T07:40:41.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.002+0000 7f44fa9d3740 1 -- 172.21.15.160:0/56543783 wait complete. 2024-03-07T07:40:41.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.002+0000 7f44fa9d3740 1 -- start start 2024-03-07T07:40:41.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.002+0000 7f44fa9d3740 1 Processor -- start 2024-03-07T07:40:41.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.003+0000 7f44fa9d3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56365c0320b0 0x56365c03f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:41.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.003+0000 7f44fa9d3740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56365c032aa0 0x56365c03f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:41.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.003+0000 7f44fa9d3740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56365c037c10 0x56365c042670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:41.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.003+0000 7f44fa9d3740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56365c04a410 con 0x56365c032aa0 2024-03-07T07:40:41.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.003+0000 7f44fa9d3740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x56365c049db0 con 0x56365c037c10 2024-03-07T07:40:41.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.003+0000 7f44fa9d3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x56365c04aa70 con 0x56365c0320b0 2024-03-07T07:40:41.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.003+0000 7f44fa9d3740 10 client.0 _ll_register_callbacks cb 0x7f44ec005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-07T07:40:41.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.003+0000 7f44f5186640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56365c032aa0 0x56365c03f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:41.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.003+0000 7f44f5186640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56365c032aa0 0x56365c03f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59216/0 (socket says 172.21.15.160:59216) 2024-03-07T07:40:41.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.003+0000 7f44f5186640 1 -- 172.21.15.160:0/2059372773 learned_addr learned my addr 172.21.15.160:0/2059372773 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:40:41.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.003+0000 7f44f5987640 1 --2- 172.21.15.160:0/2059372773 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56365c0320b0 0x56365c03f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:41.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.003+0000 7f44f6188640 1 --2- 172.21.15.160:0/2059372773 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56365c037c10 0x56365c042670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:41.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.003+0000 7f44f5186640 1 -- 172.21.15.160:0/2059372773 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56365c0320b0 msgr2=0x56365c03f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:40:41.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.003+0000 7f44f5186640 1 --2- 172.21.15.160:0/2059372773 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56365c0320b0 0x56365c03f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:41.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.003+0000 7f44f5186640 1 -- 172.21.15.160:0/2059372773 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56365c037c10 msgr2=0x56365c042670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:40:41.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.003+0000 7f44f5186640 1 --2- 172.21.15.160:0/2059372773 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56365c037c10 0x56365c042670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:41.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.003+0000 7f44f5186640 1 -- 172.21.15.160:0/2059372773 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56365c042c60 con 0x56365c032aa0 2024-03-07T07:40:41.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.003+0000 7f44f5186640 1 --2- 172.21.15.160:0/2059372773 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56365c032aa0 0x56365c03f6a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f44ec0ba9f0 tx=0x7f44ec005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=371583dc6eb2af0a server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:41.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.003+0000 7f44d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-03-07T07:40:41.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.003+0000 7f44d97fa640 1 -- 172.21.15.160:0/2059372773 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ec04b040 con 0x56365c032aa0 2024-03-07T07:40:41.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.004+0000 7f44d97fa640 1 -- 172.21.15.160:0/2059372773 <== mon.0 v2:172.21.15.160:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f44ec0c49a0 con 0x56365c032aa0 2024-03-07T07:40:41.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.004+0000 7f44fa9d3740 1 -- 172.21.15.160:0/2059372773 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f44e001f040 con 0x56365c032aa0 2024-03-07T07:40:41.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.004+0000 7f44d97fa640 1 -- 172.21.15.160:0/2059372773 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44ec0c07e0 con 0x56365c032aa0 2024-03-07T07:40:41.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.004+0000 7f44fa9d3740 10 client.15069 fetch_fsmap learned FSMap version 21 2024-03-07T07:40:41.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.004+0000 7f44fa9d3740 1 -- 172.21.15.160:0/2059372773 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x56365c0a42d0 con 0x56365c032aa0 2024-03-07T07:40:41.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.004+0000 7f44d97fa640 1 -- 172.21.15.160:0/2059372773 <== mon.0 v2:172.21.15.160:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f44e001f040 con 0x56365c032aa0 2024-03-07T07:40:41.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.004+0000 7f44d97fa640 1 -- 172.21.15.160:0/2059372773 <== mon.0 v2:172.21.15.160:3300/0 5 ==== fsmap(e 21) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f44ec0bbb90 con 0x56365c032aa0 2024-03-07T07:40:41.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.004+0000 7f44fa9d3740 10 client.15069 fetch_fsmap finished waiting for FSMap version 21 2024-03-07T07:40:41.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.004+0000 7f44fa9d3740 10 client.15069 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-07T07:40:41.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.004+0000 7f44fa9d3740 20 client.15069 populate_metadata read hostname 'smithi160' 2024-03-07T07:40:41.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.004+0000 7f44fa9d3740 1 --2- 172.21.15.160:0/2059372773 >> [v2:172.21.15.177:6838/3329773838,v1:172.21.15.177:6839/3329773838] conn(0x56365c0a8750 0x56365c0a4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:41.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.004+0000 7f44fa9d3740 4 client.15069 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-07T07:40:41.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.004+0000 7f44fa9d3740 1 -- 172.21.15.160:0/2059372773 --> [v2:172.21.15.177:6838/3329773838,v1:172.21.15.177:6839/3329773838] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56365bf9f020 con 0x56365c0a8750 2024-03-07T07:40:41.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.004+0000 7f44f5987640 1 --2- 172.21.15.160:0/2059372773 >> [v2:172.21.15.177:6838/3329773838,v1:172.21.15.177:6839/3329773838] conn(0x56365c0a8750 0x56365c0a4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:41.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.004+0000 7f44d97fa640 10 client.15069 ms_handle_connect on v2:172.21.15.177:6838/3329773838 2024-03-07T07:40:41.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.004+0000 7f44f5987640 1 --2- 172.21.15.160:0/2059372773 >> [v2:172.21.15.177:6838/3329773838,v1:172.21.15.177:6839/3329773838] conn(0x56365c0a8750 0x56365c0a4500 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c8a0f384db269015 server_cookie=5705d970a5f70d87 in_seq=0 out_seq=0 2024-03-07T07:40:41.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.005+0000 7f44d97fa640 1 -- 172.21.15.160:0/2059372773 <== mds.1 v2:172.21.15.177:6838/3329773838 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x56365bf9f020 con 0x56365c0a8750 2024-03-07T07:40:41.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.005+0000 7f44d97fa640 10 client.15069 handle_command_reply: tid=0 2024-03-07T07:40:41.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.005+0000 7f44fa9d3740 1 client.15069 shutdown 2024-03-07T07:40:41.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.005+0000 7f44fa9d3740 1 -- 172.21.15.160:0/2059372773 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56365c032aa0 msgr2=0x56365c03f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:41.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.005+0000 7f44fa9d3740 1 --2- 172.21.15.160:0/2059372773 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56365c032aa0 0x56365c03f6a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f44ec0ba9f0 tx=0x7f44ec005e50 comp rx=0 tx=0).stop 2024-03-07T07:40:41.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.005+0000 7f44fa9d3740 1 -- 172.21.15.160:0/2059372773 shutdown_connections 2024-03-07T07:40:41.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.005+0000 7f44fa9d3740 1 --2- 172.21.15.160:0/2059372773 >> [v2:172.21.15.177:6838/3329773838,v1:172.21.15.177:6839/3329773838] conn(0x56365c0a8750 0x56365c0a4500 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:41.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.005+0000 7f44fa9d3740 1 --2- 172.21.15.160:0/2059372773 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56365c037c10 0x56365c042670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:41.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.005+0000 7f44fa9d3740 1 --2- 172.21.15.160:0/2059372773 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56365c032aa0 0x56365c03f6a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:41.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.005+0000 7f44d97fa640 0 client.15069 ms_handle_reset on v2:172.21.15.177:6838/3329773838 2024-03-07T07:40:41.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.005+0000 7f44fa9d3740 1 --2- 172.21.15.160:0/2059372773 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56365c0320b0 0x56365c03f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:41.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.005+0000 7f44fa9d3740 1 -- 172.21.15.160:0/2059372773 >> 172.21.15.160:0/2059372773 conn(0x56365bffa3d0 msgr2=0x56365c033f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:41.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.005+0000 7f44fa9d3740 1 -- 172.21.15.160:0/2059372773 shutdown_connections 2024-03-07T07:40:41.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.005+0000 7f44fa9d3740 1 -- 172.21.15.160:0/2059372773 wait complete. 2024-03-07T07:40:41.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.005+0000 7f44fa9d3740 20 client.15069 trim_cache size 0 max 16384 2024-03-07T07:40:41.013 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-03-07T07:40:41.013 INFO:teuthology.orchestra.run.smithi160.stdout: "cluster_fsid": "826f6582-dc54-11ee-95c6-87774f69a715", 2024-03-07T07:40:41.013 INFO:teuthology.orchestra.run.smithi160.stdout: "whoami": 1, 2024-03-07T07:40:41.013 INFO:teuthology.orchestra.run.smithi160.stdout: "id": 24497, 2024-03-07T07:40:41.013 INFO:teuthology.orchestra.run.smithi160.stdout: "want_state": "up:active", 2024-03-07T07:40:41.013 INFO:teuthology.orchestra.run.smithi160.stdout: "state": "up:active", 2024-03-07T07:40:41.013 INFO:teuthology.orchestra.run.smithi160.stdout: "fs_name": "cephfs", 2024-03-07T07:40:41.013 INFO:teuthology.orchestra.run.smithi160.stdout: "rank_uptime": 84.157505236999995, 2024-03-07T07:40:41.013 INFO:teuthology.orchestra.run.smithi160.stdout: "mdsmap_epoch": 21, 2024-03-07T07:40:41.014 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_epoch": 87, 2024-03-07T07:40:41.014 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_epoch_barrier": 85, 2024-03-07T07:40:41.014 INFO:teuthology.orchestra.run.smithi160.stdout: "uptime": 93.272169507000001 2024-03-07T07:40:41.014 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-03-07T07:40:41.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.005+0000 7f44f7beb640 1 -- 172.21.15.160:0/777296806 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f44dc0a1d80 msgr2=0x7f44dc0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:41.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.005+0000 7f44f7beb640 1 --2- 172.21.15.160:0/777296806 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f44dc0a1d80 0x7f44dc0a4240 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x56365c033300 tx=0x7f44e8009500 comp rx=0 tx=0).stop 2024-03-07T07:40:41.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.005+0000 7f44f7beb640 1 -- 172.21.15.160:0/777296806 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f44f0101770 msgr2=0x7f44f0197f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:41.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.005+0000 7f44f7beb640 1 --2- 172.21.15.160:0/777296806 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f44f0101770 0x7f44f0197f50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f44ec009230 tx=0x7f44ec02c650 comp rx=0 tx=0).stop 2024-03-07T07:40:41.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.005+0000 7f44f7beb640 1 -- 172.21.15.160:0/777296806 shutdown_connections 2024-03-07T07:40:41.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.005+0000 7f44f7beb640 1 --2- 172.21.15.160:0/777296806 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f44dc0a1d80 0x7f44dc0a4240 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:41.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.005+0000 7f44f7beb640 1 --2- 172.21.15.160:0/777296806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f44f0102130 0x7f44f019ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:41.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.005+0000 7f44f7beb640 1 --2- 172.21.15.160:0/777296806 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f44f0101770 0x7f44f0197f50 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:41.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.006+0000 7f44f7beb640 1 --2- 172.21.15.160:0/777296806 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44f0100580 0x7f44f0197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:41.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.006+0000 7f44f7beb640 1 -- 172.21.15.160:0/777296806 >> 172.21.15.160:0/777296806 conn(0x7f44f00fbb10 msgr2=0x7f44f00fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:41.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.006+0000 7f44f7beb640 1 -- 172.21.15.160:0/777296806 shutdown_connections 2024-03-07T07:40:41.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.006+0000 7f44f7beb640 1 -- 172.21.15.160:0/777296806 wait complete. 2024-03-07T07:40:41.017 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-07T07:40:41.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.119+0000 7f135151d640 1 -- 172.21.15.160:0/3747835059 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f134c104a10 msgr2=0x7f134c106e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:41.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.119+0000 7f135151d640 1 --2- 172.21.15.160:0/3747835059 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f134c104a10 0x7f134c106e00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1334009860 tx=0x7f133402bb70 comp rx=0 tx=0).stop 2024-03-07T07:40:41.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.119+0000 7f135151d640 1 -- 172.21.15.160:0/3747835059 shutdown_connections 2024-03-07T07:40:41.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.119+0000 7f135151d640 1 --2- 172.21.15.160:0/3747835059 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f134c107340 0x7f134c109790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:41.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.119+0000 7f135151d640 1 --2- 172.21.15.160:0/3747835059 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f134c104a10 0x7f134c106e00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:41.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.119+0000 7f135151d640 1 --2- 172.21.15.160:0/3747835059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f134c1020e0 0x7f134c1044d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:41.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.119+0000 7f135151d640 1 -- 172.21.15.160:0/3747835059 >> 172.21.15.160:0/3747835059 conn(0x7f134c0fbaf0 msgr2=0x7f134c0fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:41.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.119+0000 7f135151d640 1 -- 172.21.15.160:0/3747835059 shutdown_connections 2024-03-07T07:40:41.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.119+0000 7f135151d640 1 -- 172.21.15.160:0/3747835059 wait complete. 2024-03-07T07:40:41.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.119+0000 7f135151d640 1 Processor -- start 2024-03-07T07:40:41.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.119+0000 7f135151d640 1 -- start start 2024-03-07T07:40:41.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.120+0000 7f135151d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f134c1020e0 0x7f134c101a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:41.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.120+0000 7f135151d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f134c104a10 0x7f134c100090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:41.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.120+0000 7f135151d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f134c107340 0x7f134c1005d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:41.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.120+0000 7f135151d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f134c10d2e0 con 0x7f134c104a10 2024-03-07T07:40:41.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.120+0000 7f135151d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f134c10d160 con 0x7f134c107340 2024-03-07T07:40:41.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.120+0000 7f135151d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f134c10d460 con 0x7f134c1020e0 2024-03-07T07:40:41.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.120+0000 7f134affd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f134c1020e0 0x7f134c101a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:41.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.120+0000 7f1343fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f134c104a10 0x7f134c100090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:41.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.120+0000 7f134affd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f134c1020e0 0x7f134c101a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.160:48634/0 (socket says 172.21.15.160:48634) 2024-03-07T07:40:41.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.120+0000 7f1343fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f134c104a10 0x7f134c100090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59238/0 (socket says 172.21.15.160:59238) 2024-03-07T07:40:41.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.120+0000 7f134affd640 1 -- 172.21.15.160:0/4247460279 learned_addr learned my addr 172.21.15.160:0/4247460279 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:40:41.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.120+0000 7f134b7fe640 1 --2- 172.21.15.160:0/4247460279 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f134c107340 0x7f134c1005d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:41.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.120+0000 7f134affd640 1 -- 172.21.15.160:0/4247460279 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f134c107340 msgr2=0x7f134c1005d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:41.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.120+0000 7f134affd640 1 --2- 172.21.15.160:0/4247460279 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f134c107340 0x7f134c1005d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:41.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.120+0000 7f134affd640 1 -- 172.21.15.160:0/4247460279 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f134c104a10 msgr2=0x7f134c100090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:41.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.120+0000 7f134affd640 1 --2- 172.21.15.160:0/4247460279 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f134c104a10 0x7f134c100090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:41.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.120+0000 7f134affd640 1 -- 172.21.15.160:0/4247460279 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f134c100e60 con 0x7f134c1020e0 2024-03-07T07:40:41.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.120+0000 7f1343fff640 1 --2- 172.21.15.160:0/4247460279 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f134c104a10 0x7f134c100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:40:41.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.120+0000 7f134affd640 1 --2- 172.21.15.160:0/4247460279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f134c1020e0 0x7f134c101a50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f133c00e9f0 tx=0x7f133c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:41.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.120+0000 7f1348ff9640 1 -- 172.21.15.160:0/4247460279 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133c00cd80 con 0x7f134c1020e0 2024-03-07T07:40:41.124 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.120+0000 7f1348ff9640 1 -- 172.21.15.160:0/4247460279 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f133c005500 con 0x7f134c1020e0 2024-03-07T07:40:41.124 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.120+0000 7f135151d640 1 -- 172.21.15.160:0/4247460279 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f134c1a1ad0 con 0x7f134c1020e0 2024-03-07T07:40:41.124 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.120+0000 7f1348ff9640 1 -- 172.21.15.160:0/4247460279 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133c010690 con 0x7f134c1020e0 2024-03-07T07:40:41.124 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.120+0000 7f135151d640 1 -- 172.21.15.160:0/4247460279 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f134c1a1ff0 con 0x7f134c1020e0 2024-03-07T07:40:41.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.123+0000 7f1348ff9640 1 -- 172.21.15.160:0/4247460279 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f133c0046c0 con 0x7f134c1020e0 2024-03-07T07:40:41.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.123+0000 7f1354407740 1 Processor -- start 2024-03-07T07:40:41.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.123+0000 7f1354407740 1 -- start start 2024-03-07T07:40:41.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.124+0000 7f1354407740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583c24cb0b0 0x5583c24cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:41.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.124+0000 7f1354407740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5583c24cbaa0 0x5583c24d05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:41.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.124+0000 7f1348ff9640 1 --2- 172.21.15.160:0/4247460279 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f13380a1e00 0x7f13380a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:41.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.124+0000 7f1354407740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5583c24d0c10 0x5583c24d30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:41.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.124+0000 7f1354407740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583c2430790 con 0x5583c24d0c10 2024-03-07T07:40:41.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.124+0000 7f1354407740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583c23d9b20 con 0x5583c24cbaa0 2024-03-07T07:40:41.131 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.124+0000 7f1354407740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583c242c830 con 0x5583c24cb0b0 2024-03-07T07:40:41.131 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.124+0000 7f134b7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583c24cb0b0 0x5583c24cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:41.131 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.124+0000 7f1343fff640 1 --2- 172.21.15.160:0/4247460279 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f13380a1e00 0x7f13380a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:41.131 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.124+0000 7f134affd640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5583c24cbaa0 0x5583c24d05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:41.131 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.124+0000 7f134b7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583c24cb0b0 0x5583c24cb4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.160:48636/0 (socket says 172.21.15.160:48636) 2024-03-07T07:40:41.131 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.124+0000 7f134b7fe640 1 -- 172.21.15.160:0/3624512246 learned_addr learned my addr 172.21.15.160:0/3624512246 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:40:41.131 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.124+0000 7f134b7fe640 1 --2- 172.21.15.160:0/3624512246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5583c24d0c10 0x5583c24d30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:41.131 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.124+0000 7f1348ff9640 1 -- 172.21.15.160:0/4247460279 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f133c0ce170 con 0x7f134c1020e0 2024-03-07T07:40:41.131 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.124+0000 7f134b7fe640 1 -- 172.21.15.160:0/3624512246 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5583c24cbaa0 msgr2=0x5583c24d05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:40:41.131 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.124+0000 7f134b7fe640 1 --2- 172.21.15.160:0/3624512246 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5583c24cbaa0 0x5583c24d05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:41.131 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.124+0000 7f134b7fe640 1 -- 172.21.15.160:0/3624512246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5583c24d0c10 msgr2=0x5583c24d30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:40:41.131 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.124+0000 7f134b7fe640 1 --2- 172.21.15.160:0/3624512246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5583c24d0c10 0x5583c24d30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:41.131 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.124+0000 7f134b7fe640 1 -- 172.21.15.160:0/3624512246 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583c23ed970 con 0x5583c24cb0b0 2024-03-07T07:40:41.132 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.124+0000 7f134affd640 1 --2- 172.21.15.160:0/3624512246 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5583c24cbaa0 0x5583c24d05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:40:41.132 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.124+0000 7f134b7fe640 1 --2- 172.21.15.160:0/3624512246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583c24cb0b0 0x5583c24cb4d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f134400ec90 tx=0x7f1344010040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fa3f25a0a69c65d server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:41.132 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.124+0000 7f1343fff640 1 --2- 172.21.15.160:0/4247460279 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f13380a1e00 0x7f13380a42c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f13340349e0 tx=0x7f133402e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:41.132 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.124+0000 7f13417fa640 1 -- 172.21.15.160:0/3624512246 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134400ee40 con 0x5583c24cb0b0 2024-03-07T07:40:41.132 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.124+0000 7f13417fa640 1 -- 172.21.15.160:0/3624512246 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1344003e50 con 0x5583c24cb0b0 2024-03-07T07:40:41.132 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.124+0000 7f13417fa640 1 -- 172.21.15.160:0/3624512246 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134400d3e0 con 0x5583c24cb0b0 2024-03-07T07:40:41.132 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.125+0000 7f1354407740 1 -- 172.21.15.160:0/3624512246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583c24cb0b0 msgr2=0x5583c24cb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:41.132 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.125+0000 7f1354407740 1 --2- 172.21.15.160:0/3624512246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583c24cb0b0 0x5583c24cb4d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f134400ec90 tx=0x7f1344010040 comp rx=0 tx=0).stop 2024-03-07T07:40:41.132 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.125+0000 7f1354407740 1 -- 172.21.15.160:0/3624512246 shutdown_connections 2024-03-07T07:40:41.132 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.125+0000 7f1354407740 1 --2- 172.21.15.160:0/3624512246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5583c24d0c10 0x5583c24d30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:41.132 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.125+0000 7f1354407740 1 --2- 172.21.15.160:0/3624512246 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5583c24cbaa0 0x5583c24d05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:41.132 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.125+0000 7f1354407740 1 --2- 172.21.15.160:0/3624512246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583c24cb0b0 0x5583c24cb4d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:41.132 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.125+0000 7f1354407740 1 -- 172.21.15.160:0/3624512246 >> 172.21.15.160:0/3624512246 conn(0x5583c24933d0 msgr2=0x5583c24c6a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:41.132 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.125+0000 7f1354407740 1 -- 172.21.15.160:0/3624512246 shutdown_connections 2024-03-07T07:40:41.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.125+0000 7f1354407740 1 -- 172.21.15.160:0/3624512246 wait complete. 2024-03-07T07:40:41.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.125+0000 7f1354407740 1 -- start start 2024-03-07T07:40:41.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.126+0000 7f1354407740 1 Processor -- start 2024-03-07T07:40:41.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.126+0000 7f1354407740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5583c24cb0b0 0x5583c24d5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:41.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.126+0000 7f1354407740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5583c24cbaa0 0x5583c24d5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:41.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.126+0000 7f1354407740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583c24d0c10 0x5583c24dcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:41.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.126+0000 7f1354407740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583c24d3bc0 con 0x5583c24cbaa0 2024-03-07T07:40:41.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.126+0000 7f1354407740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583c23ecf60 con 0x5583c24cb0b0 2024-03-07T07:40:41.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.126+0000 7f1354407740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583c24d3d40 con 0x5583c24d0c10 2024-03-07T07:40:41.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.126+0000 7f1354407740 10 client.0 _ll_register_callbacks cb 0x5583c24283d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-07T07:40:41.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.126+0000 7f1343fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5583c24cbaa0 0x5583c24d5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:41.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.126+0000 7f1343fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5583c24cbaa0 0x5583c24d5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59262/0 (socket says 172.21.15.160:59262) 2024-03-07T07:40:41.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.126+0000 7f1343fff640 1 -- 172.21.15.160:0/369061321 learned_addr learned my addr 172.21.15.160:0/369061321 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:40:41.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.126+0000 7f134b7fe640 1 --2- 172.21.15.160:0/369061321 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583c24d0c10 0x5583c24dcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:41.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.126+0000 7f134affd640 1 --2- 172.21.15.160:0/369061321 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5583c24cb0b0 0x5583c24d5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:41.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.126+0000 7f1343fff640 1 -- 172.21.15.160:0/369061321 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583c24d0c10 msgr2=0x5583c24dcbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:40:41.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.126+0000 7f1343fff640 1 --2- 172.21.15.160:0/369061321 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583c24d0c10 0x5583c24dcbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:41.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.126+0000 7f1343fff640 1 -- 172.21.15.160:0/369061321 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5583c24cb0b0 msgr2=0x5583c24d5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:40:41.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.126+0000 7f1343fff640 1 --2- 172.21.15.160:0/369061321 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5583c24cb0b0 0x5583c24d5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:41.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.126+0000 7f1343fff640 1 -- 172.21.15.160:0/369061321 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5583c21452d0 con 0x5583c24cbaa0 2024-03-07T07:40:41.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.126+0000 7f134b7fe640 1 --2- 172.21.15.160:0/369061321 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583c24d0c10 0x5583c24dcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:40:41.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.126+0000 7f134affd640 1 --2- 172.21.15.160:0/369061321 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5583c24cb0b0 0x5583c24d5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:40:41.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.126+0000 7f1343fff640 1 --2- 172.21.15.160:0/369061321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5583c24cbaa0 0x5583c24d5b90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f13340095f0 tx=0x7f1334005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e2eb643c0fe576 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:41.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.126+0000 7f13327fc640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-03-07T07:40:41.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.126+0000 7f13327fc640 1 -- 172.21.15.160:0/369061321 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133404f020 con 0x5583c24cbaa0 2024-03-07T07:40:41.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.127+0000 7f13327fc640 1 -- 172.21.15.160:0/369061321 <== mon.0 v2:172.21.15.160:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1334032640 con 0x5583c24cbaa0 2024-03-07T07:40:41.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.127+0000 7f1354407740 1 -- 172.21.15.160:0/369061321 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1344003e50 con 0x5583c24cbaa0 2024-03-07T07:40:41.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.127+0000 7f13327fc640 1 -- 172.21.15.160:0/369061321 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1334039040 con 0x5583c24cbaa0 2024-03-07T07:40:41.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.127+0000 7f1354407740 10 client.15081 fetch_fsmap learned FSMap version 21 2024-03-07T07:40:41.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.127+0000 7f1354407740 1 -- 172.21.15.160:0/369061321 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x5583c220f6b0 con 0x5583c24cbaa0 2024-03-07T07:40:41.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.127+0000 7f13327fc640 1 -- 172.21.15.160:0/369061321 <== mon.0 v2:172.21.15.160:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1344003e50 con 0x5583c24cbaa0 2024-03-07T07:40:41.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.127+0000 7f13327fc640 1 -- 172.21.15.160:0/369061321 <== mon.0 v2:172.21.15.160:3300/0 5 ==== fsmap(e 21) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f1334003920 con 0x5583c24cbaa0 2024-03-07T07:40:41.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.127+0000 7f1354407740 10 client.15081 fetch_fsmap finished waiting for FSMap version 21 2024-03-07T07:40:41.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.127+0000 7f1354407740 10 client.15081 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-07T07:40:41.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.127+0000 7f1354407740 20 client.15081 populate_metadata read hostname 'smithi160' 2024-03-07T07:40:41.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.127+0000 7f1354407740 1 --2- 172.21.15.160:0/369061321 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x5583c24e2130 0x5583c24e1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:41.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.127+0000 7f1354407740 4 client.15081 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-07T07:40:41.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.127+0000 7f1354407740 1 -- 172.21.15.160:0/369061321 --> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5583c243dcd0 con 0x5583c24e2130 2024-03-07T07:40:41.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.127+0000 7f134affd640 1 --2- 172.21.15.160:0/369061321 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x5583c24e2130 0x5583c24e1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:41.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.128+0000 7f134affd640 1 --2- 172.21.15.160:0/369061321 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x5583c24e2130 0x5583c24e1250 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a954cd38d789f6b6 server_cookie=7900967c472ef9da in_seq=0 out_seq=0 2024-03-07T07:40:41.136 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.128+0000 7f13327fc640 10 client.15081 ms_handle_connect on v2:172.21.15.178:6838/2947354024 2024-03-07T07:40:41.136 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.128+0000 7f13327fc640 1 -- 172.21.15.160:0/369061321 <== mds.0 v2:172.21.15.178:6838/2947354024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5583c243dcd0 con 0x5583c24e2130 2024-03-07T07:40:41.136 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.128+0000 7f13327fc640 10 client.15081 handle_command_reply: tid=0 2024-03-07T07:40:41.136 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.128+0000 7f1354407740 1 client.15081 shutdown 2024-03-07T07:40:41.136 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.128+0000 7f1354407740 1 -- 172.21.15.160:0/369061321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5583c24cbaa0 msgr2=0x5583c24d5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:41.136 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.128+0000 7f1354407740 1 --2- 172.21.15.160:0/369061321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5583c24cbaa0 0x5583c24d5b90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f13340095f0 tx=0x7f1334005f90 comp rx=0 tx=0).stop 2024-03-07T07:40:41.136 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.128+0000 7f1354407740 1 -- 172.21.15.160:0/369061321 shutdown_connections 2024-03-07T07:40:41.136 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.128+0000 7f1354407740 1 --2- 172.21.15.160:0/369061321 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x5583c24e2130 0x5583c24e1250 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:41.136 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.128+0000 7f1354407740 1 --2- 172.21.15.160:0/369061321 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583c24d0c10 0x5583c24dcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:41.136 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.128+0000 7f1354407740 1 --2- 172.21.15.160:0/369061321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5583c24cbaa0 0x5583c24d5b90 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:41.136 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.128+0000 7f13327fc640 0 client.15081 ms_handle_reset on v2:172.21.15.178:6838/2947354024 2024-03-07T07:40:41.136 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.128+0000 7f1354407740 1 --2- 172.21.15.160:0/369061321 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5583c24cb0b0 0x5583c24d5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:41.136 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.128+0000 7f1354407740 1 -- 172.21.15.160:0/369061321 >> 172.21.15.160:0/369061321 conn(0x5583c24933d0 msgr2=0x5583c24c5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:41.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.128+0000 7f1354407740 1 -- 172.21.15.160:0/369061321 shutdown_connections 2024-03-07T07:40:41.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.128+0000 7f1354407740 1 -- 172.21.15.160:0/369061321 wait complete. 2024-03-07T07:40:41.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.129+0000 7f1354407740 20 client.15081 trim_cache size 0 max 16384 2024-03-07T07:40:41.150 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.145+0000 7f1354407740 1 Processor -- start 2024-03-07T07:40:41.150 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.145+0000 7f1354407740 1 -- start start 2024-03-07T07:40:41.150 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.145+0000 7f1354407740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5583c24cb0b0 0x5583c24c6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:41.150 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.145+0000 7f1354407740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583c24cbaa0 0x5583c24c6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:41.150 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.145+0000 7f1354407740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5583c24d0c10 0x5583c24dcb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:41.150 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.145+0000 7f1354407740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583c24e1950 con 0x5583c24cb0b0 2024-03-07T07:40:41.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.145+0000 7f1354407740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583c24e1fb0 con 0x5583c24d0c10 2024-03-07T07:40:41.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.145+0000 7f1354407740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583c24e2610 con 0x5583c24cbaa0 2024-03-07T07:40:41.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.145+0000 7f134affd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5583c24cb0b0 0x5583c24c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:41.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.145+0000 7f1343fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583c24cbaa0 0x5583c24c6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:41.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.145+0000 7f134affd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5583c24cb0b0 0x5583c24c6540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59276/0 (socket says 172.21.15.160:59276) 2024-03-07T07:40:41.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.145+0000 7f1343fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583c24cbaa0 0x5583c24c6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.160:48660/0 (socket says 172.21.15.160:48660) 2024-03-07T07:40:41.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.145+0000 7f134affd640 1 -- 172.21.15.160:0/3357551353 learned_addr learned my addr 172.21.15.160:0/3357551353 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:40:41.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.145+0000 7f134b7fe640 1 --2- 172.21.15.160:0/3357551353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5583c24d0c10 0x5583c24dcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:41.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.145+0000 7f134affd640 1 -- 172.21.15.160:0/3357551353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583c24cbaa0 msgr2=0x5583c24c6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:40:41.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.145+0000 7f134affd640 1 --2- 172.21.15.160:0/3357551353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583c24cbaa0 0x5583c24c6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:41.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.145+0000 7f134affd640 1 -- 172.21.15.160:0/3357551353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5583c24d0c10 msgr2=0x5583c24dcb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:40:41.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.145+0000 7f134affd640 1 --2- 172.21.15.160:0/3357551353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5583c24d0c10 0x5583c24dcb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:41.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.145+0000 7f134affd640 1 -- 172.21.15.160:0/3357551353 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583c24cca40 con 0x5583c24cb0b0 2024-03-07T07:40:41.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.145+0000 7f1343fff640 1 --2- 172.21.15.160:0/3357551353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583c24cbaa0 0x5583c24c6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:40:41.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.145+0000 7f134affd640 1 --2- 172.21.15.160:0/3357551353 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5583c24cb0b0 0x5583c24c6540 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f133c097300 tx=0x7f133c0cbb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53847a8d6f0331ab server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:41.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.146+0000 7f1340ff9640 1 -- 172.21.15.160:0/3357551353 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133c09aea0 con 0x5583c24cb0b0 2024-03-07T07:40:41.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.146+0000 7f1340ff9640 1 -- 172.21.15.160:0/3357551353 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f133c09eff0 con 0x5583c24cb0b0 2024-03-07T07:40:41.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.146+0000 7f1340ff9640 1 -- 172.21.15.160:0/3357551353 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133c0a31b0 con 0x5583c24cb0b0 2024-03-07T07:40:41.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.146+0000 7f1354407740 1 -- 172.21.15.160:0/3357551353 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5583c24cb0b0 msgr2=0x5583c24c6540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:41.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.146+0000 7f1354407740 1 --2- 172.21.15.160:0/3357551353 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5583c24cb0b0 0x5583c24c6540 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f133c097300 tx=0x7f133c0cbb40 comp rx=0 tx=0).stop 2024-03-07T07:40:41.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.146+0000 7f1354407740 1 -- 172.21.15.160:0/3357551353 shutdown_connections 2024-03-07T07:40:41.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.146+0000 7f1354407740 1 --2- 172.21.15.160:0/3357551353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5583c24d0c10 0x5583c24dcb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:41.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.146+0000 7f1354407740 1 --2- 172.21.15.160:0/3357551353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583c24cbaa0 0x5583c24c6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:41.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.146+0000 7f1354407740 1 --2- 172.21.15.160:0/3357551353 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5583c24cb0b0 0x5583c24c6540 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:41.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.146+0000 7f1354407740 1 -- 172.21.15.160:0/3357551353 >> 172.21.15.160:0/3357551353 conn(0x5583c24933d0 msgr2=0x5583c24cc330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:41.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.146+0000 7f1354407740 1 -- 172.21.15.160:0/3357551353 shutdown_connections 2024-03-07T07:40:41.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.146+0000 7f1354407740 1 -- 172.21.15.160:0/3357551353 wait complete. 2024-03-07T07:40:41.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.147+0000 7f1354407740 1 -- start start 2024-03-07T07:40:41.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.147+0000 7f1354407740 1 Processor -- start 2024-03-07T07:40:41.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.147+0000 7f1354407740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5583c24cb0b0 0x5583c24dc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:41.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.147+0000 7f1354407740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5583c24cbaa0 0x5583c24d9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:41.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.147+0000 7f1354407740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583c24d0c10 0x5583c24d9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:41.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.147+0000 7f1354407740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583c24e32d0 con 0x5583c24cb0b0 2024-03-07T07:40:41.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.147+0000 7f1354407740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583c24e2c70 con 0x5583c24cbaa0 2024-03-07T07:40:41.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.147+0000 7f1354407740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583c24e3930 con 0x5583c24d0c10 2024-03-07T07:40:41.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.147+0000 7f1354407740 10 client.0 _ll_register_callbacks cb 0x7f1344009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-07T07:40:41.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.147+0000 7f134affd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5583c24cb0b0 0x5583c24dc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:41.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.147+0000 7f134affd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5583c24cb0b0 0x5583c24dc750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59282/0 (socket says 172.21.15.160:59282) 2024-03-07T07:40:41.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.147+0000 7f134affd640 1 -- 172.21.15.160:0/3394522019 learned_addr learned my addr 172.21.15.160:0/3394522019 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:40:41.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.147+0000 7f134b7fe640 1 --2- 172.21.15.160:0/3394522019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583c24d0c10 0x5583c24d9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:41.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.147+0000 7f1343fff640 1 --2- 172.21.15.160:0/3394522019 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5583c24cbaa0 0x5583c24d9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:41.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.147+0000 7f134affd640 1 -- 172.21.15.160:0/3394522019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583c24d0c10 msgr2=0x5583c24d9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:40:41.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.147+0000 7f134affd640 1 --2- 172.21.15.160:0/3394522019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583c24d0c10 0x5583c24d9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:41.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.147+0000 7f134affd640 1 -- 172.21.15.160:0/3394522019 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5583c24cbaa0 msgr2=0x5583c24d9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:40:41.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.147+0000 7f134affd640 1 --2- 172.21.15.160:0/3394522019 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5583c24cbaa0 0x5583c24d9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:41.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.147+0000 7f134affd640 1 -- 172.21.15.160:0/3394522019 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5583c24dcd80 con 0x5583c24cb0b0 2024-03-07T07:40:41.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.147+0000 7f1343fff640 1 --2- 172.21.15.160:0/3394522019 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5583c24cbaa0 0x5583c24d9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:40:41.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.147+0000 7f134affd640 1 --2- 172.21.15.160:0/3394522019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5583c24cb0b0 0x5583c24dc750 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f133c097820 tx=0x7f133c0cbf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b38466dc24eb14ec server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:41.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.147+0000 7f13327fc640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-03-07T07:40:41.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.148+0000 7f13327fc640 1 -- 172.21.15.160:0/3394522019 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133c0991a0 con 0x5583c24cb0b0 2024-03-07T07:40:41.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.148+0000 7f13327fc640 1 -- 172.21.15.160:0/3394522019 <== mon.0 v2:172.21.15.160:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f133c0aaa00 con 0x5583c24cb0b0 2024-03-07T07:40:41.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.148+0000 7f1354407740 1 -- 172.21.15.160:0/3394522019 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f133c09eff0 con 0x5583c24cb0b0 2024-03-07T07:40:41.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.148+0000 7f13327fc640 1 -- 172.21.15.160:0/3394522019 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f133c09aea0 con 0x5583c24cb0b0 2024-03-07T07:40:41.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.148+0000 7f1354407740 10 client.15087 fetch_fsmap learned FSMap version 21 2024-03-07T07:40:41.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.148+0000 7f1354407740 1 -- 172.21.15.160:0/3394522019 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x5583c253e010 con 0x5583c24cb0b0 2024-03-07T07:40:41.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.148+0000 7f13327fc640 1 -- 172.21.15.160:0/3394522019 <== mon.0 v2:172.21.15.160:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f133c09eff0 con 0x5583c24cb0b0 2024-03-07T07:40:41.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.148+0000 7f13327fc640 1 -- 172.21.15.160:0/3394522019 <== mon.0 v2:172.21.15.160:3300/0 5 ==== fsmap(e 21) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f133c0a31b0 con 0x5583c24cb0b0 2024-03-07T07:40:41.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.148+0000 7f1354407740 10 client.15087 fetch_fsmap finished waiting for FSMap version 21 2024-03-07T07:40:41.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.148+0000 7f1354407740 10 client.15087 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-07T07:40:41.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.148+0000 7f1354407740 20 client.15087 populate_metadata read hostname 'smithi160' 2024-03-07T07:40:41.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.148+0000 7f1354407740 1 --2- 172.21.15.160:0/3394522019 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x5583c2542220 0x5583c253e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:41.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.148+0000 7f1354407740 4 client.15087 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-07T07:40:41.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.148+0000 7f1354407740 1 -- 172.21.15.160:0/3394522019 --> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5583c2438020 con 0x5583c2542220 2024-03-07T07:40:41.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.148+0000 7f1343fff640 1 --2- 172.21.15.160:0/3394522019 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x5583c2542220 0x5583c253e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:41.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.149+0000 7f13327fc640 10 client.15087 ms_handle_connect on v2:172.21.15.178:6838/2947354024 2024-03-07T07:40:41.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.149+0000 7f1343fff640 1 --2- 172.21.15.160:0/3394522019 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x5583c2542220 0x5583c253e2d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7acc1181ebf6276 server_cookie=86d3f8bf71923bb3 in_seq=0 out_seq=0 2024-03-07T07:40:41.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.149+0000 7f13327fc640 1 -- 172.21.15.160:0/3394522019 <== mds.0 v2:172.21.15.178:6838/2947354024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5583c2438020 con 0x5583c2542220 2024-03-07T07:40:41.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.149+0000 7f13327fc640 10 client.15087 handle_command_reply: tid=0 2024-03-07T07:40:41.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.149+0000 7f1354407740 1 client.15087 shutdown 2024-03-07T07:40:41.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.149+0000 7f1354407740 1 -- 172.21.15.160:0/3394522019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5583c24cb0b0 msgr2=0x5583c24dc750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:41.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.149+0000 7f1354407740 1 --2- 172.21.15.160:0/3394522019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5583c24cb0b0 0x5583c24dc750 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f133c097820 tx=0x7f133c0cbf60 comp rx=0 tx=0).stop 2024-03-07T07:40:41.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.149+0000 7f1354407740 1 -- 172.21.15.160:0/3394522019 shutdown_connections 2024-03-07T07:40:41.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.149+0000 7f1354407740 1 --2- 172.21.15.160:0/3394522019 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x5583c2542220 0x5583c253e2d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:41.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.149+0000 7f1354407740 1 --2- 172.21.15.160:0/3394522019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583c24d0c10 0x5583c24d9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:41.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.149+0000 7f13327fc640 0 client.15087 ms_handle_reset on v2:172.21.15.178:6838/2947354024 2024-03-07T07:40:41.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.149+0000 7f1354407740 1 --2- 172.21.15.160:0/3394522019 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5583c24cbaa0 0x5583c24d9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:41.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.149+0000 7f1354407740 1 --2- 172.21.15.160:0/3394522019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5583c24cb0b0 0x5583c24dc750 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:41.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.149+0000 7f1354407740 1 -- 172.21.15.160:0/3394522019 >> 172.21.15.160:0/3394522019 conn(0x5583c24933d0 msgr2=0x5583c24cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:41.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.149+0000 7f1354407740 1 -- 172.21.15.160:0/3394522019 shutdown_connections 2024-03-07T07:40:41.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.149+0000 7f1354407740 1 -- 172.21.15.160:0/3394522019 wait complete. 2024-03-07T07:40:41.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.149+0000 7f1354407740 20 client.15087 trim_cache size 0 max 16384 2024-03-07T07:40:41.157 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-03-07T07:40:41.157 INFO:teuthology.orchestra.run.smithi160.stdout: "cluster_fsid": "826f6582-dc54-11ee-95c6-87774f69a715", 2024-03-07T07:40:41.157 INFO:teuthology.orchestra.run.smithi160.stdout: "whoami": 0, 2024-03-07T07:40:41.157 INFO:teuthology.orchestra.run.smithi160.stdout: "id": 24511, 2024-03-07T07:40:41.157 INFO:teuthology.orchestra.run.smithi160.stdout: "want_state": "up:active", 2024-03-07T07:40:41.157 INFO:teuthology.orchestra.run.smithi160.stdout: "state": "up:active", 2024-03-07T07:40:41.157 INFO:teuthology.orchestra.run.smithi160.stdout: "fs_name": "cephfs", 2024-03-07T07:40:41.157 INFO:teuthology.orchestra.run.smithi160.stdout: "rank_uptime": 86.318769060999998, 2024-03-07T07:40:41.157 INFO:teuthology.orchestra.run.smithi160.stdout: "mdsmap_epoch": 21, 2024-03-07T07:40:41.157 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_epoch": 87, 2024-03-07T07:40:41.157 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_epoch_barrier": 85, 2024-03-07T07:40:41.158 INFO:teuthology.orchestra.run.smithi160.stdout: "uptime": 101.81188146300001 2024-03-07T07:40:41.158 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-03-07T07:40:41.158 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.149+0000 7f135151d640 1 -- 172.21.15.160:0/4247460279 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f13380a1e00 msgr2=0x7f13380a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:41.158 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.149+0000 7f135151d640 1 --2- 172.21.15.160:0/4247460279 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f13380a1e00 0x7f13380a42c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f13340349e0 tx=0x7f133402e040 comp rx=0 tx=0).stop 2024-03-07T07:40:41.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.149+0000 7f135151d640 1 -- 172.21.15.160:0/4247460279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f134c1020e0 msgr2=0x7f134c101a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:41.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.149+0000 7f135151d640 1 --2- 172.21.15.160:0/4247460279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f134c1020e0 0x7f134c101a50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f133c00e9f0 tx=0x7f133c00edf0 comp rx=0 tx=0).stop 2024-03-07T07:40:41.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.150+0000 7f135151d640 1 -- 172.21.15.160:0/4247460279 shutdown_connections 2024-03-07T07:40:41.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.150+0000 7f135151d640 1 --2- 172.21.15.160:0/4247460279 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f13380a1e00 0x7f13380a42c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:41.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.150+0000 7f135151d640 1 --2- 172.21.15.160:0/4247460279 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f134c107340 0x7f134c1005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:41.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.150+0000 7f135151d640 1 --2- 172.21.15.160:0/4247460279 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f134c104a10 0x7f134c100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:41.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.150+0000 7f135151d640 1 --2- 172.21.15.160:0/4247460279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f134c1020e0 0x7f134c101a50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:41.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.150+0000 7f135151d640 1 -- 172.21.15.160:0/4247460279 >> 172.21.15.160:0/4247460279 conn(0x7f134c0fbaf0 msgr2=0x7f134c102b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:41.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.150+0000 7f135151d640 1 -- 172.21.15.160:0/4247460279 shutdown_connections 2024-03-07T07:40:41.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:41.150+0000 7f135151d640 1 -- 172.21.15.160:0/4247460279 wait complete. 2024-03-07T07:40:41.161 INFO:tasks.cephadm:Setup complete, yielding 2024-03-07T07:40:41.162 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-07T07:40:41.177 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi160.front.sepia.ceph.com 2024-03-07T07:40:41.177 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- bash -c 'ceph orch status' 2024-03-07T07:40:41.419 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:40:41.675 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:41 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/650016228' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:40:41.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:41 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/650016228' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:40:41.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:41 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/650016228' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:40:42.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:42 smithi160 ceph-mon[31963]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-07T07:40:42.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:42 smithi178 ceph-mon[36650]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-07T07:40:42.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:42 smithi177 ceph-mon[36578]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-07T07:40:43.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.147+0000 7f09d2b6a700 1 -- 172.21.15.160:0/3799285625 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09cc103d20 msgr2=0x7f09cc10a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:43.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.147+0000 7f09d2b6a700 1 --2- 172.21.15.160:0/3799285625 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09cc103d20 0x7f09cc10a5d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f09c400b0d0 tx=0x7f09c400b3e0 comp rx=0 tx=0).stop 2024-03-07T07:40:43.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.147+0000 7f09d2b6a700 1 -- 172.21.15.160:0/3799285625 shutdown_connections 2024-03-07T07:40:43.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.147+0000 7f09d2b6a700 1 --2- 172.21.15.160:0/3799285625 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09cc103d20 0x7f09cc10a5d0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:43.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.147+0000 7f09d2b6a700 1 --2- 172.21.15.160:0/3799285625 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09cc102180 0x7f09cc1025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:43.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.147+0000 7f09d2b6a700 1 --2- 172.21.15.160:0/3799285625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09cc103380 0x7f09cc1037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:43.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.147+0000 7f09d2b6a700 1 -- 172.21.15.160:0/3799285625 >> 172.21.15.160:0/3799285625 conn(0x7f09cc0fd760 msgr2=0x7f09cc0ffb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:43.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.147+0000 7f09d2b6a700 1 -- 172.21.15.160:0/3799285625 shutdown_connections 2024-03-07T07:40:43.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.147+0000 7f09d2b6a700 1 -- 172.21.15.160:0/3799285625 wait complete. 2024-03-07T07:40:43.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.148+0000 7f09d2b6a700 1 Processor -- start 2024-03-07T07:40:43.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.148+0000 7f09d2b6a700 1 -- start start 2024-03-07T07:40:43.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.148+0000 7f09d2b6a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09cc102180 0x7f09cc075ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:43.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.148+0000 7f09d2b6a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09cc103380 0x7f09cc074540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:43.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.148+0000 7f09d2b6a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09cc103d20 0x7f09cc074a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:43.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.148+0000 7f09d2b6a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc10dd70 con 0x7f09cc102180 2024-03-07T07:40:43.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.148+0000 7f09d2b6a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc10dbf0 con 0x7f09cc103380 2024-03-07T07:40:43.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.148+0000 7f09d2b6a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc10def0 con 0x7f09cc103d20 2024-03-07T07:40:43.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.148+0000 7f09d0906700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09cc102180 0x7f09cc075ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:43.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.148+0000 7f09d0906700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09cc102180 0x7f09cc075ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59294/0 (socket says 172.21.15.160:59294) 2024-03-07T07:40:43.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.148+0000 7f09d0906700 1 -- 172.21.15.160:0/1121560328 learned_addr learned my addr 172.21.15.160:0/1121560328 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:40:43.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.148+0000 7f09cbfff700 1 --2- 172.21.15.160:0/1121560328 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09cc103380 0x7f09cc074540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:43.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.148+0000 7f09d1107700 1 --2- 172.21.15.160:0/1121560328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09cc103d20 0x7f09cc074a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:43.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.149+0000 7f09d0906700 1 -- 172.21.15.160:0/1121560328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09cc103d20 msgr2=0x7f09cc074a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:43.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.149+0000 7f09d0906700 1 --2- 172.21.15.160:0/1121560328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09cc103d20 0x7f09cc074a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:43.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.149+0000 7f09d0906700 1 -- 172.21.15.160:0/1121560328 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09cc103380 msgr2=0x7f09cc074540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:43.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.149+0000 7f09d0906700 1 --2- 172.21.15.160:0/1121560328 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09cc103380 0x7f09cc074540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:43.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.149+0000 7f09d0906700 1 -- 172.21.15.160:0/1121560328 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09cc075330 con 0x7f09cc102180 2024-03-07T07:40:43.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.149+0000 7f09cbfff700 1 --2- 172.21.15.160:0/1121560328 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09cc103380 0x7f09cc074540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:40:43.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.149+0000 7f09d0906700 1 --2- 172.21.15.160:0/1121560328 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09cc102180 0x7f09cc075ef0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f09c000e970 tx=0x7f09c000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:43.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.149+0000 7f09c9ffb700 1 -- 172.21.15.160:0/1121560328 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c0004220 con 0x7f09cc102180 2024-03-07T07:40:43.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.149+0000 7f09c9ffb700 1 -- 172.21.15.160:0/1121560328 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f09c00043c0 con 0x7f09cc102180 2024-03-07T07:40:43.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.149+0000 7f09c9ffb700 1 -- 172.21.15.160:0/1121560328 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c0003dc0 con 0x7f09cc102180 2024-03-07T07:40:43.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.149+0000 7f09d2b6a700 1 -- 172.21.15.160:0/1121560328 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09cc1a2480 con 0x7f09cc102180 2024-03-07T07:40:43.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.149+0000 7f09d2b6a700 1 -- 172.21.15.160:0/1121560328 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09cc1a2990 con 0x7f09cc102180 2024-03-07T07:40:43.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.150+0000 7f09d2b6a700 1 -- 172.21.15.160:0/1121560328 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09cc068e10 con 0x7f09cc102180 2024-03-07T07:40:43.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.151+0000 7f09c9ffb700 1 -- 172.21.15.160:0/1121560328 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f09c000c7e0 con 0x7f09cc102180 2024-03-07T07:40:43.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.151+0000 7f09c9ffb700 1 --2- 172.21.15.160:0/1121560328 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f09bc0a1e10 0x7f09bc0a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:43.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.152+0000 7f09cbfff700 1 --2- 172.21.15.160:0/1121560328 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f09bc0a1e10 0x7f09bc0a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:43.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.152+0000 7f09c9ffb700 1 -- 172.21.15.160:0/1121560328 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f09c00cdf50 con 0x7f09cc102180 2024-03-07T07:40:43.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.152+0000 7f09cbfff700 1 --2- 172.21.15.160:0/1121560328 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f09bc0a1e10 0x7f09bc0a42d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f09b8011490 tx=0x7f09b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:43.157 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.156+0000 7f09c9ffb700 1 -- 172.21.15.160:0/1121560328 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f09c009b380 con 0x7f09cc102180 2024-03-07T07:40:43.368 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.368+0000 7f09d2b6a700 1 -- 172.21.15.160:0/1121560328 --> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f09cc062590 con 0x7f09bc0a1e10 2024-03-07T07:40:43.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.369+0000 7f09c9ffb700 1 -- 172.21.15.160:0/1121560328 <== mgr.14152 v2:172.21.15.160:6800/419630619 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f09cc062590 con 0x7f09bc0a1e10 2024-03-07T07:40:43.369 INFO:teuthology.orchestra.run.smithi160.stdout:Backend: cephadm 2024-03-07T07:40:43.369 INFO:teuthology.orchestra.run.smithi160.stdout:Available: Yes 2024-03-07T07:40:43.369 INFO:teuthology.orchestra.run.smithi160.stdout:Paused: No 2024-03-07T07:40:43.372 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.371+0000 7f09d2b6a700 1 -- 172.21.15.160:0/1121560328 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f09bc0a1e10 msgr2=0x7f09bc0a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:43.372 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.371+0000 7f09d2b6a700 1 --2- 172.21.15.160:0/1121560328 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f09bc0a1e10 0x7f09bc0a42d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f09b8011490 tx=0x7f09b8009260 comp rx=0 tx=0).stop 2024-03-07T07:40:43.372 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.371+0000 7f09d2b6a700 1 -- 172.21.15.160:0/1121560328 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09cc102180 msgr2=0x7f09cc075ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:43.372 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.371+0000 7f09d2b6a700 1 --2- 172.21.15.160:0/1121560328 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09cc102180 0x7f09cc075ef0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f09c000e970 tx=0x7f09c000ec80 comp rx=0 tx=0).stop 2024-03-07T07:40:43.372 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.371+0000 7f09d2b6a700 1 -- 172.21.15.160:0/1121560328 shutdown_connections 2024-03-07T07:40:43.372 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.371+0000 7f09d2b6a700 1 --2- 172.21.15.160:0/1121560328 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f09bc0a1e10 0x7f09bc0a42d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:43.372 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.371+0000 7f09d2b6a700 1 --2- 172.21.15.160:0/1121560328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09cc103d20 0x7f09cc074a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:43.372 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.371+0000 7f09d2b6a700 1 --2- 172.21.15.160:0/1121560328 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09cc102180 0x7f09cc075ef0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:43.372 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.371+0000 7f09d2b6a700 1 --2- 172.21.15.160:0/1121560328 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09cc103380 0x7f09cc074540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:43.372 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.371+0000 7f09d2b6a700 1 -- 172.21.15.160:0/1121560328 >> 172.21.15.160:0/1121560328 conn(0x7f09cc0fd760 msgr2=0x7f09cc106680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:43.372 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.371+0000 7f09d2b6a700 1 -- 172.21.15.160:0/1121560328 shutdown_connections 2024-03-07T07:40:43.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:43.371+0000 7f09d2b6a700 1 -- 172.21.15.160:0/1121560328 wait complete. 2024-03-07T07:40:44.048 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-07T07:40:44.294 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:40:45.391 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:45 smithi160 ceph-mon[31963]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:40:45.391 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:45 smithi160 ceph-mon[31963]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-07T07:40:45.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:45 smithi177 ceph-mon[36578]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:40:45.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:45 smithi177 ceph-mon[36578]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-07T07:40:45.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:45 smithi178 ceph-mon[36650]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:40:45.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:45 smithi178 ceph-mon[36650]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-07T07:40:45.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:45.932+0000 7f16fffc9700 1 -- 172.21.15.160:0/4091271894 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16f8100850 msgr2=0x7f16f8107860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:45.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:45.932+0000 7f16fffc9700 1 --2- 172.21.15.160:0/4091271894 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16f8100850 0x7f16f8107860 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f16f4009990 tx=0x7f16f4009ca0 comp rx=0 tx=0).stop 2024-03-07T07:40:45.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:45.932+0000 7f16fffc9700 1 -- 172.21.15.160:0/4091271894 shutdown_connections 2024-03-07T07:40:45.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:45.932+0000 7f16fffc9700 1 --2- 172.21.15.160:0/4091271894 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16f8107e30 0x7f16f810a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:45.935 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:45.932+0000 7f16fffc9700 1 --2- 172.21.15.160:0/4091271894 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16f80ffef0 0x7f16f8100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:45.935 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:45.932+0000 7f16fffc9700 1 --2- 172.21.15.160:0/4091271894 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16f8100850 0x7f16f8107860 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:45.935 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:45.932+0000 7f16fffc9700 1 -- 172.21.15.160:0/4091271894 >> 172.21.15.160:0/4091271894 conn(0x7f16f80fb950 msgr2=0x7f16f80fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:45.935 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:45.932+0000 7f16fffc9700 1 -- 172.21.15.160:0/4091271894 shutdown_connections 2024-03-07T07:40:45.935 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:45.932+0000 7f16fffc9700 1 -- 172.21.15.160:0/4091271894 wait complete. 2024-03-07T07:40:45.935 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:45.933+0000 7f16fffc9700 1 Processor -- start 2024-03-07T07:40:45.935 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:45.933+0000 7f16fffc9700 1 -- start start 2024-03-07T07:40:45.935 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:45.933+0000 7f16fffc9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16f80ffef0 0x7f16f81983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:45.935 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:45.933+0000 7f16fffc9700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16f8100850 0x7f16f8198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:45.935 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:45.933+0000 7f16fffc9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16f8107e30 0x7f16f819f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:45.935 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:45.933+0000 7f16fffc9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f810dd70 con 0x7f16f8107e30 2024-03-07T07:40:45.935 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:45.933+0000 7f16fffc9700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f810dbf0 con 0x7f16f8100850 2024-03-07T07:40:45.935 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:45.933+0000 7f16fffc9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f810def0 con 0x7f16f80ffef0 2024-03-07T07:40:45.935 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:45.933+0000 7f16fdd65700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16f80ffef0 0x7f16f81983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:45.936 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:45.933+0000 7f16fdd65700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16f80ffef0 0x7f16f81983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.160:48722/0 (socket says 172.21.15.160:48722) 2024-03-07T07:40:45.936 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:45.933+0000 7f16fdd65700 1 -- 172.21.15.160:0/2833119343 learned_addr learned my addr 172.21.15.160:0/2833119343 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:40:45.936 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:45.933+0000 7f16fe566700 1 --2- 172.21.15.160:0/2833119343 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16f8107e30 0x7f16f819f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:45.936 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:45.933+0000 7f16fd564700 1 --2- 172.21.15.160:0/2833119343 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16f8100850 0x7f16f8198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:45.936 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:45.934+0000 7f16fdd65700 1 -- 172.21.15.160:0/2833119343 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16f8100850 msgr2=0x7f16f8198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:45.936 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:45.934+0000 7f16fdd65700 1 --2- 172.21.15.160:0/2833119343 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16f8100850 0x7f16f8198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:45.936 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:45.934+0000 7f16fdd65700 1 -- 172.21.15.160:0/2833119343 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16f8107e30 msgr2=0x7f16f819f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:45.936 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:45.934+0000 7f16fdd65700 1 --2- 172.21.15.160:0/2833119343 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16f8107e30 0x7f16f819f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:45.936 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:45.934+0000 7f16fdd65700 1 -- 172.21.15.160:0/2833119343 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16f81a0040 con 0x7f16f80ffef0 2024-03-07T07:40:45.936 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:45.934+0000 7f16fe566700 1 --2- 172.21.15.160:0/2833119343 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16f8107e30 0x7f16f819f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:40:45.936 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:45.934+0000 7f16fdd65700 1 --2- 172.21.15.160:0/2833119343 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16f80ffef0 0x7f16f81983e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f16e800e9f0 tx=0x7f16e800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:45.936 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:45.934+0000 7f16eeffd700 1 -- 172.21.15.160:0/2833119343 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e800cbd0 con 0x7f16f80ffef0 2024-03-07T07:40:45.936 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:45.934+0000 7f16fffc9700 1 -- 172.21.15.160:0/2833119343 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16f81a0330 con 0x7f16f80ffef0 2024-03-07T07:40:45.937 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:45.934+0000 7f16eeffd700 1 -- 172.21.15.160:0/2833119343 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f16e800ee90 con 0x7f16f80ffef0 2024-03-07T07:40:45.937 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:45.934+0000 7f16eeffd700 1 -- 172.21.15.160:0/2833119343 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e80056c0 con 0x7f16f80ffef0 2024-03-07T07:40:45.937 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:45.934+0000 7f16fffc9700 1 -- 172.21.15.160:0/2833119343 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16f81a0820 con 0x7f16f80ffef0 2024-03-07T07:40:45.938 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:45.935+0000 7f16fffc9700 1 -- 172.21.15.160:0/2833119343 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16f8068e10 con 0x7f16f80ffef0 2024-03-07T07:40:45.938 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:45.936+0000 7f16eeffd700 1 -- 172.21.15.160:0/2833119343 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f16e8004750 con 0x7f16f80ffef0 2024-03-07T07:40:45.938 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:45.937+0000 7f16eeffd700 1 --2- 172.21.15.160:0/2833119343 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f16e40a1e10 0x7f16e40a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:45.938 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:45.937+0000 7f16fd564700 1 --2- 172.21.15.160:0/2833119343 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f16e40a1e10 0x7f16e40a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:45.938 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:45.937+0000 7f16eeffd700 1 -- 172.21.15.160:0/2833119343 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f16e80ce270 con 0x7f16f80ffef0 2024-03-07T07:40:45.938 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:45.937+0000 7f16fd564700 1 --2- 172.21.15.160:0/2833119343 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f16e40a1e10 0x7f16e40a42d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f16f400b8e0 tx=0x7f16f4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:45.940 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:45.940+0000 7f16eeffd700 1 -- 172.21.15.160:0/2833119343 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f16e8053280 con 0x7f16f80ffef0 2024-03-07T07:40:46.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:46.145+0000 7f16fffc9700 1 -- 172.21.15.160:0/2833119343 --> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f16f8062590 con 0x7f16e40a1e10 2024-03-07T07:40:46.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:46.159+0000 7f16eeffd700 1 -- 172.21.15.160:0/2833119343 <== mgr.14152 v2:172.21.15.160:6800/419630619 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7f16f8062590 con 0x7f16e40a1e10 2024-03-07T07:40:46.159 INFO:teuthology.orchestra.run.smithi160.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-07T07:40:46.159 INFO:teuthology.orchestra.run.smithi160.stdout:mds.a smithi160 running (101s) 15s ago 101s 13.8M - 19.0.0-1848-g8b896a9e e362cd7cebfe 9cb5a59bf356 2024-03-07T07:40:46.159 INFO:teuthology.orchestra.run.smithi160.stdout:mds.b smithi177 running (98s) 32s ago 98s 17.4M - 19.0.0-1848-g8b896a9e e362cd7cebfe 5121d6ee5cd4 2024-03-07T07:40:46.159 INFO:teuthology.orchestra.run.smithi160.stdout:mds.c smithi178 running (112s) 32s ago 112s 16.7M - 19.0.0-1848-g8b896a9e e362cd7cebfe 51b264020bfc 2024-03-07T07:40:46.160 INFO:teuthology.orchestra.run.smithi160.stdout:mds.d smithi160 running (2m) 15s ago 2m 16.2M - 19.0.0-1848-g8b896a9e e362cd7cebfe b876bcd750c2 2024-03-07T07:40:46.160 INFO:teuthology.orchestra.run.smithi160.stdout:mds.e smithi177 running (118s) 32s ago 118s 16.1M - 19.0.0-1848-g8b896a9e e362cd7cebfe f8fb2c249ecd 2024-03-07T07:40:46.160 INFO:teuthology.orchestra.run.smithi160.stdout:mds.f smithi178 running (115s) 32s ago 115s 17.6M - 19.0.0-1848-g8b896a9e e362cd7cebfe 4cedda18d060 2024-03-07T07:40:46.160 INFO:teuthology.orchestra.run.smithi160.stdout:mds.g smithi160 running (104s) 15s ago 103s 13.6M - 19.0.0-1848-g8b896a9e e362cd7cebfe ad2e3e0c37fb 2024-03-07T07:40:46.160 INFO:teuthology.orchestra.run.smithi160.stdout:mds.h smithi177 running (109s) 32s ago 109s 15.8M - 19.0.0-1848-g8b896a9e e362cd7cebfe 41cf76184494 2024-03-07T07:40:46.160 INFO:teuthology.orchestra.run.smithi160.stdout:mds.i smithi178 running (107s) 32s ago 106s 25.9M - 19.0.0-1848-g8b896a9e e362cd7cebfe cc48c1537daf 2024-03-07T07:40:46.160 INFO:teuthology.orchestra.run.smithi160.stdout:mds.j smithi160 running (2m) 15s ago 2m 15.3M - 19.0.0-1848-g8b896a9e e362cd7cebfe 219cb8715400 2024-03-07T07:40:46.160 INFO:teuthology.orchestra.run.smithi160.stdout:mds.k smithi177 running (2m) 32s ago 2m 15.3M - 19.0.0-1848-g8b896a9e e362cd7cebfe 3a9e5ab28686 2024-03-07T07:40:46.160 INFO:teuthology.orchestra.run.smithi160.stdout:mds.l smithi178 running (2m) 32s ago 2m 16.5M - 19.0.0-1848-g8b896a9e e362cd7cebfe 0ca9fa5a8119 2024-03-07T07:40:46.160 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.x smithi160 *:9283,8765 running (9m) 15s ago 9m 513M - 19.0.0-1848-g8b896a9e e362cd7cebfe c6ae33bdae0b 2024-03-07T07:40:46.160 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.y smithi177 *:8443,8765 running (7m) 32s ago 7m 452M - 19.0.0-1848-g8b896a9e e362cd7cebfe 44e7a1cb82b1 2024-03-07T07:40:46.160 INFO:teuthology.orchestra.run.smithi160.stdout:mgr.z smithi178 *:8443,8765 running (7m) 32s ago 7m 451M - 19.0.0-1848-g8b896a9e e362cd7cebfe e1f247c6e191 2024-03-07T07:40:46.160 INFO:teuthology.orchestra.run.smithi160.stdout:mon.a smithi160 running (9m) 15s ago 9m 50.0M 2048M 19.0.0-1848-g8b896a9e e362cd7cebfe 23e49f05d5ab 2024-03-07T07:40:46.160 INFO:teuthology.orchestra.run.smithi160.stdout:mon.b smithi177 running (7m) 32s ago 7m 38.3M 2048M 19.0.0-1848-g8b896a9e e362cd7cebfe f09afba66d4c 2024-03-07T07:40:46.160 INFO:teuthology.orchestra.run.smithi160.stdout:mon.c smithi178 running (7m) 32s ago 7m 38.2M 2048M 19.0.0-1848-g8b896a9e e362cd7cebfe 3f6302e362f6 2024-03-07T07:40:46.161 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 smithi160 running (7m) 15s ago 7m 47.2M 4272M 19.0.0-1848-g8b896a9e e362cd7cebfe 75b6c5128948 2024-03-07T07:40:46.161 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 smithi177 running (6m) 32s ago 6m 45.3M 4270M 19.0.0-1848-g8b896a9e e362cd7cebfe 65b9067952e3 2024-03-07T07:40:46.161 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 smithi178 running (6m) 32s ago 6m 55.9M 4270M 19.0.0-1848-g8b896a9e e362cd7cebfe 6ce198dedec2 2024-03-07T07:40:46.161 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 smithi160 running (5m) 15s ago 5m 45.3M 4272M 19.0.0-1848-g8b896a9e e362cd7cebfe ecbaaa7faf76 2024-03-07T07:40:46.161 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 smithi177 running (5m) 32s ago 5m 42.4M 4270M 19.0.0-1848-g8b896a9e e362cd7cebfe 3a198df1c446 2024-03-07T07:40:46.161 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 smithi178 running (5m) 32s ago 5m 44.1M 4270M 19.0.0-1848-g8b896a9e e362cd7cebfe b4fa9bf9b2b4 2024-03-07T07:40:46.161 INFO:teuthology.orchestra.run.smithi160.stdout:osd.6 smithi160 running (4m) 15s ago 4m 51.9M 4272M 19.0.0-1848-g8b896a9e e362cd7cebfe 76f2218e15b2 2024-03-07T07:40:46.161 INFO:teuthology.orchestra.run.smithi160.stdout:osd.7 smithi177 running (4m) 32s ago 4m 54.5M 4270M 19.0.0-1848-g8b896a9e e362cd7cebfe ea563134bafc 2024-03-07T07:40:46.161 INFO:teuthology.orchestra.run.smithi160.stdout:osd.8 smithi178 running (3m) 32s ago 3m 41.8M 4270M 19.0.0-1848-g8b896a9e e362cd7cebfe bb785fe71ea5 2024-03-07T07:40:46.161 INFO:teuthology.orchestra.run.smithi160.stdout:osd.9 smithi160 running (3m) 15s ago 3m 41.1M 4272M 19.0.0-1848-g8b896a9e e362cd7cebfe f3e963a7dfe3 2024-03-07T07:40:46.161 INFO:teuthology.orchestra.run.smithi160.stdout:osd.10 smithi177 running (2m) 32s ago 2m 43.3M 4270M 19.0.0-1848-g8b896a9e e362cd7cebfe 85c773932687 2024-03-07T07:40:46.161 INFO:teuthology.orchestra.run.smithi160.stdout:osd.11 smithi178 running (2m) 32s ago 2m 52.2M 4270M 19.0.0-1848-g8b896a9e e362cd7cebfe 809c2f9a6671 2024-03-07T07:40:46.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:46.161+0000 7f16fffc9700 1 -- 172.21.15.160:0/2833119343 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f16e40a1e10 msgr2=0x7f16e40a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:46.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:46.161+0000 7f16fffc9700 1 --2- 172.21.15.160:0/2833119343 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f16e40a1e10 0x7f16e40a42d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f16f400b8e0 tx=0x7f16f4005fd0 comp rx=0 tx=0).stop 2024-03-07T07:40:46.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:46.161+0000 7f16fffc9700 1 -- 172.21.15.160:0/2833119343 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16f80ffef0 msgr2=0x7f16f81983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:46.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:46.161+0000 7f16fffc9700 1 --2- 172.21.15.160:0/2833119343 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16f80ffef0 0x7f16f81983e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f16e800e9f0 tx=0x7f16e800ed00 comp rx=0 tx=0).stop 2024-03-07T07:40:46.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:46.161+0000 7f16fffc9700 1 -- 172.21.15.160:0/2833119343 shutdown_connections 2024-03-07T07:40:46.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:46.161+0000 7f16fffc9700 1 --2- 172.21.15.160:0/2833119343 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16f8107e30 0x7f16f819f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:46.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:46.161+0000 7f16fffc9700 1 --2- 172.21.15.160:0/2833119343 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f16e40a1e10 0x7f16e40a42d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:46.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:46.161+0000 7f16fffc9700 1 --2- 172.21.15.160:0/2833119343 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16f80ffef0 0x7f16f81983e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:46.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:46.161+0000 7f16fffc9700 1 --2- 172.21.15.160:0/2833119343 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16f8100850 0x7f16f8198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:46.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:46.161+0000 7f16fffc9700 1 -- 172.21.15.160:0/2833119343 >> 172.21.15.160:0/2833119343 conn(0x7f16f80fb950 msgr2=0x7f16f8103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:46.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:46.161+0000 7f16fffc9700 1 -- 172.21.15.160:0/2833119343 shutdown_connections 2024-03-07T07:40:46.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:46.161+0000 7f16fffc9700 1 -- 172.21.15.160:0/2833119343 wait complete. 2024-03-07T07:40:46.912 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-07T07:40:47.153 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:40:47.539 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:47 smithi160 ceph-mon[31963]: from='client.24973 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:40:47.539 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:47 smithi160 ceph-mon[31963]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-07T07:40:47.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:47 smithi177 ceph-mon[36578]: from='client.24973 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:40:47.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:47 smithi177 ceph-mon[36578]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-07T07:40:47.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:47 smithi178 ceph-mon[36650]: from='client.24973 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:40:47.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:47 smithi178 ceph-mon[36650]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-07T07:40:48.946 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:48.944+0000 7f7110fe9700 1 -- 172.21.15.160:0/2622703501 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f710c103370 msgr2=0x7f710c1037d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:48.946 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:48.944+0000 7f7110fe9700 1 --2- 172.21.15.160:0/2622703501 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f710c103370 0x7f710c1037d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f71000099c0 tx=0x7f7100009cd0 comp rx=0 tx=0).stop 2024-03-07T07:40:48.946 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:48.944+0000 7f7110fe9700 1 -- 172.21.15.160:0/2622703501 shutdown_connections 2024-03-07T07:40:48.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:48.944+0000 7f7110fe9700 1 --2- 172.21.15.160:0/2622703501 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f710c103d10 0x7f710c10a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:48.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:48.944+0000 7f7110fe9700 1 --2- 172.21.15.160:0/2622703501 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f710c102170 0x7f710c102590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:48.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:48.944+0000 7f7110fe9700 1 --2- 172.21.15.160:0/2622703501 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f710c103370 0x7f710c1037d0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:48.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:48.944+0000 7f7110fe9700 1 -- 172.21.15.160:0/2622703501 >> 172.21.15.160:0/2622703501 conn(0x7f710c0fd730 msgr2=0x7f710c0ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:48.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:48.944+0000 7f7110fe9700 1 -- 172.21.15.160:0/2622703501 shutdown_connections 2024-03-07T07:40:48.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:48.944+0000 7f7110fe9700 1 -- 172.21.15.160:0/2622703501 wait complete. 2024-03-07T07:40:48.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:48.945+0000 7f7110fe9700 1 Processor -- start 2024-03-07T07:40:48.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:48.945+0000 7f7110fe9700 1 -- start start 2024-03-07T07:40:48.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:48.945+0000 7f7110fe9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f710c102170 0x7f710c198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:48.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:48.945+0000 7f7110fe9700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f710c103370 0x7f710c1988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:48.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:48.945+0000 7f7110fe9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f710c103d10 0x7f710c19f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:48.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:48.945+0000 7f7110fe9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710c10dd60 con 0x7f710c102170 2024-03-07T07:40:48.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:48.945+0000 7f7110fe9700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710c10dbe0 con 0x7f710c103370 2024-03-07T07:40:48.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:48.945+0000 7f7110fe9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710c10dee0 con 0x7f710c103d10 2024-03-07T07:40:48.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:48.945+0000 7f710a59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f710c102170 0x7f710c198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:48.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:48.945+0000 7f710a59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f710c102170 0x7f710c198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59318/0 (socket says 172.21.15.160:59318) 2024-03-07T07:40:48.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:48.945+0000 7f710a59c700 1 -- 172.21.15.160:0/3400514824 learned_addr learned my addr 172.21.15.160:0/3400514824 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:40:48.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:48.945+0000 7f7109d9b700 1 --2- 172.21.15.160:0/3400514824 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f710c103370 0x7f710c1988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:48.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:48.945+0000 7f710ad9d700 1 --2- 172.21.15.160:0/3400514824 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f710c103d10 0x7f710c19f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:48.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:48.945+0000 7f710a59c700 1 -- 172.21.15.160:0/3400514824 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f710c103d10 msgr2=0x7f710c19f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:48.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:48.945+0000 7f710a59c700 1 --2- 172.21.15.160:0/3400514824 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f710c103d10 0x7f710c19f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:48.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:48.945+0000 7f710a59c700 1 -- 172.21.15.160:0/3400514824 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f710c103370 msgr2=0x7f710c1988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:48.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:48.945+0000 7f710a59c700 1 --2- 172.21.15.160:0/3400514824 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f710c103370 0x7f710c1988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:48.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:48.945+0000 7f710a59c700 1 -- 172.21.15.160:0/3400514824 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f710c19ffe0 con 0x7f710c102170 2024-03-07T07:40:48.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:48.945+0000 7f7109d9b700 1 --2- 172.21.15.160:0/3400514824 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f710c103370 0x7f710c1988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:40:48.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:48.945+0000 7f710a59c700 1 --2- 172.21.15.160:0/3400514824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f710c102170 0x7f710c198380 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f70fc00e9a0 tx=0x7f70fc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:48.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:48.946+0000 7f70fb7fe700 1 -- 172.21.15.160:0/3400514824 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc004220 con 0x7f710c102170 2024-03-07T07:40:48.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:48.946+0000 7f7110fe9700 1 -- 172.21.15.160:0/3400514824 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f710c1a02d0 con 0x7f710c102170 2024-03-07T07:40:48.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:48.946+0000 7f70fb7fe700 1 -- 172.21.15.160:0/3400514824 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f70fc0043c0 con 0x7f710c102170 2024-03-07T07:40:48.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:48.946+0000 7f70fb7fe700 1 -- 172.21.15.160:0/3400514824 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc003df0 con 0x7f710c102170 2024-03-07T07:40:48.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:48.946+0000 7f7110fe9700 1 -- 172.21.15.160:0/3400514824 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f710c1a0810 con 0x7f710c102170 2024-03-07T07:40:48.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:48.947+0000 7f7110fe9700 1 -- 172.21.15.160:0/3400514824 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f710c068e10 con 0x7f710c102170 2024-03-07T07:40:48.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:48.948+0000 7f70fb7fe700 1 -- 172.21.15.160:0/3400514824 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f70fc00c810 con 0x7f710c102170 2024-03-07T07:40:48.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:48.948+0000 7f70fb7fe700 1 --2- 172.21.15.160:0/3400514824 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f70f40a1e10 0x7f70f40a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:48.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:48.948+0000 7f7109d9b700 1 --2- 172.21.15.160:0/3400514824 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f70f40a1e10 0x7f70f40a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:48.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:48.948+0000 7f70fb7fe700 1 -- 172.21.15.160:0/3400514824 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f70fc0cdfc0 con 0x7f710c102170 2024-03-07T07:40:48.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:48.949+0000 7f7109d9b700 1 --2- 172.21.15.160:0/3400514824 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f70f40a1e10 0x7f70f40a42d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7100000c00 tx=0x7f7100005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:48.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:48.952+0000 7f70fb7fe700 1 -- 172.21.15.160:0/3400514824 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f70fc09b520 con 0x7f710c102170 2024-03-07T07:40:49.157 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:49.157+0000 7f7110fe9700 1 -- 172.21.15.160:0/3400514824 --> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f710c062590 con 0x7f70f40a1e10 2024-03-07T07:40:49.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:49.163+0000 7f70fb7fe700 1 -- 172.21.15.160:0/3400514824 <== mgr.14152 v2:172.21.15.160:6800/419630619 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f710c062590 con 0x7f70f40a1e10 2024-03-07T07:40:49.163 INFO:teuthology.orchestra.run.smithi160.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-07T07:40:49.164 INFO:teuthology.orchestra.run.smithi160.stdout:mds.all 12/12 35s ago 2m smithi160=a;smithi160=d;smithi160=g;smithi160=j;smithi177=b;smithi177=e;smithi177=h;smithi177=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l;count:12 2024-03-07T07:40:49.164 INFO:teuthology.orchestra.run.smithi160.stdout:mgr 3/3 35s ago 7m smithi160=x;smithi177=y;smithi178=z;count:3 2024-03-07T07:40:49.164 INFO:teuthology.orchestra.run.smithi160.stdout:mon 3/3 35s ago 8m smithi160:172.21.15.160=a;smithi177:172.21.15.177=b;smithi178:172.21.15.178=c;count:3 2024-03-07T07:40:49.164 INFO:teuthology.orchestra.run.smithi160.stdout:osd 12 35s ago - 2024-03-07T07:40:49.166 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:49.165+0000 7f7110fe9700 1 -- 172.21.15.160:0/3400514824 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f70f40a1e10 msgr2=0x7f70f40a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:49.166 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:49.165+0000 7f7110fe9700 1 --2- 172.21.15.160:0/3400514824 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f70f40a1e10 0x7f70f40a42d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7100000c00 tx=0x7f7100005fb0 comp rx=0 tx=0).stop 2024-03-07T07:40:49.166 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:49.165+0000 7f7110fe9700 1 -- 172.21.15.160:0/3400514824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f710c102170 msgr2=0x7f710c198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:49.166 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:49.165+0000 7f7110fe9700 1 --2- 172.21.15.160:0/3400514824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f710c102170 0x7f710c198380 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f70fc00e9a0 tx=0x7f70fc00ecb0 comp rx=0 tx=0).stop 2024-03-07T07:40:49.166 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:49.165+0000 7f7110fe9700 1 -- 172.21.15.160:0/3400514824 shutdown_connections 2024-03-07T07:40:49.166 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:49.165+0000 7f7110fe9700 1 --2- 172.21.15.160:0/3400514824 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f70f40a1e10 0x7f70f40a42d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:49.166 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:49.165+0000 7f7110fe9700 1 --2- 172.21.15.160:0/3400514824 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f710c103d10 0x7f710c19f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:49.166 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:49.165+0000 7f7110fe9700 1 --2- 172.21.15.160:0/3400514824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f710c102170 0x7f710c198380 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:49.166 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:49.165+0000 7f7110fe9700 1 --2- 172.21.15.160:0/3400514824 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f710c103370 0x7f710c1988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:49.166 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:49.165+0000 7f7110fe9700 1 -- 172.21.15.160:0/3400514824 >> 172.21.15.160:0/3400514824 conn(0x7f710c0fd730 msgr2=0x7f710c106670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:49.166 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:49.165+0000 7f7110fe9700 1 -- 172.21.15.160:0/3400514824 shutdown_connections 2024-03-07T07:40:49.167 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:49.165+0000 7f7110fe9700 1 -- 172.21.15.160:0/3400514824 wait complete. 2024-03-07T07:40:49.427 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:49 smithi160 ceph-mon[31963]: pgmap v337: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-07T07:40:49.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:49 smithi177 ceph-mon[36578]: pgmap v337: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-07T07:40:49.607 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:49 smithi178 ceph-mon[36650]: pgmap v337: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-07T07:40:49.873 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-07T07:40:50.111 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:40:50.575 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:50 smithi160 ceph-mon[31963]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:40:50.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:50 smithi178 ceph-mon[36650]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:40:50.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:50 smithi177 ceph-mon[36578]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:40:51.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:51 smithi160 ceph-mon[31963]: pgmap v338: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-03-07T07:40:51.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:51 smithi178 ceph-mon[36650]: pgmap v338: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-03-07T07:40:51.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:51 smithi177 ceph-mon[36578]: pgmap v338: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-03-07T07:40:51.997 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:51.997+0000 7f2961be9700 1 -- 172.21.15.160:0/2536665794 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f295c102c20 msgr2=0x7f295c1094d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:51.997 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:51.997+0000 7f2961be9700 1 --2- 172.21.15.160:0/2536665794 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f295c102c20 0x7f295c1094d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f294c0099c0 tx=0x7f294c009cd0 comp rx=0 tx=0).stop 2024-03-07T07:40:51.997 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:51.997+0000 7f2961be9700 1 -- 172.21.15.160:0/2536665794 shutdown_connections 2024-03-07T07:40:51.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:51.997+0000 7f2961be9700 1 --2- 172.21.15.160:0/2536665794 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f295c102c20 0x7f295c1094d0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:51.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:51.997+0000 7f2961be9700 1 --2- 172.21.15.160:0/2536665794 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f295c101080 0x7f295c1014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:51.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:51.997+0000 7f2961be9700 1 --2- 172.21.15.160:0/2536665794 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f295c102280 0x7f295c1026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:51.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:51.997+0000 7f2961be9700 1 -- 172.21.15.160:0/2536665794 >> 172.21.15.160:0/2536665794 conn(0x7f295c0fc620 msgr2=0x7f295c0fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:51.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:51.997+0000 7f2961be9700 1 -- 172.21.15.160:0/2536665794 shutdown_connections 2024-03-07T07:40:51.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:51.997+0000 7f2961be9700 1 -- 172.21.15.160:0/2536665794 wait complete. 2024-03-07T07:40:52.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:51.997+0000 7f2961be9700 1 Processor -- start 2024-03-07T07:40:52.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:51.998+0000 7f2961be9700 1 -- start start 2024-03-07T07:40:52.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:51.998+0000 7f2961be9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f295c101080 0x7f295c197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:52.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:51.998+0000 7f2961be9700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f295c102280 0x7f295c197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:52.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:51.998+0000 7f2961be9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f295c102c20 0x7f295c19e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:52.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:51.998+0000 7f2961be9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295c10cc60 con 0x7f295c102c20 2024-03-07T07:40:52.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:51.998+0000 7f2961be9700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295c10cae0 con 0x7f295c102280 2024-03-07T07:40:52.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:51.998+0000 7f2961be9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295c10cde0 con 0x7f295c101080 2024-03-07T07:40:52.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:51.998+0000 7f295bfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f295c102c20 0x7f295c19e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:52.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:51.998+0000 7f295b7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f295c101080 0x7f295c197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:52.001 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:51.998+0000 7f295affd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f295c102280 0x7f295c197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:52.001 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:51.998+0000 7f295bfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f295c102c20 0x7f295c19e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:51854/0 (socket says 172.21.15.160:51854) 2024-03-07T07:40:52.001 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:51.998+0000 7f295bfff700 1 -- 172.21.15.160:0/3289187333 learned_addr learned my addr 172.21.15.160:0/3289187333 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:40:52.001 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:51.998+0000 7f295bfff700 1 -- 172.21.15.160:0/3289187333 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f295c101080 msgr2=0x7f295c197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:52.001 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:51.998+0000 7f295bfff700 1 --2- 172.21.15.160:0/3289187333 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f295c101080 0x7f295c197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:52.001 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:51.998+0000 7f295bfff700 1 -- 172.21.15.160:0/3289187333 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f295c102280 msgr2=0x7f295c197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:52.001 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:51.998+0000 7f295bfff700 1 --2- 172.21.15.160:0/3289187333 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f295c102280 0x7f295c197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:52.001 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:51.998+0000 7f295bfff700 1 -- 172.21.15.160:0/3289187333 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f295c069f00 con 0x7f295c102c20 2024-03-07T07:40:52.001 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:51.998+0000 7f295affd700 1 --2- 172.21.15.160:0/3289187333 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f295c102280 0x7f295c197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:40:52.001 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:51.998+0000 7f295bfff700 1 --2- 172.21.15.160:0/3289187333 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f295c102c20 0x7f295c19e810 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f294c006010 tx=0x7f294c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:52.001 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:51.999+0000 7f2958ff9700 1 -- 172.21.15.160:0/3289187333 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c009e60 con 0x7f295c102c20 2024-03-07T07:40:52.001 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:51.999+0000 7f2958ff9700 1 -- 172.21.15.160:0/3289187333 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f294c003960 con 0x7f295c102c20 2024-03-07T07:40:52.001 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:51.999+0000 7f2961be9700 1 -- 172.21.15.160:0/3289187333 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f295c06a190 con 0x7f295c102c20 2024-03-07T07:40:52.002 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:51.999+0000 7f2958ff9700 1 -- 172.21.15.160:0/3289187333 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c0173f0 con 0x7f295c102c20 2024-03-07T07:40:52.002 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:51.999+0000 7f2961be9700 1 -- 172.21.15.160:0/3289187333 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f295c06a670 con 0x7f295c102c20 2024-03-07T07:40:52.002 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:52.000+0000 7f2961be9700 1 -- 172.21.15.160:0/3289187333 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f295c068e10 con 0x7f295c102c20 2024-03-07T07:40:52.002 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:52.001+0000 7f2958ff9700 1 -- 172.21.15.160:0/3289187333 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f294c003b10 con 0x7f295c102c20 2024-03-07T07:40:52.003 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:52.001+0000 7f2958ff9700 1 --2- 172.21.15.160:0/3289187333 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f29440a1e90 0x7f29440a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:52.003 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:52.001+0000 7f295b7fe700 1 --2- 172.21.15.160:0/3289187333 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f29440a1e90 0x7f29440a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:52.003 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:52.001+0000 7f2958ff9700 1 -- 172.21.15.160:0/3289187333 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f294c0ceb00 con 0x7f295c102c20 2024-03-07T07:40:52.003 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:52.002+0000 7f295b7fe700 1 --2- 172.21.15.160:0/3289187333 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f29440a1e90 0x7f29440a4350 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f295c1020e0 tx=0x7f2954009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:52.006 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:52.006+0000 7f2958ff9700 1 -- 172.21.15.160:0/3289187333 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f294c013070 con 0x7f295c102c20 2024-03-07T07:40:52.208 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:52.208+0000 7f2961be9700 1 -- 172.21.15.160:0/3289187333 --> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f295c062590 con 0x7f29440a1e90 2024-03-07T07:40:52.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:52.209+0000 7f2958ff9700 1 -- 172.21.15.160:0/3289187333 <== mgr.14152 v2:172.21.15.160:6800/419630619 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f295c062590 con 0x7f29440a1e90 2024-03-07T07:40:52.210 INFO:teuthology.orchestra.run.smithi160.stdout:HOST ADDR LABELS STATUS 2024-03-07T07:40:52.210 INFO:teuthology.orchestra.run.smithi160.stdout:smithi160 172.21.15.160 2024-03-07T07:40:52.210 INFO:teuthology.orchestra.run.smithi160.stdout:smithi177 172.21.15.177 2024-03-07T07:40:52.210 INFO:teuthology.orchestra.run.smithi160.stdout:smithi178 172.21.15.178 2024-03-07T07:40:52.210 INFO:teuthology.orchestra.run.smithi160.stdout:3 hosts in cluster 2024-03-07T07:40:52.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:52.211+0000 7f2961be9700 1 -- 172.21.15.160:0/3289187333 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f29440a1e90 msgr2=0x7f29440a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:52.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:52.211+0000 7f2961be9700 1 --2- 172.21.15.160:0/3289187333 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f29440a1e90 0x7f29440a4350 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f295c1020e0 tx=0x7f2954009260 comp rx=0 tx=0).stop 2024-03-07T07:40:52.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:52.211+0000 7f2961be9700 1 -- 172.21.15.160:0/3289187333 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f295c102c20 msgr2=0x7f295c19e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:52.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:52.211+0000 7f2961be9700 1 --2- 172.21.15.160:0/3289187333 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f295c102c20 0x7f295c19e810 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f294c006010 tx=0x7f294c00f690 comp rx=0 tx=0).stop 2024-03-07T07:40:52.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:52.211+0000 7f2961be9700 1 -- 172.21.15.160:0/3289187333 shutdown_connections 2024-03-07T07:40:52.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:52.212+0000 7f2961be9700 1 --2- 172.21.15.160:0/3289187333 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f295c102c20 0x7f295c19e810 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:52.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:52.212+0000 7f2961be9700 1 --2- 172.21.15.160:0/3289187333 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f29440a1e90 0x7f29440a4350 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:52.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:52.212+0000 7f2961be9700 1 --2- 172.21.15.160:0/3289187333 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f295c101080 0x7f295c197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:52.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:52.212+0000 7f2961be9700 1 --2- 172.21.15.160:0/3289187333 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f295c102280 0x7f295c197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:52.214 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:52.212+0000 7f2961be9700 1 -- 172.21.15.160:0/3289187333 >> 172.21.15.160:0/3289187333 conn(0x7f295c0fc620 msgr2=0x7f295c105580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:52.214 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:52.212+0000 7f2961be9700 1 -- 172.21.15.160:0/3289187333 shutdown_connections 2024-03-07T07:40:52.214 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:52.212+0000 7f2961be9700 1 -- 172.21.15.160:0/3289187333 wait complete. 2024-03-07T07:40:52.799 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-07T07:40:53.041 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:40:53.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:53 smithi160 ceph-mon[31963]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:40:53.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:53 smithi160 ceph-mon[31963]: pgmap v339: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-07T07:40:53.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:53 smithi178 ceph-mon[36650]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:40:53.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:53 smithi178 ceph-mon[36650]: pgmap v339: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-07T07:40:53.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:53 smithi177 ceph-mon[36578]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:40:53.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:53 smithi177 ceph-mon[36578]: pgmap v339: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-07T07:40:54.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:54.795+0000 7f80fe2f2700 1 -- 172.21.15.160:0/334354583 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80f8103d20 msgr2=0x7f80f810a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:54.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:54.795+0000 7f80fe2f2700 1 --2- 172.21.15.160:0/334354583 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80f8103d20 0x7f80f810a5d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f80f000b0d0 tx=0x7f80f000b3e0 comp rx=0 tx=0).stop 2024-03-07T07:40:54.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:54.795+0000 7f80fe2f2700 1 -- 172.21.15.160:0/334354583 shutdown_connections 2024-03-07T07:40:54.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:54.795+0000 7f80fe2f2700 1 --2- 172.21.15.160:0/334354583 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80f8103d20 0x7f80f810a5d0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:54.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:54.795+0000 7f80fe2f2700 1 --2- 172.21.15.160:0/334354583 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80f8102180 0x7f80f81025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:54.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:54.795+0000 7f80fe2f2700 1 --2- 172.21.15.160:0/334354583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80f8103380 0x7f80f81037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:54.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:54.795+0000 7f80fe2f2700 1 -- 172.21.15.160:0/334354583 >> 172.21.15.160:0/334354583 conn(0x7f80f80fd760 msgr2=0x7f80f80ffb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:54.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:54.796+0000 7f80fe2f2700 1 -- 172.21.15.160:0/334354583 shutdown_connections 2024-03-07T07:40:54.799 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:54.796+0000 7f80fe2f2700 1 -- 172.21.15.160:0/334354583 wait complete. 2024-03-07T07:40:54.799 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:54.796+0000 7f80fe2f2700 1 Processor -- start 2024-03-07T07:40:54.799 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:54.796+0000 7f80fe2f2700 1 -- start start 2024-03-07T07:40:54.799 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:54.797+0000 7f80fe2f2700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80f8102180 0x7f80f8198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:54.799 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:54.797+0000 7f80fe2f2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80f8103380 0x7f80f8198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:54.799 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:54.797+0000 7f80fe2f2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80f8103d20 0x7f80f819f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:54.799 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:54.797+0000 7f80fe2f2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f810dd70 con 0x7f80f8103380 2024-03-07T07:40:54.799 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:54.797+0000 7f80fe2f2700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f810dbf0 con 0x7f80f8102180 2024-03-07T07:40:54.799 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:54.797+0000 7f80fe2f2700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f810def0 con 0x7f80f8103d20 2024-03-07T07:40:54.799 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:54.797+0000 7f80f7fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80f8102180 0x7f80f8198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:54.799 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:54.797+0000 7f80f77fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80f8103380 0x7f80f8198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:54.799 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:54.797+0000 7f80f7fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80f8102180 0x7f80f8198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.160:41872/0 (socket says 172.21.15.160:41872) 2024-03-07T07:40:54.799 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:54.797+0000 7f80f7fff700 1 -- 172.21.15.160:0/4088573367 learned_addr learned my addr 172.21.15.160:0/4088573367 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:40:54.799 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:54.797+0000 7f80fc88f700 1 --2- 172.21.15.160:0/4088573367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80f8103d20 0x7f80f819f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:54.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:54.797+0000 7f80f77fe700 1 -- 172.21.15.160:0/4088573367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80f8103d20 msgr2=0x7f80f819f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:54.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:54.797+0000 7f80f77fe700 1 --2- 172.21.15.160:0/4088573367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80f8103d20 0x7f80f819f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:54.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:54.797+0000 7f80f77fe700 1 -- 172.21.15.160:0/4088573367 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80f8102180 msgr2=0x7f80f8198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:54.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:54.797+0000 7f80f77fe700 1 --2- 172.21.15.160:0/4088573367 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80f8102180 0x7f80f8198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:54.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:54.797+0000 7f80f77fe700 1 -- 172.21.15.160:0/4088573367 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80f819ff60 con 0x7f80f8103380 2024-03-07T07:40:54.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:54.797+0000 7f80f7fff700 1 --2- 172.21.15.160:0/4088573367 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80f8102180 0x7f80f8198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:40:54.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:54.797+0000 7f80fc88f700 1 --2- 172.21.15.160:0/4088573367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80f8103d20 0x7f80f819f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:40:54.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:54.797+0000 7f80f77fe700 1 --2- 172.21.15.160:0/4088573367 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80f8103380 0x7f80f8198840 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f80ec00ea20 tx=0x7f80ec00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:54.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:54.797+0000 7f80f57fa700 1 -- 172.21.15.160:0/4088573367 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ec00cbd0 con 0x7f80f8103380 2024-03-07T07:40:54.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:54.797+0000 7f80f57fa700 1 -- 172.21.15.160:0/4088573367 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f80ec00ee60 con 0x7f80f8103380 2024-03-07T07:40:54.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:54.798+0000 7f80fe2f2700 1 -- 172.21.15.160:0/4088573367 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80f81a0250 con 0x7f80f8103380 2024-03-07T07:40:54.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:54.798+0000 7f80f57fa700 1 -- 172.21.15.160:0/4088573367 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ec018680 con 0x7f80f8103380 2024-03-07T07:40:54.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:54.798+0000 7f80fe2f2700 1 -- 172.21.15.160:0/4088573367 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80f81a0790 con 0x7f80f8103380 2024-03-07T07:40:54.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:54.799+0000 7f80fe2f2700 1 -- 172.21.15.160:0/4088573367 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80f8068e10 con 0x7f80f8103380 2024-03-07T07:40:54.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:54.800+0000 7f80f57fa700 1 -- 172.21.15.160:0/4088573367 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f80ec018820 con 0x7f80f8103380 2024-03-07T07:40:54.802 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:54.800+0000 7f80f57fa700 1 --2- 172.21.15.160:0/4088573367 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f80e00a1e10 0x7f80e00a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:54.802 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:54.800+0000 7f80f7fff700 1 --2- 172.21.15.160:0/4088573367 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f80e00a1e10 0x7f80e00a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:54.802 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:54.800+0000 7f80f57fa700 1 -- 172.21.15.160:0/4088573367 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f80ec0ce2f0 con 0x7f80f8103380 2024-03-07T07:40:54.802 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:54.801+0000 7f80f7fff700 1 --2- 172.21.15.160:0/4088573367 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f80e00a1e10 0x7f80e00a42d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f80e8005950 tx=0x7f80e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:54.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:54.804+0000 7f80f57fa700 1 -- 172.21.15.160:0/4088573367 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f80ec09b720 con 0x7f80f8103380 2024-03-07T07:40:55.008 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:55.008+0000 7f80fe2f2700 1 -- 172.21.15.160:0/4088573367 --> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f80f8062650 con 0x7f80e00a1e10 2024-03-07T07:40:55.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:55.010+0000 7f80f57fa700 1 -- 172.21.15.160:0/4088573367 <== mgr.14152 v2:172.21.15.160:6800/419630619 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f80f8062650 con 0x7f80e00a1e10 2024-03-07T07:40:55.010 INFO:teuthology.orchestra.run.smithi160.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-07T07:40:55.010 INFO:teuthology.orchestra.run.smithi160.stdout:smithi160 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300JY400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-07T07:40:55.010 INFO:teuthology.orchestra.run.smithi160.stdout:smithi177 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233007S400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-03-07T07:40:55.010 INFO:teuthology.orchestra.run.smithi160.stdout:smithi178 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB75130029480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-03-07T07:40:55.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:55.012+0000 7f80fe2f2700 1 -- 172.21.15.160:0/4088573367 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f80e00a1e10 msgr2=0x7f80e00a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:55.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:55.012+0000 7f80fe2f2700 1 --2- 172.21.15.160:0/4088573367 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f80e00a1e10 0x7f80e00a42d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f80e8005950 tx=0x7f80e8009260 comp rx=0 tx=0).stop 2024-03-07T07:40:55.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:55.012+0000 7f80fe2f2700 1 -- 172.21.15.160:0/4088573367 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80f8103380 msgr2=0x7f80f8198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:55.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:55.012+0000 7f80fe2f2700 1 --2- 172.21.15.160:0/4088573367 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80f8103380 0x7f80f8198840 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f80ec00ea20 tx=0x7f80ec00ed30 comp rx=0 tx=0).stop 2024-03-07T07:40:55.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:55.012+0000 7f80fe2f2700 1 -- 172.21.15.160:0/4088573367 shutdown_connections 2024-03-07T07:40:55.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:55.012+0000 7f80fe2f2700 1 --2- 172.21.15.160:0/4088573367 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f80e00a1e10 0x7f80e00a42d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:55.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:55.012+0000 7f80fe2f2700 1 --2- 172.21.15.160:0/4088573367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80f8103d20 0x7f80f819f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:55.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:55.012+0000 7f80fe2f2700 1 --2- 172.21.15.160:0/4088573367 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80f8102180 0x7f80f8198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:55.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:55.012+0000 7f80fe2f2700 1 --2- 172.21.15.160:0/4088573367 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80f8103380 0x7f80f8198840 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:55.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:55.012+0000 7f80fe2f2700 1 -- 172.21.15.160:0/4088573367 >> 172.21.15.160:0/4088573367 conn(0x7f80f80fd760 msgr2=0x7f80f8106680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:55.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:55.012+0000 7f80fe2f2700 1 -- 172.21.15.160:0/4088573367 shutdown_connections 2024-03-07T07:40:55.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:55.012+0000 7f80fe2f2700 1 -- 172.21.15.160:0/4088573367 wait complete. 2024-03-07T07:40:55.641 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:55 smithi160 ceph-mon[31963]: pgmap v340: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-07T07:40:55.669 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-07T07:40:55.678 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi160.front.sepia.ceph.com 2024-03-07T07:40:55.678 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-07T07:40:55.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:55 smithi178 ceph-mon[36650]: pgmap v340: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-07T07:40:55.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:55 smithi177 ceph-mon[36578]: pgmap v340: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-07T07:40:55.920 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:40:56.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:56 smithi160 ceph-mon[31963]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:40:56.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:56 smithi178 ceph-mon[36650]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:40:56.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:56 smithi177 ceph-mon[36578]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:40:57.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.586+0000 7f49efca2700 1 -- 172.21.15.160:0/82809481 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49e81010c0 msgr2=0x7f49e81014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:57.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.586+0000 7f49efca2700 1 --2- 172.21.15.160:0/82809481 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49e81010c0 0x7f49e81014e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f49d80099c0 tx=0x7f49d8009cd0 comp rx=0 tx=0).stop 2024-03-07T07:40:57.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.586+0000 7f49efca2700 1 -- 172.21.15.160:0/82809481 shutdown_connections 2024-03-07T07:40:57.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.586+0000 7f49efca2700 1 --2- 172.21.15.160:0/82809481 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49e8102c60 0x7f49e8109510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:57.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.586+0000 7f49efca2700 1 --2- 172.21.15.160:0/82809481 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49e81010c0 0x7f49e81014e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:57.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.586+0000 7f49efca2700 1 --2- 172.21.15.160:0/82809481 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49e81022c0 0x7f49e8102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:57.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.586+0000 7f49efca2700 1 -- 172.21.15.160:0/82809481 >> 172.21.15.160:0/82809481 conn(0x7f49e80fc640 msgr2=0x7f49e80feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:57.603 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.586+0000 7f49efca2700 1 -- 172.21.15.160:0/82809481 shutdown_connections 2024-03-07T07:40:57.603 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.586+0000 7f49efca2700 1 -- 172.21.15.160:0/82809481 wait complete. 2024-03-07T07:40:57.603 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.587+0000 7f49efca2700 1 Processor -- start 2024-03-07T07:40:57.603 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.587+0000 7f49efca2700 1 -- start start 2024-03-07T07:40:57.603 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.587+0000 7f49efca2700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49e81010c0 0x7f49e806b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:57.603 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.587+0000 7f49efca2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49e81022c0 0x7f49e819d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:57.603 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.587+0000 7f49efca2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49e8102c60 0x7f49e819f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:57.603 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.587+0000 7f49efca2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e810ccb0 con 0x7f49e81022c0 2024-03-07T07:40:57.603 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.587+0000 7f49efca2700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e810cb30 con 0x7f49e81010c0 2024-03-07T07:40:57.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.587+0000 7f49efca2700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e810ce30 con 0x7f49e8102c60 2024-03-07T07:40:57.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.587+0000 7f49ed23d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49e81022c0 0x7f49e819d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:57.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.587+0000 7f49ed23d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49e81022c0 0x7f49e819d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:51884/0 (socket says 172.21.15.160:51884) 2024-03-07T07:40:57.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.587+0000 7f49ed23d700 1 -- 172.21.15.160:0/1752782945 learned_addr learned my addr 172.21.15.160:0/1752782945 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:40:57.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.587+0000 7f49eda3e700 1 --2- 172.21.15.160:0/1752782945 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49e81010c0 0x7f49e806b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:57.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.587+0000 7f49ee23f700 1 --2- 172.21.15.160:0/1752782945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49e8102c60 0x7f49e819f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:57.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.588+0000 7f49ed23d700 1 -- 172.21.15.160:0/1752782945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49e8102c60 msgr2=0x7f49e819f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:57.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.588+0000 7f49ed23d700 1 --2- 172.21.15.160:0/1752782945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49e8102c60 0x7f49e819f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:57.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.588+0000 7f49ed23d700 1 -- 172.21.15.160:0/1752782945 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49e81010c0 msgr2=0x7f49e806b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:57.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.588+0000 7f49ed23d700 1 --2- 172.21.15.160:0/1752782945 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49e81010c0 0x7f49e806b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:57.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.588+0000 7f49ed23d700 1 -- 172.21.15.160:0/1752782945 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e81a0020 con 0x7f49e81022c0 2024-03-07T07:40:57.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.588+0000 7f49eda3e700 1 --2- 172.21.15.160:0/1752782945 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49e81010c0 0x7f49e806b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:40:57.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.588+0000 7f49ed23d700 1 --2- 172.21.15.160:0/1752782945 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49e81022c0 0x7f49e819d100 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f49e400e9c0 tx=0x7f49e400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:57.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.588+0000 7f49deffd700 1 -- 172.21.15.160:0/1752782945 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e400cbd0 con 0x7f49e81022c0 2024-03-07T07:40:57.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.588+0000 7f49efca2700 1 -- 172.21.15.160:0/1752782945 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49e81a0310 con 0x7f49e81022c0 2024-03-07T07:40:57.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.588+0000 7f49deffd700 1 -- 172.21.15.160:0/1752782945 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f49e400ee60 con 0x7f49e81022c0 2024-03-07T07:40:57.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.588+0000 7f49deffd700 1 -- 172.21.15.160:0/1752782945 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e4005630 con 0x7f49e81022c0 2024-03-07T07:40:57.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.588+0000 7f49efca2700 1 -- 172.21.15.160:0/1752782945 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49e81a0850 con 0x7f49e81022c0 2024-03-07T07:40:57.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.589+0000 7f49efca2700 1 -- 172.21.15.160:0/1752782945 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49e8068e10 con 0x7f49e81022c0 2024-03-07T07:40:57.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.590+0000 7f49deffd700 1 -- 172.21.15.160:0/1752782945 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f49e4004750 con 0x7f49e81022c0 2024-03-07T07:40:57.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.591+0000 7f49deffd700 1 --2- 172.21.15.160:0/1752782945 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f49d40a1e10 0x7f49d40a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:40:57.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.591+0000 7f49eda3e700 1 --2- 172.21.15.160:0/1752782945 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f49d40a1e10 0x7f49d40a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:40:57.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.591+0000 7f49deffd700 1 -- 172.21.15.160:0/1752782945 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f49e40ce220 con 0x7f49e81022c0 2024-03-07T07:40:57.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.591+0000 7f49eda3e700 1 --2- 172.21.15.160:0/1752782945 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f49d40a1e10 0x7f49d40a42d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f49d800b580 tx=0x7f49d8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:40:57.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.595+0000 7f49deffd700 1 -- 172.21.15.160:0/1752782945 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f49e4053280 con 0x7f49e81022c0 2024-03-07T07:40:57.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:57 smithi160 ceph-mon[31963]: pgmap v341: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-07T07:40:57.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:57 smithi178 ceph-mon[36650]: pgmap v341: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-07T07:40:57.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.842+0000 7f49efca2700 1 -- 172.21.15.160:0/1752782945 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f49e802d9f0 con 0x7f49e81022c0 2024-03-07T07:40:57.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.842+0000 7f49deffd700 1 -- 172.21.15.160:0/1752782945 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+3004 (secure 0 0 0) 0x7f49e409b650 con 0x7f49e81022c0 2024-03-07T07:40:57.843 INFO:teuthology.orchestra.run.smithi160.stdout:e21 2024-03-07T07:40:57.843 INFO:teuthology.orchestra.run.smithi160.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-07T07:40:57.843 INFO:teuthology.orchestra.run.smithi160.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-07T07:40:57.843 INFO:teuthology.orchestra.run.smithi160.stdout:legacy client fscid: 1 2024-03-07T07:40:57.843 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:40:57.843 INFO:teuthology.orchestra.run.smithi160.stdout:Filesystem 'cephfs' (1) 2024-03-07T07:40:57.843 INFO:teuthology.orchestra.run.smithi160.stdout:fs_name cephfs 2024-03-07T07:40:57.843 INFO:teuthology.orchestra.run.smithi160.stdout:epoch 21 2024-03-07T07:40:57.843 INFO:teuthology.orchestra.run.smithi160.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-07T07:40:57.844 INFO:teuthology.orchestra.run.smithi160.stdout:created 2024-03-07T07:39:14.164243+0000 2024-03-07T07:40:57.844 INFO:teuthology.orchestra.run.smithi160.stdout:modified 2024-03-07T07:39:18.861903+0000 2024-03-07T07:40:57.844 INFO:teuthology.orchestra.run.smithi160.stdout:tableserver 0 2024-03-07T07:40:57.844 INFO:teuthology.orchestra.run.smithi160.stdout:root 0 2024-03-07T07:40:57.844 INFO:teuthology.orchestra.run.smithi160.stdout:session_timeout 300 2024-03-07T07:40:57.844 INFO:teuthology.orchestra.run.smithi160.stdout:session_autoclose 300 2024-03-07T07:40:57.844 INFO:teuthology.orchestra.run.smithi160.stdout:max_file_size 1099511627776 2024-03-07T07:40:57.844 INFO:teuthology.orchestra.run.smithi160.stdout:max_xattr_size 65536 2024-03-07T07:40:57.844 INFO:teuthology.orchestra.run.smithi160.stdout:required_client_features {} 2024-03-07T07:40:57.844 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure 0 2024-03-07T07:40:57.844 INFO:teuthology.orchestra.run.smithi160.stdout:last_failure_osd_epoch 0 2024-03-07T07:40:57.844 INFO:teuthology.orchestra.run.smithi160.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-07T07:40:57.844 INFO:teuthology.orchestra.run.smithi160.stdout:max_mds 3 2024-03-07T07:40:57.844 INFO:teuthology.orchestra.run.smithi160.stdout:in 0,1,2 2024-03-07T07:40:57.845 INFO:teuthology.orchestra.run.smithi160.stdout:up {0=24511,1=24497,2=24487} 2024-03-07T07:40:57.845 INFO:teuthology.orchestra.run.smithi160.stdout:failed 2024-03-07T07:40:57.845 INFO:teuthology.orchestra.run.smithi160.stdout:damaged 2024-03-07T07:40:57.845 INFO:teuthology.orchestra.run.smithi160.stdout:stopped 2024-03-07T07:40:57.845 INFO:teuthology.orchestra.run.smithi160.stdout:data_pools [3] 2024-03-07T07:40:57.845 INFO:teuthology.orchestra.run.smithi160.stdout:metadata_pool 2 2024-03-07T07:40:57.845 INFO:teuthology.orchestra.run.smithi160.stdout:inline_data disabled 2024-03-07T07:40:57.845 INFO:teuthology.orchestra.run.smithi160.stdout:balancer 2024-03-07T07:40:57.845 INFO:teuthology.orchestra.run.smithi160.stdout:bal_rank_mask -1 2024-03-07T07:40:57.845 INFO:teuthology.orchestra.run.smithi160.stdout:standby_count_wanted 0 2024-03-07T07:40:57.845 INFO:teuthology.orchestra.run.smithi160.stdout:qdb_cluster leader: 24511 members: 24487,24511,24497 2024-03-07T07:40:57.845 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.i{0:24511} state up:active seq 5 addr [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] compat {c=[1],r=[1],i=[fff]}] 2024-03-07T07:40:57.845 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.f{0:24475} state up:standby-replay seq 1 addr [v2:172.21.15.178:6834/647840547,v1:172.21.15.178:6835/647840547] compat {c=[1],r=[1],i=[fff]}] 2024-03-07T07:40:57.845 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.b{1:24497} state up:active seq 4 addr [v2:172.21.15.177:6838/3329773838,v1:172.21.15.177:6839/3329773838] compat {c=[1],r=[1],i=[fff]}] 2024-03-07T07:40:57.845 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.h{1:24458} state up:standby-replay seq 1 addr [v2:172.21.15.177:6836/4088036516,v1:172.21.15.177:6837/4088036516] compat {c=[1],r=[1],i=[fff]}] 2024-03-07T07:40:57.846 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.c{2:24487} state up:active seq 8 addr [v2:172.21.15.178:6836/1247829154,v1:172.21.15.178:6837/1247829154] compat {c=[1],r=[1],i=[fff]}] 2024-03-07T07:40:57.846 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.l{2:24436} state up:standby-replay seq 1 addr [v2:172.21.15.178:6832/645365846,v1:172.21.15.178:6833/645365846] compat {c=[1],r=[1],i=[fff]}] 2024-03-07T07:40:57.846 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:40:57.846 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:40:57.846 INFO:teuthology.orchestra.run.smithi160.stdout:Standby daemons: 2024-03-07T07:40:57.846 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:40:57.846 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.j{-1:14538} state up:standby seq 1 addr [v2:172.21.15.160:6834/2615961083,v1:172.21.15.160:6835/2615961083] compat {c=[1],r=[1],i=[fff]}] 2024-03-07T07:40:57.846 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.d{-1:14550} state up:standby seq 1 addr [v2:172.21.15.160:6836/1759006033,v1:172.21.15.160:6837/1759006033] compat {c=[1],r=[1],i=[fff]}] 2024-03-07T07:40:57.846 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.g{-1:14613} state up:standby seq 1 addr [v2:172.21.15.160:6838/3763944721,v1:172.21.15.160:6839/3763944721] compat {c=[1],r=[1],i=[fff]}] 2024-03-07T07:40:57.846 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.a{-1:14625} state up:standby seq 1 addr [v2:172.21.15.160:6840/3611426637,v1:172.21.15.160:6841/3611426637] compat {c=[1],r=[1],i=[fff]}] 2024-03-07T07:40:57.846 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.k{-1:24407} state up:standby seq 1 addr [v2:172.21.15.177:6832/1457364761,v1:172.21.15.177:6833/1457364761] compat {c=[1],r=[1],i=[fff]}] 2024-03-07T07:40:57.846 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.e{-1:24431} state up:standby seq 1 addr [v2:172.21.15.177:6834/170428292,v1:172.21.15.177:6835/170428292] compat {c=[1],r=[1],i=[fff]}] 2024-03-07T07:40:57.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.844+0000 7f49efca2700 1 -- 172.21.15.160:0/1752782945 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f49d40a1e10 msgr2=0x7f49d40a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:57.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.844+0000 7f49efca2700 1 --2- 172.21.15.160:0/1752782945 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f49d40a1e10 0x7f49d40a42d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f49d800b580 tx=0x7f49d8005fb0 comp rx=0 tx=0).stop 2024-03-07T07:40:57.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.844+0000 7f49efca2700 1 -- 172.21.15.160:0/1752782945 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49e81022c0 msgr2=0x7f49e819d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:40:57.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.844+0000 7f49efca2700 1 --2- 172.21.15.160:0/1752782945 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49e81022c0 0x7f49e819d100 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f49e400e9c0 tx=0x7f49e400ecd0 comp rx=0 tx=0).stop 2024-03-07T07:40:57.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.844+0000 7f49efca2700 1 -- 172.21.15.160:0/1752782945 shutdown_connections 2024-03-07T07:40:57.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.844+0000 7f49efca2700 1 --2- 172.21.15.160:0/1752782945 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f49d40a1e10 0x7f49d40a42d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:57.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.844+0000 7f49efca2700 1 --2- 172.21.15.160:0/1752782945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49e8102c60 0x7f49e819f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:57.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.844+0000 7f49efca2700 1 --2- 172.21.15.160:0/1752782945 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49e81010c0 0x7f49e806b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:57.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.844+0000 7f49efca2700 1 --2- 172.21.15.160:0/1752782945 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49e81022c0 0x7f49e819d100 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:40:57.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.844+0000 7f49efca2700 1 -- 172.21.15.160:0/1752782945 >> 172.21.15.160:0/1752782945 conn(0x7f49e80fc640 msgr2=0x7f49e81055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:40:57.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.845+0000 7f49efca2700 1 -- 172.21.15.160:0/1752782945 shutdown_connections 2024-03-07T07:40:57.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:40:57.845+0000 7f49efca2700 1 -- 172.21.15.160:0/1752782945 wait complete. 2024-03-07T07:40:57.848 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 21 2024-03-07T07:40:57.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:57 smithi177 ceph-mon[36578]: pgmap v341: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-07T07:40:58.427 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:58 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/1752782945' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-07T07:40:58.637 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- bash -c 'ceph osd dump' 2024-03-07T07:40:58.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:58 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/1752782945' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-07T07:40:58.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:58 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/1752782945' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-07T07:40:58.879 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:40:59.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:40:59 smithi160 ceph-mon[31963]: pgmap v342: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-07T07:40:59.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:40:59 smithi178 ceph-mon[36650]: pgmap v342: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-07T07:40:59.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:40:59 smithi177 ceph-mon[36578]: pgmap v342: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-07T07:41:00.566 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.563+0000 7fa88ebc5700 1 -- 172.21.15.160:0/2300358016 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8881022f0 msgr2=0x7fa888102750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:00.566 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.563+0000 7fa88ebc5700 1 --2- 172.21.15.160:0/2300358016 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8881022f0 0x7fa888102750 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa8740099c0 tx=0x7fa874009cd0 comp rx=0 tx=0).stop 2024-03-07T07:41:00.566 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.563+0000 7fa88ebc5700 1 -- 172.21.15.160:0/2300358016 shutdown_connections 2024-03-07T07:41:00.566 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.563+0000 7fa88ebc5700 1 --2- 172.21.15.160:0/2300358016 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa888102c90 0x7fa888109540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:00.566 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.563+0000 7fa88ebc5700 1 --2- 172.21.15.160:0/2300358016 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8881010f0 0x7fa888101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:00.566 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.563+0000 7fa88ebc5700 1 --2- 172.21.15.160:0/2300358016 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8881022f0 0x7fa888102750 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:00.566 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.563+0000 7fa88ebc5700 1 -- 172.21.15.160:0/2300358016 >> 172.21.15.160:0/2300358016 conn(0x7fa8880fc6b0 msgr2=0x7fa8880fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:00.566 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.563+0000 7fa88ebc5700 1 -- 172.21.15.160:0/2300358016 shutdown_connections 2024-03-07T07:41:00.566 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.563+0000 7fa88ebc5700 1 -- 172.21.15.160:0/2300358016 wait complete. 2024-03-07T07:41:00.566 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.564+0000 7fa88ebc5700 1 Processor -- start 2024-03-07T07:41:00.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.564+0000 7fa88ebc5700 1 -- start start 2024-03-07T07:41:00.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.564+0000 7fa88ebc5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8881010f0 0x7fa88806b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:00.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.564+0000 7fa88ebc5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8881022f0 0x7fa88819d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:00.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.564+0000 7fa88ebc5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa888102c90 0x7fa88819f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:00.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.564+0000 7fa88ebc5700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88810cc70 con 0x7fa8881010f0 2024-03-07T07:41:00.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.564+0000 7fa88ebc5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88810caf0 con 0x7fa888102c90 2024-03-07T07:41:00.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.564+0000 7fa88ebc5700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88810cdf0 con 0x7fa8881022f0 2024-03-07T07:41:00.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.564+0000 7fa88c961700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8881010f0 0x7fa88806b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:00.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.564+0000 7fa88c961700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8881010f0 0x7fa88806b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57674/0 (socket says 172.21.15.160:57674) 2024-03-07T07:41:00.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.564+0000 7fa88c961700 1 -- 172.21.15.160:0/1238650176 learned_addr learned my addr 172.21.15.160:0/1238650176 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:41:00.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.564+0000 7fa887fff700 1 --2- 172.21.15.160:0/1238650176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8881022f0 0x7fa88819d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:00.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.564+0000 7fa88d162700 1 --2- 172.21.15.160:0/1238650176 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa888102c90 0x7fa88819f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:00.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.565+0000 7fa88c961700 1 -- 172.21.15.160:0/1238650176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8881022f0 msgr2=0x7fa88819d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:00.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.565+0000 7fa88c961700 1 --2- 172.21.15.160:0/1238650176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8881022f0 0x7fa88819d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:00.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.565+0000 7fa88c961700 1 -- 172.21.15.160:0/1238650176 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa888102c90 msgr2=0x7fa88819f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:00.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.565+0000 7fa88c961700 1 --2- 172.21.15.160:0/1238650176 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa888102c90 0x7fa88819f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:00.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.565+0000 7fa88c961700 1 -- 172.21.15.160:0/1238650176 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8881a0000 con 0x7fa8881010f0 2024-03-07T07:41:00.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.565+0000 7fa88c961700 1 --2- 172.21.15.160:0/1238650176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8881010f0 0x7fa88806b3b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa87c00e950 tx=0x7fa87c00ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:00.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.565+0000 7fa885ffb700 1 -- 172.21.15.160:0/1238650176 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87c004230 con 0x7fa8881010f0 2024-03-07T07:41:00.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.565+0000 7fa885ffb700 1 -- 172.21.15.160:0/1238650176 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa87c0043d0 con 0x7fa8881010f0 2024-03-07T07:41:00.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.565+0000 7fa88ebc5700 1 -- 172.21.15.160:0/1238650176 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8881a02f0 con 0x7fa8881010f0 2024-03-07T07:41:00.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.565+0000 7fa885ffb700 1 -- 172.21.15.160:0/1238650176 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87c003dd0 con 0x7fa8881010f0 2024-03-07T07:41:00.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.565+0000 7fa88ebc5700 1 -- 172.21.15.160:0/1238650176 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8881a0800 con 0x7fa8881010f0 2024-03-07T07:41:00.569 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.566+0000 7fa88ebc5700 1 -- 172.21.15.160:0/1238650176 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa888068e10 con 0x7fa8881010f0 2024-03-07T07:41:00.569 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.567+0000 7fa885ffb700 1 -- 172.21.15.160:0/1238650176 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 136754+0+0 (secure 0 0 0) 0x7fa87c00c7e0 con 0x7fa8881010f0 2024-03-07T07:41:00.569 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.568+0000 7fa885ffb700 1 --2- 172.21.15.160:0/1238650176 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa8780a1e90 0x7fa8780a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:00.569 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.568+0000 7fa887fff700 1 --2- 172.21.15.160:0/1238650176 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa8780a1e90 0x7fa8780a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:00.570 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.568+0000 7fa885ffb700 1 -- 172.21.15.160:0/1238650176 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa87c0cdfb0 con 0x7fa8881010f0 2024-03-07T07:41:00.570 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.568+0000 7fa887fff700 1 --2- 172.21.15.160:0/1238650176 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa8780a1e90 0x7fa8780a4350 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa874000c00 tx=0x7fa874005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:00.572 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.572+0000 7fa885ffb700 1 -- 172.21.15.160:0/1238650176 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa87c09b3e0 con 0x7fa8881010f0 2024-03-07T07:41:00.767 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.766+0000 7fa88ebc5700 1 -- 172.21.15.160:0/1238650176 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7fa888062d60 con 0x7fa8881010f0 2024-03-07T07:41:00.767 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.767+0000 7fa885ffb700 1 -- 172.21.15.160:0/1238650176 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v87) v1 ==== 56+0+5323 (secure 0 0 0) 0x7fa87c09ace0 con 0x7fa8881010f0 2024-03-07T07:41:00.767 INFO:teuthology.orchestra.run.smithi160.stdout:epoch 87 2024-03-07T07:41:00.768 INFO:teuthology.orchestra.run.smithi160.stdout:fsid 826f6582-dc54-11ee-95c6-87774f69a715 2024-03-07T07:41:00.768 INFO:teuthology.orchestra.run.smithi160.stdout:created 2024-03-07T07:31:14.229288+0000 2024-03-07T07:41:00.768 INFO:teuthology.orchestra.run.smithi160.stdout:modified 2024-03-07T07:40:09.354265+0000 2024-03-07T07:41:00.768 INFO:teuthology.orchestra.run.smithi160.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-03-07T07:41:00.768 INFO:teuthology.orchestra.run.smithi160.stdout:crush_version 26 2024-03-07T07:41:00.768 INFO:teuthology.orchestra.run.smithi160.stdout:full_ratio 0.95 2024-03-07T07:41:00.768 INFO:teuthology.orchestra.run.smithi160.stdout:backfillfull_ratio 0.9 2024-03-07T07:41:00.768 INFO:teuthology.orchestra.run.smithi160.stdout:nearfull_ratio 0.85 2024-03-07T07:41:00.768 INFO:teuthology.orchestra.run.smithi160.stdout:require_min_compat_client luminous 2024-03-07T07:41:00.768 INFO:teuthology.orchestra.run.smithi160.stdout:min_compat_client luminous 2024-03-07T07:41:00.768 INFO:teuthology.orchestra.run.smithi160.stdout:require_osd_release squid 2024-03-07T07:41:00.768 INFO:teuthology.orchestra.run.smithi160.stdout:stretch_mode_enabled false 2024-03-07T07:41:00.769 INFO:teuthology.orchestra.run.smithi160.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-03-07T07:41:00.769 INFO:teuthology.orchestra.run.smithi160.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-03-07T07:41:00.769 INFO:teuthology.orchestra.run.smithi160.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-03-07T07:41:00.769 INFO:teuthology.orchestra.run.smithi160.stdout:max_osd 12 2024-03-07T07:41:00.769 INFO:teuthology.orchestra.run.smithi160.stdout:osd.0 up in weight 1 up_from 8 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.160:6802/1368482926,v1:172.21.15.160:6803/1368482926] [v2:172.21.15.160:6804/1368482926,v1:172.21.15.160:6805/1368482926] exists,up bee66c67-fb6e-4717-9685-ca62587d5f90 2024-03-07T07:41:00.769 INFO:teuthology.orchestra.run.smithi160.stdout:osd.1 up in weight 1 up_from 13 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.177:6800/2922391884,v1:172.21.15.177:6801/2922391884] [v2:172.21.15.177:6802/2922391884,v1:172.21.15.177:6803/2922391884] exists,up 49236251-f750-424e-992b-bddf55762556 2024-03-07T07:41:00.769 INFO:teuthology.orchestra.run.smithi160.stdout:osd.2 up in weight 1 up_from 18 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6800/1217901551,v1:172.21.15.178:6801/1217901551] [v2:172.21.15.178:6802/1217901551,v1:172.21.15.178:6803/1217901551] exists,up 7a432b28-f7ee-43fe-9196-7aaeccfecf77 2024-03-07T07:41:00.769 INFO:teuthology.orchestra.run.smithi160.stdout:osd.3 up in weight 1 up_from 25 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.160:6810/42543989,v1:172.21.15.160:6811/42543989] [v2:172.21.15.160:6812/42543989,v1:172.21.15.160:6813/42543989] exists,up 4ad22a20-f1be-4002-a48d-ecd3953e63f3 2024-03-07T07:41:00.769 INFO:teuthology.orchestra.run.smithi160.stdout:osd.4 up in weight 1 up_from 33 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.177:6808/88178553,v1:172.21.15.177:6809/88178553] [v2:172.21.15.177:6810/88178553,v1:172.21.15.177:6811/88178553] exists,up 2b65b104-ac82-4ce3-b6c5-11a399103045 2024-03-07T07:41:00.769 INFO:teuthology.orchestra.run.smithi160.stdout:osd.5 up in weight 1 up_from 38 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6808/964732891,v1:172.21.15.178:6809/964732891] [v2:172.21.15.178:6810/964732891,v1:172.21.15.178:6811/964732891] exists,up 30a1e714-08ff-44e8-9264-9ef164604b60 2024-03-07T07:41:00.769 INFO:teuthology.orchestra.run.smithi160.stdout:osd.6 up in weight 1 up_from 46 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.160:6818/3251327001,v1:172.21.15.160:6819/3251327001] [v2:172.21.15.160:6820/3251327001,v1:172.21.15.160:6821/3251327001] exists,up b1e1646e-f8a9-42b8-b62d-9d7b5d6a1ff4 2024-03-07T07:41:00.769 INFO:teuthology.orchestra.run.smithi160.stdout:osd.7 up in weight 1 up_from 51 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.177:6816/1175500850,v1:172.21.15.177:6817/1175500850] [v2:172.21.15.177:6818/1175500850,v1:172.21.15.177:6819/1175500850] exists,up e66b400b-9747-4f6d-b05b-db22468e2750 2024-03-07T07:41:00.770 INFO:teuthology.orchestra.run.smithi160.stdout:osd.8 up in weight 1 up_from 59 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6816/688015531,v1:172.21.15.178:6817/688015531] [v2:172.21.15.178:6818/688015531,v1:172.21.15.178:6819/688015531] exists,up 3ee0f2c4-fca4-49fc-a98b-07aa0fcba0e8 2024-03-07T07:41:00.770 INFO:teuthology.orchestra.run.smithi160.stdout:osd.9 up in weight 1 up_from 67 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.160:6826/3265797851,v1:172.21.15.160:6827/3265797851] [v2:172.21.15.160:6828/3265797851,v1:172.21.15.160:6829/3265797851] exists,up 856039d7-f7ee-4dbd-b019-0073e705ce0f 2024-03-07T07:41:00.770 INFO:teuthology.orchestra.run.smithi160.stdout:osd.10 up in weight 1 up_from 72 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.177:6824/2867473412,v1:172.21.15.177:6825/2867473412] [v2:172.21.15.177:6826/2867473412,v1:172.21.15.177:6827/2867473412] exists,up bdc9899d-d720-4429-ad53-4c388a4dbe75 2024-03-07T07:41:00.770 INFO:teuthology.orchestra.run.smithi160.stdout:osd.11 up in weight 1 up_from 80 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6824/4068724731,v1:172.21.15.178:6825/4068724731] [v2:172.21.15.178:6826/4068724731,v1:172.21.15.178:6827/4068724731] exists,up cc9adb54-ef49-47c6-90e3-fc03bc0f5cda 2024-03-07T07:41:00.770 INFO:teuthology.orchestra.run.smithi160.stdout:pg_upmap_items 2.e [2,1] 2024-03-07T07:41:00.770 INFO:teuthology.orchestra.run.smithi160.stdout:pg_upmap_items 2.10 [8,11] 2024-03-07T07:41:00.770 INFO:teuthology.orchestra.run.smithi160.stdout:pg_upmap_items 2.14 [2,4] 2024-03-07T07:41:00.770 INFO:teuthology.orchestra.run.smithi160.stdout:pg_upmap_items 2.1d [8,11] 2024-03-07T07:41:00.770 INFO:teuthology.orchestra.run.smithi160.stdout:pg_upmap_items 3.5 [3,9] 2024-03-07T07:41:00.770 INFO:teuthology.orchestra.run.smithi160.stdout:pg_upmap_items 3.6 [4,7] 2024-03-07T07:41:00.770 INFO:teuthology.orchestra.run.smithi160.stdout:pg_upmap_items 3.e [5,9] 2024-03-07T07:41:00.770 INFO:teuthology.orchestra.run.smithi160.stdout:pg_upmap_items 3.36 [4,9] 2024-03-07T07:41:00.770 INFO:teuthology.orchestra.run.smithi160.stdout:pg_upmap_items 3.3a [4,8] 2024-03-07T07:41:00.770 INFO:teuthology.orchestra.run.smithi160.stdout:blocklist 172.21.15.160:0/3027450612 expires 2024-03-08T07:32:08.190837+0000 2024-03-07T07:41:00.770 INFO:teuthology.orchestra.run.smithi160.stdout:blocklist 172.21.15.160:0/874709722 expires 2024-03-08T07:32:08.190837+0000 2024-03-07T07:41:00.771 INFO:teuthology.orchestra.run.smithi160.stdout:blocklist 172.21.15.160:6800/1130675225 expires 2024-03-08T07:31:41.166858+0000 2024-03-07T07:41:00.771 INFO:teuthology.orchestra.run.smithi160.stdout:blocklist 172.21.15.160:0/3857193980 expires 2024-03-08T07:32:08.190837+0000 2024-03-07T07:41:00.771 INFO:teuthology.orchestra.run.smithi160.stdout:blocklist 172.21.15.160:0/3861968297 expires 2024-03-08T07:31:41.166858+0000 2024-03-07T07:41:00.771 INFO:teuthology.orchestra.run.smithi160.stdout:blocklist 172.21.15.160:6800/3859174315 expires 2024-03-08T07:32:08.190837+0000 2024-03-07T07:41:00.771 INFO:teuthology.orchestra.run.smithi160.stdout:blocklist 172.21.15.160:0/3013534743 expires 2024-03-08T07:31:41.166858+0000 2024-03-07T07:41:00.771 INFO:teuthology.orchestra.run.smithi160.stdout:blocklist 172.21.15.160:6801/3859174315 expires 2024-03-08T07:32:08.190837+0000 2024-03-07T07:41:00.771 INFO:teuthology.orchestra.run.smithi160.stdout:blocklist 172.21.15.160:0/1310937667 expires 2024-03-08T07:31:41.166858+0000 2024-03-07T07:41:00.771 INFO:teuthology.orchestra.run.smithi160.stdout:blocklist 172.21.15.160:6801/1130675225 expires 2024-03-08T07:31:41.166858+0000 2024-03-07T07:41:00.772 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.769+0000 7fa88ebc5700 1 -- 172.21.15.160:0/1238650176 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa8780a1e90 msgr2=0x7fa8780a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:00.772 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.769+0000 7fa88ebc5700 1 --2- 172.21.15.160:0/1238650176 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa8780a1e90 0x7fa8780a4350 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa874000c00 tx=0x7fa874005fb0 comp rx=0 tx=0).stop 2024-03-07T07:41:00.772 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.769+0000 7fa88ebc5700 1 -- 172.21.15.160:0/1238650176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8881010f0 msgr2=0x7fa88806b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:00.772 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.769+0000 7fa88ebc5700 1 --2- 172.21.15.160:0/1238650176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8881010f0 0x7fa88806b3b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa87c00e950 tx=0x7fa87c00ec60 comp rx=0 tx=0).stop 2024-03-07T07:41:00.772 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.769+0000 7fa88ebc5700 1 -- 172.21.15.160:0/1238650176 shutdown_connections 2024-03-07T07:41:00.772 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.769+0000 7fa88ebc5700 1 --2- 172.21.15.160:0/1238650176 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa888102c90 0x7fa88819f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:00.772 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.769+0000 7fa88ebc5700 1 --2- 172.21.15.160:0/1238650176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8881010f0 0x7fa88806b3b0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:00.772 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.769+0000 7fa88ebc5700 1 --2- 172.21.15.160:0/1238650176 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa8780a1e90 0x7fa8780a4350 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:00.773 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.769+0000 7fa88ebc5700 1 --2- 172.21.15.160:0/1238650176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8881022f0 0x7fa88819d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:00.773 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.769+0000 7fa88ebc5700 1 -- 172.21.15.160:0/1238650176 >> 172.21.15.160:0/1238650176 conn(0x7fa8880fc6b0 msgr2=0x7fa8881055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:00.773 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.770+0000 7fa88ebc5700 1 -- 172.21.15.160:0/1238650176 shutdown_connections 2024-03-07T07:41:00.773 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:00.770+0000 7fa88ebc5700 1 -- 172.21.15.160:0/1238650176 wait complete. 2024-03-07T07:41:01.427 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:01 smithi160 ceph-mon[31963]: pgmap v343: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-07T07:41:01.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:01 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/1238650176' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-07T07:41:01.536 INFO:teuthology.run_tasks:Running task fs.ready... 2024-03-07T07:41:01.547 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T07:41:01.655 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.653+0000 7fc4463ee640 1 -- 172.21.15.160:0/2026016347 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc440100580 msgr2=0x7fc4401009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:01.656 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.653+0000 7fc4463ee640 1 --2- 172.21.15.160:0/2026016347 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc440100580 0x7fc4401009a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc434009900 tx=0x7fc43402bc30 comp rx=0 tx=0).stop 2024-03-07T07:41:01.656 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.653+0000 7fc4463ee640 1 -- 172.21.15.160:0/2026016347 shutdown_connections 2024-03-07T07:41:01.656 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.653+0000 7fc4463ee640 1 --2- 172.21.15.160:0/2026016347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc440102130 0x7fc4401089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:01.656 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.653+0000 7fc4463ee640 1 --2- 172.21.15.160:0/2026016347 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc440101770 0x7fc440101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:01.656 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.653+0000 7fc4463ee640 1 --2- 172.21.15.160:0/2026016347 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc440100580 0x7fc4401009a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:01.656 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.653+0000 7fc4463ee640 1 -- 172.21.15.160:0/2026016347 >> 172.21.15.160:0/2026016347 conn(0x7fc4400fbaf0 msgr2=0x7fc4400fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:01.656 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.653+0000 7fc4463ee640 1 -- 172.21.15.160:0/2026016347 shutdown_connections 2024-03-07T07:41:01.656 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.653+0000 7fc4463ee640 1 -- 172.21.15.160:0/2026016347 wait complete. 2024-03-07T07:41:01.656 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.654+0000 7fc4463ee640 1 Processor -- start 2024-03-07T07:41:01.656 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.654+0000 7fc4463ee640 1 -- start start 2024-03-07T07:41:01.656 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.654+0000 7fc4463ee640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc440100580 0x7fc4401968a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:01.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.654+0000 7fc4463ee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc440101770 0x7fc440196de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:01.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.654+0000 7fc4463ee640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc440102130 0x7fc44019de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:01.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.654+0000 7fc4463ee640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44010c210 con 0x7fc440100580 2024-03-07T07:41:01.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.654+0000 7fc4463ee640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44010c090 con 0x7fc440102130 2024-03-07T07:41:01.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.654+0000 7fc4463ee640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44010c390 con 0x7fc440101770 2024-03-07T07:41:01.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.654+0000 7fc43ffff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc440100580 0x7fc4401968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:01.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.654+0000 7fc43ffff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc440100580 0x7fc4401968a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57694/0 (socket says 172.21.15.160:57694) 2024-03-07T07:41:01.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.654+0000 7fc43ffff640 1 -- 172.21.15.160:0/2343518890 learned_addr learned my addr 172.21.15.160:0/2343518890 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:41:01.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.654+0000 7fc43f7fe640 1 --2- 172.21.15.160:0/2343518890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc440101770 0x7fc440196de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:01.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.654+0000 7fc44498b640 1 --2- 172.21.15.160:0/2343518890 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc440102130 0x7fc44019de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:01.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.654+0000 7fc43ffff640 1 -- 172.21.15.160:0/2343518890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc440101770 msgr2=0x7fc440196de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:01.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.654+0000 7fc43ffff640 1 --2- 172.21.15.160:0/2343518890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc440101770 0x7fc440196de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:01.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.654+0000 7fc43ffff640 1 -- 172.21.15.160:0/2343518890 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc440102130 msgr2=0x7fc44019de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:01.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.654+0000 7fc43ffff640 1 --2- 172.21.15.160:0/2343518890 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc440102130 0x7fc44019de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:01.658 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.654+0000 7fc43ffff640 1 -- 172.21.15.160:0/2343518890 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc44019e4c0 con 0x7fc440100580 2024-03-07T07:41:01.658 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.654+0000 7fc43ffff640 1 --2- 172.21.15.160:0/2343518890 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc440100580 0x7fc4401968a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc434000c00 tx=0x7fc43402c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:01.658 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.655+0000 7fc43d7fa640 1 -- 172.21.15.160:0/2343518890 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc434041070 con 0x7fc440100580 2024-03-07T07:41:01.658 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.655+0000 7fc4463ee640 1 -- 172.21.15.160:0/2343518890 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc44019e750 con 0x7fc440100580 2024-03-07T07:41:01.658 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.655+0000 7fc43d7fa640 1 -- 172.21.15.160:0/2343518890 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc434004450 con 0x7fc440100580 2024-03-07T07:41:01.658 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.655+0000 7fc43d7fa640 1 -- 172.21.15.160:0/2343518890 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc434034440 con 0x7fc440100580 2024-03-07T07:41:01.658 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.655+0000 7fc4463ee640 1 -- 172.21.15.160:0/2343518890 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc44019ec40 con 0x7fc440100580 2024-03-07T07:41:01.659 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.656+0000 7fc4463ee640 1 -- 172.21.15.160:0/2343518890 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4401078a0 con 0x7fc440100580 2024-03-07T07:41:01.659 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.657+0000 7fc43d7fa640 1 -- 172.21.15.160:0/2343518890 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 136754+0+0 (secure 0 0 0) 0x7fc434044070 con 0x7fc440100580 2024-03-07T07:41:01.659 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.657+0000 7fc43d7fa640 1 --2- 172.21.15.160:0/2343518890 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fc4280a1d80 0x7fc4280a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:01.659 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.657+0000 7fc43f7fe640 1 --2- 172.21.15.160:0/2343518890 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fc4280a1d80 0x7fc4280a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:01.659 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.657+0000 7fc43d7fa640 1 -- 172.21.15.160:0/2343518890 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc4340ef820 con 0x7fc440100580 2024-03-07T07:41:01.659 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.658+0000 7fc43f7fe640 1 --2- 172.21.15.160:0/2343518890 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fc4280a1d80 0x7fc4280a4240 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc440197db0 tx=0x7fc430005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:01.662 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.661+0000 7fc43d7fa640 1 -- 172.21.15.160:0/2343518890 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc4340bc690 con 0x7fc440100580 2024-03-07T07:41:01.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:01 smithi178 ceph-mon[36650]: pgmap v343: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-07T07:41:01.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:01 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/1238650176' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-07T07:41:01.836 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.835+0000 7fc4463ee640 1 -- 172.21.15.160:0/2343518890 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc440062260 con 0x7fc440100580 2024-03-07T07:41:01.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.837+0000 7fc43d7fa640 1 -- 172.21.15.160:0/2343518890 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22721 (secure 0 0 0) 0x7fc4340c0730 con 0x7fc440100580 2024-03-07T07:41:01.838 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:41:01.838 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":87,"fsid":"826f6582-dc54-11ee-95c6-87774f69a715","created":"2024-03-07T07:31:14.229288+0000","modified":"2024-03-07T07:40:09.354265+0000","last_up_change":"2024-03-07T07:38:20.776225+0000","last_in_change":"2024-03-07T07:38:00.508917+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T07:34:34.529424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T07:39:12.075432+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T07:39:13.367820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bee66c67-fb6e-4717-9685-ca62587d5f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6803","nonce":1368482926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6805","nonce":1368482926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6809","nonce":1368482926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6807","nonce":1368482926}]},"public_addr":"172.21.15.160:6803/1368482926","cluster_addr":"172.21.15.160:6805/1368482926","heartbeat_back_addr":"172.21.15.160:6809/1368482926","heartbeat_front_addr":"172.21.15.160:6807/1368482926","state":["exists","up"]},{"osd":1,"uuid":"49236251-f750-424e-992b-bddf55762556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6801","nonce":2922391884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6803","nonce":2922391884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6807","nonce":2922391884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6805","nonce":2922391884}]},"public_addr":"172.21.15.177:6801/2922391884","cluster_addr":"172.21.15.177:6803/2922391884","heartbeat_back_addr":"172.21.15.177:6807/2922391884","heartbeat_front_addr":"172.21.15.177:6805/2922391884","state":["exists","up"]},{"osd":2,"uuid":"7a432b28-f7ee-43fe-9196-7aaeccfecf77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6801","nonce":1217901551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6803","nonce":1217901551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6807","nonce":1217901551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6805","nonce":1217901551}]},"public_addr":"172.21.15.178:6801/1217901551","cluster_addr":"172.21.15.178:6803/1217901551","heartbeat_back_addr":"172.21.15.178:6807/1217901551","heartbeat_front_addr":"172.21.15.178:6805/1217901551","state":["exists","up"]},{"osd":3,"uuid":"4ad22a20-f1be-4002-a48d-ecd3953e63f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6811","nonce":42543989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6813","nonce":42543989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6817","nonce":42543989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6815","nonce":42543989}]},"public_addr":"172.21.15.160:6811/42543989","cluster_addr":"172.21.15.160:6813/42543989","heartbeat_back_addr":"172.21.15.160:6817/42543989","heartbeat_front_addr":"172.21.15.160:6815/42543989","state":["exists","up"]},{"osd":4,"uuid":"2b65b104-ac82-4ce3-b6c5-11a399103045","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6809","nonce":88178553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6811","nonce":88178553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6815","nonce":88178553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6813","nonce":88178553}]},"public_addr":"172.21.15.177:6809/88178553","cluster_addr":"172.21.15.177:6811/88178553","heartbeat_back_addr":"172.21.15.177:6815/88178553","heartbeat_front_addr":"172.21.15.177:6813/88178553","state":["exists","up"]},{"osd":5,"uuid":"30a1e714-08ff-44e8-9264-9ef164604b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6809","nonce":964732891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6811","nonce":964732891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6815","nonce":964732891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6813","nonce":964732891}]},"public_addr":"172.21.15.178:6809/964732891","cluster_addr":"172.21.15.178:6811/964732891","heartbeat_back_addr":"172.21.15.178:6815/964732891","heartbeat_front_addr":"172.21.15.178:6813/964732891","state":["exists","up"]},{"osd":6,"uuid":"b1e1646e-f8a9-42b8-b62d-9d7b5d6a1ff4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6819","nonce":3251327001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6821","nonce":3251327001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6825","nonce":3251327001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6823","nonce":3251327001}]},"public_addr":"172.21.15.160:6819/3251327001","cluster_addr":"172.21.15.160:6821/3251327001","heartbeat_back_addr":"172.21.15.160:6825/3251327001","heartbeat_front_addr":"172.21.15.160:6823/3251327001","state":["exists","up"]},{"osd":7,"uuid":"e66b400b-9747-4f6d-b05b-db22468e2750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6817","nonce":1175500850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6819","nonce":1175500850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6823","nonce":1175500850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6821","nonce":1175500850}]},"public_addr":"172.21.15.177:6817/1175500850","cluster_addr":"172.21.15.177:6819/1175500850","heartbeat_back_addr":"172.21.15.177:6823/1175500850","heartbeat_front_addr":"172.21.15.177:6821/1175500850","state":["exists","up"]},{"osd":8,"uuid":"3ee0f2c4-fca4-49fc-a98b-07aa0fcba0e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6817","nonce":688015531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6819","nonce":688015531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6823","nonce":688015531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6821","nonce":688015531}]},"public_addr":"172.21.15.178:6817/688015531","cluster_addr":"172.21.15.178:6819/688015531","heartbeat_back_addr":"172.21.15.178:6823/688015531","heartbeat_front_addr":"172.21.15.178:6821/688015531","state":["exists","up"]},{"osd":9,"uuid":"856039d7-f7ee-4dbd-b019-0073e705ce0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6827","nonce":3265797851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6829","nonce":3265797851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6833","nonce":3265797851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6831","nonce":3265797851}]},"public_addr":"172.21.15.160:6827/3265797851","cluster_addr":"172.21.15.160:6829/3265797851","heartbeat_back_addr":"172.21.15.160:6833/3265797851","heartbeat_front_addr":"172.21.15.160:6831/3265797851","state":["exists","up"]},{"osd":10,"uuid":"bdc9899d-d720-4429-ad53-4c388a4dbe75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6825","nonce":2867473412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6827","nonce":2867473412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6831","nonce":2867473412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6829","nonce":2867473412}]},"public_addr":"172.21.15.177:6825/2867473412","cluster_addr":"172.21.15.177:6827/2867473412","heartbeat_back_addr":"172.21.15.177:6831/2867473412","heartbeat_front_addr":"172.21.15.177:6829/2867473412","state":["exists","up"]},{"osd":11,"uuid":"cc9adb54-ef49-47c6-90e3-fc03bc0f5cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6825","nonce":4068724731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6827","nonce":4068724731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6831","nonce":4068724731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6829","nonce":4068724731}]},"public_addr":"172.21.15.178:6825/4068724731","cluster_addr":"172.21.15.178:6827/4068724731","heartbeat_back_addr":"172.21.15.178:6831/4068724731","heartbeat_front_addr":"172.21.15.178:6829/4068724731","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:33:44.141585+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:07.685585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:31.044386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:56.308925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:35:21.582097+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:35:45.983540+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:36:11.530242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:36:36.844681+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:02.303437+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:27.629867+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:53.289495+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:38:18.536169+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/3027450612":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/874709722":"2024-03-08T07:32:08.190837+0000","172.21.15.160:6800/1130675225":"2024-03-08T07:31:41.166858+0000","172.21.15.160:0/3857193980":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/3861968297":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6800/3859174315":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/3013534743":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6801/3859174315":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/1310937667":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6801/1130675225":"2024-03-08T07:31:41.166858+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T07:41:01.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.839+0000 7fc4463ee640 1 -- 172.21.15.160:0/2343518890 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fc4280a1d80 msgr2=0x7fc4280a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:01.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.839+0000 7fc4463ee640 1 --2- 172.21.15.160:0/2343518890 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fc4280a1d80 0x7fc4280a4240 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc440197db0 tx=0x7fc430005f90 comp rx=0 tx=0).stop 2024-03-07T07:41:01.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.839+0000 7fc4463ee640 1 -- 172.21.15.160:0/2343518890 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc440100580 msgr2=0x7fc4401968a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:01.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.839+0000 7fc4463ee640 1 --2- 172.21.15.160:0/2343518890 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc440100580 0x7fc4401968a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc434000c00 tx=0x7fc43402c640 comp rx=0 tx=0).stop 2024-03-07T07:41:01.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.839+0000 7fc4463ee640 1 -- 172.21.15.160:0/2343518890 shutdown_connections 2024-03-07T07:41:01.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.839+0000 7fc4463ee640 1 --2- 172.21.15.160:0/2343518890 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fc4280a1d80 0x7fc4280a4240 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:01.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.839+0000 7fc4463ee640 1 --2- 172.21.15.160:0/2343518890 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc440102130 0x7fc44019de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:01.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.839+0000 7fc4463ee640 1 --2- 172.21.15.160:0/2343518890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc440101770 0x7fc440196de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:01.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.839+0000 7fc4463ee640 1 --2- 172.21.15.160:0/2343518890 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc440100580 0x7fc4401968a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:01.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.839+0000 7fc4463ee640 1 -- 172.21.15.160:0/2343518890 >> 172.21.15.160:0/2343518890 conn(0x7fc4400fbaf0 msgr2=0x7fc440104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:01.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.839+0000 7fc4463ee640 1 -- 172.21.15.160:0/2343518890 shutdown_connections 2024-03-07T07:41:01.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.839+0000 7fc4463ee640 1 -- 172.21.15.160:0/2343518890 wait complete. 2024-03-07T07:41:01.851 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-07T07:34:34.529424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-07T07:39:12.075432+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-07T07:39:13.367820+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-07T07:41:01.851 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-07T07:41:01.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:01 smithi177 ceph-mon[36578]: pgmap v343: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-07T07:41:01.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:01 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/1238650176' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-07T07:41:01.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.956+0000 7f61aa7eb640 1 -- 172.21.15.160:0/735825545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61a4107350 msgr2=0x7f61a41097a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:01.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.956+0000 7f61aa7eb640 1 --2- 172.21.15.160:0/735825545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61a4107350 0x7f61a41097a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f619c009ca0 tx=0x7f619c02c0f0 comp rx=0 tx=0).stop 2024-03-07T07:41:01.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.956+0000 7f61aa7eb640 1 -- 172.21.15.160:0/735825545 shutdown_connections 2024-03-07T07:41:01.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.956+0000 7f61aa7eb640 1 --2- 172.21.15.160:0/735825545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61a4107350 0x7f61a41097a0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:01.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.956+0000 7f61aa7eb640 1 --2- 172.21.15.160:0/735825545 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61a4104a20 0x7f61a4106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:01.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.956+0000 7f61aa7eb640 1 --2- 172.21.15.160:0/735825545 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61a41020f0 0x7f61a41044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:01.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.956+0000 7f61aa7eb640 1 -- 172.21.15.160:0/735825545 >> 172.21.15.160:0/735825545 conn(0x7f61a40fbb00 msgr2=0x7f61a40fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:01.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.956+0000 7f61aa7eb640 1 -- 172.21.15.160:0/735825545 shutdown_connections 2024-03-07T07:41:01.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.956+0000 7f61aa7eb640 1 -- 172.21.15.160:0/735825545 wait complete. 2024-03-07T07:41:01.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.957+0000 7f61aa7eb640 1 Processor -- start 2024-03-07T07:41:01.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.957+0000 7f61aa7eb640 1 -- start start 2024-03-07T07:41:01.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.957+0000 7f61aa7eb640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61a41020f0 0x7f61a4197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:01.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.957+0000 7f61aa7eb640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61a4104a20 0x7f61a4197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:01.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.957+0000 7f61aa7eb640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61a4107350 0x7f61a419eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:01.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.957+0000 7f61aa7eb640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a410d2f0 con 0x7f61a4107350 2024-03-07T07:41:01.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.957+0000 7f61aa7eb640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a410d170 con 0x7f61a41020f0 2024-03-07T07:41:01.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.957+0000 7f61aa7eb640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a410d470 con 0x7f61a4104a20 2024-03-07T07:41:01.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.957+0000 7f61a8d88640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61a4107350 0x7f61a419eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:01.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.957+0000 7f61a8d88640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61a4107350 0x7f61a419eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57706/0 (socket says 172.21.15.160:57706) 2024-03-07T07:41:01.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.957+0000 7f61a8d88640 1 -- 172.21.15.160:0/3120310287 learned_addr learned my addr 172.21.15.160:0/3120310287 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:41:01.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.957+0000 7f61a37fe640 1 --2- 172.21.15.160:0/3120310287 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61a4104a20 0x7f61a4197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:01.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.957+0000 7f61a3fff640 1 --2- 172.21.15.160:0/3120310287 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61a41020f0 0x7f61a4197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:01.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.957+0000 7f61a8d88640 1 -- 172.21.15.160:0/3120310287 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61a4104a20 msgr2=0x7f61a4197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:01.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.957+0000 7f61a8d88640 1 --2- 172.21.15.160:0/3120310287 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61a4104a20 0x7f61a4197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:01.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.957+0000 7f61a8d88640 1 -- 172.21.15.160:0/3120310287 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61a41020f0 msgr2=0x7f61a4197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:01.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.957+0000 7f61a8d88640 1 --2- 172.21.15.160:0/3120310287 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61a41020f0 0x7f61a4197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:01.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.957+0000 7f61a8d88640 1 -- 172.21.15.160:0/3120310287 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a419f550 con 0x7f61a4107350 2024-03-07T07:41:01.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.957+0000 7f61a8d88640 1 --2- 172.21.15.160:0/3120310287 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61a4107350 0x7f61a419eee0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f619c037040 tx=0x7f619c032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:01.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.958+0000 7f61a17fa640 1 -- 172.21.15.160:0/3120310287 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619c041020 con 0x7f61a4107350 2024-03-07T07:41:01.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.958+0000 7f61aa7eb640 1 -- 172.21.15.160:0/3120310287 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61a419f7e0 con 0x7f61a4107350 2024-03-07T07:41:01.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.958+0000 7f61a17fa640 1 -- 172.21.15.160:0/3120310287 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f619c034cd0 con 0x7f61a4107350 2024-03-07T07:41:01.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.958+0000 7f61a17fa640 1 -- 172.21.15.160:0/3120310287 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619c039040 con 0x7f61a4107350 2024-03-07T07:41:01.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.958+0000 7f61aa7eb640 1 -- 172.21.15.160:0/3120310287 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61a419fd20 con 0x7f61a4107350 2024-03-07T07:41:01.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.959+0000 7f61aa7eb640 1 -- 172.21.15.160:0/3120310287 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61a40fd5f0 con 0x7f61a4107350 2024-03-07T07:41:01.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.960+0000 7f61a17fa640 1 -- 172.21.15.160:0/3120310287 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f619c044020 con 0x7f61a4107350 2024-03-07T07:41:01.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.960+0000 7f61a17fa640 1 --2- 172.21.15.160:0/3120310287 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f618c0a1d80 0x7f618c0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:01.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.960+0000 7f61a3fff640 1 --2- 172.21.15.160:0/3120310287 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f618c0a1d80 0x7f618c0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:01.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.960+0000 7f61a17fa640 1 -- 172.21.15.160:0/3120310287 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f619c0efc00 con 0x7f61a4107350 2024-03-07T07:41:01.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.961+0000 7f61a3fff640 1 --2- 172.21.15.160:0/3120310287 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f618c0a1d80 0x7f618c0a4240 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6194009bd0 tx=0x7f6194005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:01.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:01.965+0000 7f61a17fa640 1 -- 172.21.15.160:0/3120310287 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f619c04b020 con 0x7f61a4107350 2024-03-07T07:41:02.138 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.138+0000 7f61aa7eb640 1 -- 172.21.15.160:0/3120310287 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f61a4062260 con 0x7f61a4107350 2024-03-07T07:41:02.138 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.138+0000 7f61a17fa640 1 -- 172.21.15.160:0/3120310287 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f619c0bd0c0 con 0x7f61a4107350 2024-03-07T07:41:02.138 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 1 2024-03-07T07:41:02.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.140+0000 7f61aa7eb640 1 -- 172.21.15.160:0/3120310287 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f618c0a1d80 msgr2=0x7f618c0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:02.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.140+0000 7f61aa7eb640 1 --2- 172.21.15.160:0/3120310287 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f618c0a1d80 0x7f618c0a4240 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6194009bd0 tx=0x7f6194005f90 comp rx=0 tx=0).stop 2024-03-07T07:41:02.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.140+0000 7f61aa7eb640 1 -- 172.21.15.160:0/3120310287 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61a4107350 msgr2=0x7f61a419eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:02.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.140+0000 7f61aa7eb640 1 --2- 172.21.15.160:0/3120310287 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61a4107350 0x7f61a419eee0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f619c037040 tx=0x7f619c032f40 comp rx=0 tx=0).stop 2024-03-07T07:41:02.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.140+0000 7f61aa7eb640 1 -- 172.21.15.160:0/3120310287 shutdown_connections 2024-03-07T07:41:02.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.140+0000 7f61aa7eb640 1 --2- 172.21.15.160:0/3120310287 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f618c0a1d80 0x7f618c0a4240 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:02.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.140+0000 7f61aa7eb640 1 --2- 172.21.15.160:0/3120310287 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61a4107350 0x7f61a419eee0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:02.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.140+0000 7f61aa7eb640 1 --2- 172.21.15.160:0/3120310287 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61a4104a20 0x7f61a4197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:02.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.140+0000 7f61aa7eb640 1 --2- 172.21.15.160:0/3120310287 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61a41020f0 0x7f61a4197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:02.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.140+0000 7f61aa7eb640 1 -- 172.21.15.160:0/3120310287 >> 172.21.15.160:0/3120310287 conn(0x7f61a40fbb00 msgr2=0x7f61a4102a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:02.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.140+0000 7f61aa7eb640 1 -- 172.21.15.160:0/3120310287 shutdown_connections 2024-03-07T07:41:02.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.140+0000 7f61aa7eb640 1 -- 172.21.15.160:0/3120310287 wait complete. 2024-03-07T07:41:02.152 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-07T07:41:02.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.256+0000 7f476aedd640 1 -- 172.21.15.160:0/1829670304 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4764100560 msgr2=0x7f4764100980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:02.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.256+0000 7f476aedd640 1 --2- 172.21.15.160:0/1829670304 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4764100560 0x7f4764100980 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4758009900 tx=0x7f475802bc30 comp rx=0 tx=0).stop 2024-03-07T07:41:02.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.256+0000 7f476aedd640 1 -- 172.21.15.160:0/1829670304 shutdown_connections 2024-03-07T07:41:02.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.256+0000 7f476aedd640 1 --2- 172.21.15.160:0/1829670304 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4764102110 0x7f47641089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:02.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.256+0000 7f476aedd640 1 --2- 172.21.15.160:0/1829670304 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4764101750 0x7f4764101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:02.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.256+0000 7f476aedd640 1 --2- 172.21.15.160:0/1829670304 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4764100560 0x7f4764100980 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:02.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.257+0000 7f476aedd640 1 -- 172.21.15.160:0/1829670304 >> 172.21.15.160:0/1829670304 conn(0x7f47640fbaf0 msgr2=0x7f47640fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:02.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.257+0000 7f476aedd640 1 -- 172.21.15.160:0/1829670304 shutdown_connections 2024-03-07T07:41:02.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.257+0000 7f476aedd640 1 -- 172.21.15.160:0/1829670304 wait complete. 2024-03-07T07:41:02.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.257+0000 7f476aedd640 1 Processor -- start 2024-03-07T07:41:02.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.257+0000 7f476aedd640 1 -- start start 2024-03-07T07:41:02.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.257+0000 7f476aedd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4764100560 0x7f47641979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:02.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.257+0000 7f476aedd640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4764101750 0x7f4764197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:02.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.257+0000 7f476aedd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4764102110 0x7f476419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:02.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.257+0000 7f476aedd640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476410d300 con 0x7f4764100560 2024-03-07T07:41:02.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.257+0000 7f476aedd640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476410d180 con 0x7f4764101750 2024-03-07T07:41:02.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.257+0000 7f476aedd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476410d480 con 0x7f4764102110 2024-03-07T07:41:02.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.257+0000 7f4768c79640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4764100560 0x7f47641979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:02.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.257+0000 7f4768c79640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4764100560 0x7f47641979e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57722/0 (socket says 172.21.15.160:57722) 2024-03-07T07:41:02.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.257+0000 7f4768c79640 1 -- 172.21.15.160:0/1671649614 learned_addr learned my addr 172.21.15.160:0/1671649614 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:41:02.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.257+0000 7f4763fff640 1 --2- 172.21.15.160:0/1671649614 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4764101750 0x7f4764197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:02.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.257+0000 7f476947a640 1 --2- 172.21.15.160:0/1671649614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4764102110 0x7f476419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:02.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.258+0000 7f4768c79640 1 -- 172.21.15.160:0/1671649614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4764102110 msgr2=0x7f476419ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:02.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.258+0000 7f4768c79640 1 --2- 172.21.15.160:0/1671649614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4764102110 0x7f476419ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:02.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.258+0000 7f4768c79640 1 -- 172.21.15.160:0/1671649614 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4764101750 msgr2=0x7f4764197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:02.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.258+0000 7f4768c79640 1 --2- 172.21.15.160:0/1671649614 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4764101750 0x7f4764197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:02.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.258+0000 7f4768c79640 1 -- 172.21.15.160:0/1671649614 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f476419f600 con 0x7f4764100560 2024-03-07T07:41:02.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.258+0000 7f4768c79640 1 --2- 172.21.15.160:0/1671649614 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4764100560 0x7f47641979e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4758000c00 tx=0x7f475802c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:02.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.258+0000 7f4761ffb640 1 -- 172.21.15.160:0/1671649614 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4758041070 con 0x7f4764100560 2024-03-07T07:41:02.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.258+0000 7f4761ffb640 1 -- 172.21.15.160:0/1671649614 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4758004450 con 0x7f4764100560 2024-03-07T07:41:02.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.258+0000 7f476aedd640 1 -- 172.21.15.160:0/1671649614 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f476419f890 con 0x7f4764100560 2024-03-07T07:41:02.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.258+0000 7f4761ffb640 1 -- 172.21.15.160:0/1671649614 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4758034440 con 0x7f4764100560 2024-03-07T07:41:02.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.258+0000 7f476aedd640 1 -- 172.21.15.160:0/1671649614 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f476419fd80 con 0x7f4764100560 2024-03-07T07:41:02.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.259+0000 7f476aedd640 1 -- 172.21.15.160:0/1671649614 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4764107920 con 0x7f4764100560 2024-03-07T07:41:02.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.260+0000 7f4761ffb640 1 -- 172.21.15.160:0/1671649614 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f4758044070 con 0x7f4764100560 2024-03-07T07:41:02.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.261+0000 7f4761ffb640 1 --2- 172.21.15.160:0/1671649614 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f47540a1d80 0x7f47540a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:02.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.261+0000 7f4763fff640 1 --2- 172.21.15.160:0/1671649614 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f47540a1d80 0x7f47540a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:02.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.261+0000 7f4761ffb640 1 -- 172.21.15.160:0/1671649614 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f47580efa70 con 0x7f4764100560 2024-03-07T07:41:02.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.261+0000 7f4763fff640 1 --2- 172.21.15.160:0/1671649614 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f47540a1d80 0x7f47540a4240 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4764198ef0 tx=0x7f4750005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:02.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.264+0000 7f4761ffb640 1 -- 172.21.15.160:0/1671649614 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f47580bc8e0 con 0x7f4764100560 2024-03-07T07:41:02.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:02 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/2343518890' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:41:02.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:02 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/3120310287' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-07T07:41:02.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.440+0000 7f476aedd640 1 -- 172.21.15.160:0/1671649614 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4764062260 con 0x7f4764100560 2024-03-07T07:41:02.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.440+0000 7f4761ffb640 1 -- 172.21.15.160:0/1671649614 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f47580c0980 con 0x7f4764100560 2024-03-07T07:41:02.441 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 32 2024-03-07T07:41:02.443 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.442+0000 7f476aedd640 1 -- 172.21.15.160:0/1671649614 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f47540a1d80 msgr2=0x7f47540a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:02.443 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.442+0000 7f476aedd640 1 --2- 172.21.15.160:0/1671649614 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f47540a1d80 0x7f47540a4240 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4764198ef0 tx=0x7f4750005f90 comp rx=0 tx=0).stop 2024-03-07T07:41:02.443 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.442+0000 7f476aedd640 1 -- 172.21.15.160:0/1671649614 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4764100560 msgr2=0x7f47641979e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:02.443 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.442+0000 7f476aedd640 1 --2- 172.21.15.160:0/1671649614 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4764100560 0x7f47641979e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4758000c00 tx=0x7f475802c640 comp rx=0 tx=0).stop 2024-03-07T07:41:02.443 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.442+0000 7f476aedd640 1 -- 172.21.15.160:0/1671649614 shutdown_connections 2024-03-07T07:41:02.443 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.442+0000 7f476aedd640 1 --2- 172.21.15.160:0/1671649614 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f47540a1d80 0x7f47540a4240 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:02.443 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.442+0000 7f476aedd640 1 --2- 172.21.15.160:0/1671649614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4764102110 0x7f476419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:02.443 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.442+0000 7f476aedd640 1 --2- 172.21.15.160:0/1671649614 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4764101750 0x7f4764197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:02.443 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.442+0000 7f476aedd640 1 --2- 172.21.15.160:0/1671649614 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4764100560 0x7f47641979e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:02.443 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.442+0000 7f476aedd640 1 -- 172.21.15.160:0/1671649614 >> 172.21.15.160:0/1671649614 conn(0x7f47640fbaf0 msgr2=0x7f4764104af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:02.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.443+0000 7f476aedd640 1 -- 172.21.15.160:0/1671649614 shutdown_connections 2024-03-07T07:41:02.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.443+0000 7f476aedd640 1 -- 172.21.15.160:0/1671649614 wait complete. 2024-03-07T07:41:02.454 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-07T07:41:02.562 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.559+0000 7faf54399640 1 -- 172.21.15.160:0/4113621630 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf4c1021a0 msgr2=0x7faf4c104590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:02.562 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.559+0000 7faf54399640 1 --2- 172.21.15.160:0/4113621630 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf4c1021a0 0x7faf4c104590 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7faf44009ca0 tx=0x7faf4402c0f0 comp rx=0 tx=0).stop 2024-03-07T07:41:02.562 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.559+0000 7faf54399640 1 -- 172.21.15.160:0/4113621630 shutdown_connections 2024-03-07T07:41:02.562 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.559+0000 7faf54399640 1 --2- 172.21.15.160:0/4113621630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf4c107400 0x7faf4c1097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:02.562 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.559+0000 7faf54399640 1 --2- 172.21.15.160:0/4113621630 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faf4c104ad0 0x7faf4c106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:02.562 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.559+0000 7faf54399640 1 --2- 172.21.15.160:0/4113621630 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf4c1021a0 0x7faf4c104590 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:02.562 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.559+0000 7faf54399640 1 -- 172.21.15.160:0/4113621630 >> 172.21.15.160:0/4113621630 conn(0x7faf4c0fbaf0 msgr2=0x7faf4c0fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:02.562 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.559+0000 7faf54399640 1 -- 172.21.15.160:0/4113621630 shutdown_connections 2024-03-07T07:41:02.562 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.559+0000 7faf54399640 1 -- 172.21.15.160:0/4113621630 wait complete. 2024-03-07T07:41:02.562 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.560+0000 7faf54399640 1 Processor -- start 2024-03-07T07:41:02.562 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.560+0000 7faf54399640 1 -- start start 2024-03-07T07:41:02.563 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.560+0000 7faf54399640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf4c1021a0 0x7faf4c197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:02.563 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.560+0000 7faf54399640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faf4c104ad0 0x7faf4c197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:02.563 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.560+0000 7faf54399640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf4c107400 0x7faf4c19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:02.563 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.560+0000 7faf54399640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c10d310 con 0x7faf4c107400 2024-03-07T07:41:02.563 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.560+0000 7faf54399640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c10d190 con 0x7faf4c104ad0 2024-03-07T07:41:02.563 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.560+0000 7faf54399640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c10d490 con 0x7faf4c1021a0 2024-03-07T07:41:02.563 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.560+0000 7faf52936640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf4c107400 0x7faf4c19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:02.563 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.560+0000 7faf52936640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf4c107400 0x7faf4c19efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57750/0 (socket says 172.21.15.160:57750) 2024-03-07T07:41:02.563 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.560+0000 7faf52936640 1 -- 172.21.15.160:0/3941428160 learned_addr learned my addr 172.21.15.160:0/3941428160 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:41:02.563 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.560+0000 7faf52135640 1 --2- 172.21.15.160:0/3941428160 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf4c1021a0 0x7faf4c197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:02.563 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.560+0000 7faf51934640 1 --2- 172.21.15.160:0/3941428160 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faf4c104ad0 0x7faf4c197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:02.563 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.560+0000 7faf52936640 1 -- 172.21.15.160:0/3941428160 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf4c1021a0 msgr2=0x7faf4c197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:02.564 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.560+0000 7faf52936640 1 --2- 172.21.15.160:0/3941428160 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf4c1021a0 0x7faf4c197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:02.564 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.560+0000 7faf52936640 1 -- 172.21.15.160:0/3941428160 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faf4c104ad0 msgr2=0x7faf4c197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:02.564 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.560+0000 7faf52936640 1 --2- 172.21.15.160:0/3941428160 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faf4c104ad0 0x7faf4c197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:02.564 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.560+0000 7faf52936640 1 -- 172.21.15.160:0/3941428160 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf4c19f630 con 0x7faf4c107400 2024-03-07T07:41:02.564 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.560+0000 7faf52135640 1 --2- 172.21.15.160:0/3941428160 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf4c1021a0 0x7faf4c197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:41:02.564 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.560+0000 7faf51934640 1 --2- 172.21.15.160:0/3941428160 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faf4c104ad0 0x7faf4c197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:41:02.564 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.560+0000 7faf52936640 1 --2- 172.21.15.160:0/3941428160 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf4c107400 0x7faf4c19efc0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7faf4800e970 tx=0x7faf4800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:02.564 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.561+0000 7faf437fe640 1 -- 172.21.15.160:0/3941428160 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4800ccc0 con 0x7faf4c107400 2024-03-07T07:41:02.564 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.561+0000 7faf437fe640 1 -- 172.21.15.160:0/3941428160 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faf4800ce60 con 0x7faf4c107400 2024-03-07T07:41:02.564 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.561+0000 7faf54399640 1 -- 172.21.15.160:0/3941428160 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf4c19f920 con 0x7faf4c107400 2024-03-07T07:41:02.564 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.561+0000 7faf437fe640 1 -- 172.21.15.160:0/3941428160 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf48010490 con 0x7faf4c107400 2024-03-07T07:41:02.564 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.561+0000 7faf54399640 1 -- 172.21.15.160:0/3941428160 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf4c19fe70 con 0x7faf4c107400 2024-03-07T07:41:02.564 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.562+0000 7faf54399640 1 -- 172.21.15.160:0/3941428160 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf4c0fd5e0 con 0x7faf4c107400 2024-03-07T07:41:02.565 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.563+0000 7faf437fe640 1 -- 172.21.15.160:0/3941428160 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 136754+0+0 (secure 0 0 0) 0x7faf48005020 con 0x7faf4c107400 2024-03-07T07:41:02.565 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.563+0000 7faf437fe640 1 --2- 172.21.15.160:0/3941428160 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7faf380a1d80 0x7faf380a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:02.565 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.563+0000 7faf52135640 1 --2- 172.21.15.160:0/3941428160 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7faf380a1d80 0x7faf380a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:02.565 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.563+0000 7faf437fe640 1 -- 172.21.15.160:0/3941428160 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7faf480ce1d0 con 0x7faf4c107400 2024-03-07T07:41:02.566 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.564+0000 7faf52135640 1 --2- 172.21.15.160:0/3941428160 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7faf380a1d80 0x7faf380a4240 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7faf44006010 tx=0x7faf44032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:02.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.568+0000 7faf437fe640 1 -- 172.21.15.160:0/3941428160 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7faf4809b600 con 0x7faf4c107400 2024-03-07T07:41:02.742 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.742+0000 7faf54399640 1 -- 172.21.15.160:0/3941428160 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faf4c062260 con 0x7faf4c107400 2024-03-07T07:41:02.743 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.743+0000 7faf437fe640 1 -- 172.21.15.160:0/3941428160 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7faf4809af00 con 0x7faf4c107400 2024-03-07T07:41:02.743 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 64 2024-03-07T07:41:02.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.744+0000 7faf54399640 1 -- 172.21.15.160:0/3941428160 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7faf380a1d80 msgr2=0x7faf380a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:02.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.744+0000 7faf54399640 1 --2- 172.21.15.160:0/3941428160 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7faf380a1d80 0x7faf380a4240 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7faf44006010 tx=0x7faf44032590 comp rx=0 tx=0).stop 2024-03-07T07:41:02.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.744+0000 7faf54399640 1 -- 172.21.15.160:0/3941428160 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf4c107400 msgr2=0x7faf4c19efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:02.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.744+0000 7faf54399640 1 --2- 172.21.15.160:0/3941428160 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf4c107400 0x7faf4c19efc0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7faf4800e970 tx=0x7faf4800ed70 comp rx=0 tx=0).stop 2024-03-07T07:41:02.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.745+0000 7faf54399640 1 -- 172.21.15.160:0/3941428160 shutdown_connections 2024-03-07T07:41:02.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.745+0000 7faf54399640 1 --2- 172.21.15.160:0/3941428160 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7faf380a1d80 0x7faf380a4240 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:02.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.745+0000 7faf54399640 1 --2- 172.21.15.160:0/3941428160 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf4c107400 0x7faf4c19efc0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:02.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.745+0000 7faf54399640 1 --2- 172.21.15.160:0/3941428160 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faf4c104ad0 0x7faf4c197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:02.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.745+0000 7faf54399640 1 --2- 172.21.15.160:0/3941428160 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf4c1021a0 0x7faf4c197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:02.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.745+0000 7faf54399640 1 -- 172.21.15.160:0/3941428160 >> 172.21.15.160:0/3941428160 conn(0x7faf4c0fbaf0 msgr2=0x7faf4c102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:02.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.745+0000 7faf54399640 1 -- 172.21.15.160:0/3941428160 shutdown_connections 2024-03-07T07:41:02.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.745+0000 7faf54399640 1 -- 172.21.15.160:0/3941428160 wait complete. 2024-03-07T07:41:02.756 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T07:41:02.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:02 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/2343518890' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:41:02.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:02 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/3120310287' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-07T07:41:02.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:02 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/2343518890' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:41:02.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:02 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/3120310287' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-07T07:41:02.860 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.858+0000 7f395a85e640 1 -- 172.21.15.160:0/793123831 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3954103250 msgr2=0x7f3954109ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:02.860 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.858+0000 7f395a85e640 1 --2- 172.21.15.160:0/793123831 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3954103250 0x7f3954109ae0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f394c009ca0 tx=0x7f394c02c0f0 comp rx=0 tx=0).stop 2024-03-07T07:41:02.860 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.858+0000 7f395a85e640 1 -- 172.21.15.160:0/793123831 shutdown_connections 2024-03-07T07:41:02.861 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.858+0000 7f395a85e640 1 --2- 172.21.15.160:0/793123831 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3954103250 0x7f3954109ae0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:02.861 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.858+0000 7f395a85e640 1 --2- 172.21.15.160:0/793123831 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3954102890 0x7f3954102d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:02.861 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.858+0000 7f395a85e640 1 --2- 172.21.15.160:0/793123831 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f39541016a0 0x7f3954101ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:02.861 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.858+0000 7f395a85e640 1 -- 172.21.15.160:0/793123831 >> 172.21.15.160:0/793123831 conn(0x7f39540fcc10 msgr2=0x7f39540ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:02.861 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.858+0000 7f395a85e640 1 -- 172.21.15.160:0/793123831 shutdown_connections 2024-03-07T07:41:02.861 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.858+0000 7f395a85e640 1 -- 172.21.15.160:0/793123831 wait complete. 2024-03-07T07:41:02.861 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.858+0000 7f395a85e640 1 Processor -- start 2024-03-07T07:41:02.861 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.858+0000 7f395a85e640 1 -- start start 2024-03-07T07:41:02.861 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.859+0000 7f395a85e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39541016a0 0x7f3954197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:02.861 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.859+0000 7f395a85e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3954102890 0x7f3954197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:02.861 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.859+0000 7f395a85e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3954103250 0x7f395419eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:02.862 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.859+0000 7f395a85e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f395410d300 con 0x7f39541016a0 2024-03-07T07:41:02.862 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.859+0000 7f395a85e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f395410d180 con 0x7f3954102890 2024-03-07T07:41:02.862 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.859+0000 7f395a85e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f395410d480 con 0x7f3954103250 2024-03-07T07:41:02.862 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.859+0000 7f3953fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39541016a0 0x7f3954197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:02.862 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.859+0000 7f3953fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39541016a0 0x7f3954197920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57762/0 (socket says 172.21.15.160:57762) 2024-03-07T07:41:02.862 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.859+0000 7f3953fff640 1 -- 172.21.15.160:0/756136389 learned_addr learned my addr 172.21.15.160:0/756136389 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:41:02.862 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.859+0000 7f3958dfb640 1 --2- 172.21.15.160:0/756136389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3954103250 0x7f395419eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:02.862 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.859+0000 7f39537fe640 1 --2- 172.21.15.160:0/756136389 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3954102890 0x7f3954197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:02.862 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.859+0000 7f3953fff640 1 -- 172.21.15.160:0/756136389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3954103250 msgr2=0x7f395419eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:02.862 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.859+0000 7f3953fff640 1 --2- 172.21.15.160:0/756136389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3954103250 0x7f395419eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:02.862 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.859+0000 7f3953fff640 1 -- 172.21.15.160:0/756136389 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3954102890 msgr2=0x7f3954197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:02.862 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.859+0000 7f3953fff640 1 --2- 172.21.15.160:0/756136389 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3954102890 0x7f3954197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:02.862 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.859+0000 7f3953fff640 1 -- 172.21.15.160:0/756136389 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f395419f540 con 0x7f39541016a0 2024-03-07T07:41:02.863 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.859+0000 7f3953fff640 1 --2- 172.21.15.160:0/756136389 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39541016a0 0x7f3954197920 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f394800e9c0 tx=0x7f394800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:02.863 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.859+0000 7f39517fa640 1 -- 172.21.15.160:0/756136389 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394800cd80 con 0x7f39541016a0 2024-03-07T07:41:02.863 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.859+0000 7f395a85e640 1 -- 172.21.15.160:0/756136389 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f395419f830 con 0x7f39541016a0 2024-03-07T07:41:02.863 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.859+0000 7f39517fa640 1 -- 172.21.15.160:0/756136389 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f39480035f0 con 0x7f39541016a0 2024-03-07T07:41:02.863 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.859+0000 7f39517fa640 1 -- 172.21.15.160:0/756136389 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3948010690 con 0x7f39541016a0 2024-03-07T07:41:02.863 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.859+0000 7f395a85e640 1 -- 172.21.15.160:0/756136389 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f395419fd80 con 0x7f39541016a0 2024-03-07T07:41:02.864 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.860+0000 7f395a85e640 1 -- 172.21.15.160:0/756136389 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39541089c0 con 0x7f39541016a0 2024-03-07T07:41:02.864 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.861+0000 7f39517fa640 1 -- 172.21.15.160:0/756136389 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f39480037a0 con 0x7f39541016a0 2024-03-07T07:41:02.864 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.862+0000 7f39517fa640 1 --2- 172.21.15.160:0/756136389 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f393c0a1d80 0x7f393c0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:02.864 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.862+0000 7f39537fe640 1 --2- 172.21.15.160:0/756136389 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f393c0a1d80 0x7f393c0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:02.864 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.862+0000 7f39517fa640 1 -- 172.21.15.160:0/756136389 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f39480ce130 con 0x7f39541016a0 2024-03-07T07:41:02.864 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.862+0000 7f39537fe640 1 --2- 172.21.15.160:0/756136389 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f393c0a1d80 0x7f393c0a4240 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3954198e30 tx=0x7f3944005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:02.866 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:02.866+0000 7f39517fa640 1 -- 172.21.15.160:0/756136389 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f394809b560 con 0x7f39541016a0 2024-03-07T07:41:03.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.076+0000 7f395a85e640 1 -- 172.21.15.160:0/756136389 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f395402d030 con 0x7f39541016a0 2024-03-07T07:41:03.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.077+0000 7f39517fa640 1 -- 172.21.15.160:0/756136389 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+11411 (secure 0 0 0) 0x7f395402d030 con 0x7f39541016a0 2024-03-07T07:41:03.078 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:41:03.078 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/2615961083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2615961083},{"type":"v1","addr":"172.21.15.160:6835","nonce":2615961083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1759006033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1759006033},{"type":"v1","addr":"172.21.15.160:6837","nonce":1759006033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6839/3763944721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":3763944721},{"type":"v1","addr":"172.21.15.160:6839","nonce":3763944721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6841/3611426637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6840","nonce":3611426637},{"type":"v1","addr":"172.21.15.160:6841","nonce":3611426637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1457364761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1457364761},{"type":"v1","addr":"172.21.15.177:6833","nonce":1457364761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/170428292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":170428292},{"type":"v1","addr":"172.21.15.177:6835","nonce":170428292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-07T07:39:14.164243+0000","modified":"2024-03-07T07:39:18.861903+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24511,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/645365846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":645365846},{"type":"v1","addr":"172.21.15.178:6833","nonce":645365846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6837/4088036516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":4088036516},{"type":"v1","addr":"172.21.15.177:6837","nonce":4088036516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/647840547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":647840547},{"type":"v1","addr":"172.21.15.178:6835","nonce":647840547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.178:6837/1247829154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1247829154},{"type":"v1","addr":"172.21.15.178:6837","nonce":1247829154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24497":{"gid":24497,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/3329773838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3329773838},{"type":"v1","addr":"172.21.15.177:6839","nonce":3329773838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24511":{"gid":24511,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.178:6839/2947354024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":2947354024},{"type":"v1","addr":"172.21.15.178:6839","nonce":2947354024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24511,"qdb_cluster":[24487,24511,24497]},"id":1}]} 2024-03-07T07:41:03.078 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 21 2024-03-07T07:41:03.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.079+0000 7f395a85e640 1 -- 172.21.15.160:0/756136389 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f393c0a1d80 msgr2=0x7f393c0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:03.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.079+0000 7f395a85e640 1 --2- 172.21.15.160:0/756136389 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f393c0a1d80 0x7f393c0a4240 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3954198e30 tx=0x7f3944005f90 comp rx=0 tx=0).stop 2024-03-07T07:41:03.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.079+0000 7f395a85e640 1 -- 172.21.15.160:0/756136389 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39541016a0 msgr2=0x7f3954197920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:03.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.079+0000 7f395a85e640 1 --2- 172.21.15.160:0/756136389 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39541016a0 0x7f3954197920 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f394800e9c0 tx=0x7f394800ee80 comp rx=0 tx=0).stop 2024-03-07T07:41:03.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.079+0000 7f395a85e640 1 -- 172.21.15.160:0/756136389 shutdown_connections 2024-03-07T07:41:03.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.079+0000 7f395a85e640 1 --2- 172.21.15.160:0/756136389 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f393c0a1d80 0x7f393c0a4240 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:03.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.079+0000 7f395a85e640 1 --2- 172.21.15.160:0/756136389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3954103250 0x7f395419eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:03.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.079+0000 7f395a85e640 1 --2- 172.21.15.160:0/756136389 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3954102890 0x7f3954197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:03.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.079+0000 7f395a85e640 1 --2- 172.21.15.160:0/756136389 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39541016a0 0x7f3954197920 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:03.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.079+0000 7f395a85e640 1 -- 172.21.15.160:0/756136389 >> 172.21.15.160:0/756136389 conn(0x7f39540fcc10 msgr2=0x7f3954105b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:03.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.079+0000 7f395a85e640 1 -- 172.21.15.160:0/756136389 shutdown_connections 2024-03-07T07:41:03.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.079+0000 7f395a85e640 1 -- 172.21.15.160:0/756136389 wait complete. 2024-03-07T07:41:03.092 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T07:41:03.200 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.196+0000 7fa39df1d640 1 -- 172.21.15.160:0/2650520303 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa398100580 msgr2=0x7fa3981009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:03.201 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.196+0000 7fa39df1d640 1 --2- 172.21.15.160:0/2650520303 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa398100580 0x7fa3981009a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa388009900 tx=0x7fa38802bc30 comp rx=0 tx=0).stop 2024-03-07T07:41:03.201 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.196+0000 7fa39df1d640 1 -- 172.21.15.160:0/2650520303 shutdown_connections 2024-03-07T07:41:03.201 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.196+0000 7fa39df1d640 1 --2- 172.21.15.160:0/2650520303 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa398102130 0x7fa3981089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:03.201 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.196+0000 7fa39df1d640 1 --2- 172.21.15.160:0/2650520303 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa398101770 0x7fa398101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:03.201 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.196+0000 7fa39df1d640 1 --2- 172.21.15.160:0/2650520303 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa398100580 0x7fa3981009a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:03.201 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.196+0000 7fa39df1d640 1 -- 172.21.15.160:0/2650520303 >> 172.21.15.160:0/2650520303 conn(0x7fa3980fbaf0 msgr2=0x7fa3980fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:03.201 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.196+0000 7fa39df1d640 1 -- 172.21.15.160:0/2650520303 shutdown_connections 2024-03-07T07:41:03.201 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.197+0000 7fa39df1d640 1 -- 172.21.15.160:0/2650520303 wait complete. 2024-03-07T07:41:03.201 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.197+0000 7fa39df1d640 1 Processor -- start 2024-03-07T07:41:03.202 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.197+0000 7fa39df1d640 1 -- start start 2024-03-07T07:41:03.202 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.197+0000 7fa39df1d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa398100580 0x7fa3981968a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:03.202 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.197+0000 7fa39df1d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa398101770 0x7fa398196de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:03.202 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.197+0000 7fa39df1d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa398102130 0x7fa39819de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:03.202 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.197+0000 7fa39df1d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39810c210 con 0x7fa398102130 2024-03-07T07:41:03.202 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.197+0000 7fa39df1d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39810c090 con 0x7fa398100580 2024-03-07T07:41:03.202 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.197+0000 7fa39df1d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39810c390 con 0x7fa398101770 2024-03-07T07:41:03.202 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.198+0000 7fa397fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa398102130 0x7fa39819de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:03.202 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.198+0000 7fa38effd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa398101770 0x7fa398196de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:03.203 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.198+0000 7fa397fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa398102130 0x7fa39819de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57768/0 (socket says 172.21.15.160:57768) 2024-03-07T07:41:03.203 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.198+0000 7fa397fff640 1 -- 172.21.15.160:0/2049381411 learned_addr learned my addr 172.21.15.160:0/2049381411 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:41:03.203 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.198+0000 7fa3977fe640 1 --2- 172.21.15.160:0/2049381411 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa398100580 0x7fa3981968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:03.203 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.198+0000 7fa38effd640 1 -- 172.21.15.160:0/2049381411 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa398100580 msgr2=0x7fa3981968a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:03.203 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.198+0000 7fa38effd640 1 --2- 172.21.15.160:0/2049381411 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa398100580 0x7fa3981968a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:03.203 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.198+0000 7fa38effd640 1 -- 172.21.15.160:0/2049381411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa398102130 msgr2=0x7fa39819de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:03.203 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.198+0000 7fa38effd640 1 --2- 172.21.15.160:0/2049381411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa398102130 0x7fa39819de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:03.203 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.198+0000 7fa38effd640 1 -- 172.21.15.160:0/2049381411 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa39819e4c0 con 0x7fa398101770 2024-03-07T07:41:03.203 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.198+0000 7fa397fff640 1 --2- 172.21.15.160:0/2049381411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa398102130 0x7fa39819de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:41:03.203 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.198+0000 7fa38effd640 1 --2- 172.21.15.160:0/2049381411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa398101770 0x7fa398196de0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa38000e9f0 tx=0x7fa38000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:03.204 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.198+0000 7fa3957fa640 1 -- 172.21.15.160:0/2049381411 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38000ccc0 con 0x7fa398101770 2024-03-07T07:41:03.204 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.198+0000 7fa3957fa640 1 -- 172.21.15.160:0/2049381411 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa380010910 con 0x7fa398101770 2024-03-07T07:41:03.204 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.198+0000 7fa39df1d640 1 -- 172.21.15.160:0/2049381411 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa39819e7b0 con 0x7fa398101770 2024-03-07T07:41:03.204 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.198+0000 7fa3957fa640 1 -- 172.21.15.160:0/2049381411 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa380018960 con 0x7fa398101770 2024-03-07T07:41:03.204 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.198+0000 7fa39df1d640 1 -- 172.21.15.160:0/2049381411 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa39819ed00 con 0x7fa398101770 2024-03-07T07:41:03.204 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.199+0000 7fa39df1d640 1 -- 172.21.15.160:0/2049381411 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3981078a0 con 0x7fa398101770 2024-03-07T07:41:03.205 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.201+0000 7fa3957fa640 1 -- 172.21.15.160:0/2049381411 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136754+0+0 (secure 0 0 0) 0x7fa380010ac0 con 0x7fa398101770 2024-03-07T07:41:03.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.201+0000 7fa3957fa640 1 --2- 172.21.15.160:0/2049381411 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa3840a1d80 0x7fa3840a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:03.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.201+0000 7fa3977fe640 1 --2- 172.21.15.160:0/2049381411 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa3840a1d80 0x7fa3840a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:03.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.201+0000 7fa3957fa640 1 -- 172.21.15.160:0/2049381411 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa3800ce5d0 con 0x7fa398101770 2024-03-07T07:41:03.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.202+0000 7fa3977fe640 1 --2- 172.21.15.160:0/2049381411 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa3840a1d80 0x7fa3840a4240 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa388000c00 tx=0x7fa388005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:03.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.205+0000 7fa3957fa640 1 -- 172.21.15.160:0/2049381411 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa38009b3b0 con 0x7fa398101770 2024-03-07T07:41:03.390 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.390+0000 7fa39df1d640 1 -- 172.21.15.160:0/2049381411 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa39802d280 con 0x7fa398101770 2024-03-07T07:41:03.391 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.391+0000 7fa3957fa640 1 -- 172.21.15.160:0/2049381411 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22721 (secure 0 0 0) 0x7fa38009f450 con 0x7fa398101770 2024-03-07T07:41:03.392 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:41:03.392 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":87,"fsid":"826f6582-dc54-11ee-95c6-87774f69a715","created":"2024-03-07T07:31:14.229288+0000","modified":"2024-03-07T07:40:09.354265+0000","last_up_change":"2024-03-07T07:38:20.776225+0000","last_in_change":"2024-03-07T07:38:00.508917+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T07:34:34.529424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T07:39:12.075432+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T07:39:13.367820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bee66c67-fb6e-4717-9685-ca62587d5f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6803","nonce":1368482926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6805","nonce":1368482926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6809","nonce":1368482926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6807","nonce":1368482926}]},"public_addr":"172.21.15.160:6803/1368482926","cluster_addr":"172.21.15.160:6805/1368482926","heartbeat_back_addr":"172.21.15.160:6809/1368482926","heartbeat_front_addr":"172.21.15.160:6807/1368482926","state":["exists","up"]},{"osd":1,"uuid":"49236251-f750-424e-992b-bddf55762556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6801","nonce":2922391884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6803","nonce":2922391884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6807","nonce":2922391884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6805","nonce":2922391884}]},"public_addr":"172.21.15.177:6801/2922391884","cluster_addr":"172.21.15.177:6803/2922391884","heartbeat_back_addr":"172.21.15.177:6807/2922391884","heartbeat_front_addr":"172.21.15.177:6805/2922391884","state":["exists","up"]},{"osd":2,"uuid":"7a432b28-f7ee-43fe-9196-7aaeccfecf77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6801","nonce":1217901551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6803","nonce":1217901551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6807","nonce":1217901551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6805","nonce":1217901551}]},"public_addr":"172.21.15.178:6801/1217901551","cluster_addr":"172.21.15.178:6803/1217901551","heartbeat_back_addr":"172.21.15.178:6807/1217901551","heartbeat_front_addr":"172.21.15.178:6805/1217901551","state":["exists","up"]},{"osd":3,"uuid":"4ad22a20-f1be-4002-a48d-ecd3953e63f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6811","nonce":42543989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6813","nonce":42543989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6817","nonce":42543989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6815","nonce":42543989}]},"public_addr":"172.21.15.160:6811/42543989","cluster_addr":"172.21.15.160:6813/42543989","heartbeat_back_addr":"172.21.15.160:6817/42543989","heartbeat_front_addr":"172.21.15.160:6815/42543989","state":["exists","up"]},{"osd":4,"uuid":"2b65b104-ac82-4ce3-b6c5-11a399103045","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6809","nonce":88178553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6811","nonce":88178553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6815","nonce":88178553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6813","nonce":88178553}]},"public_addr":"172.21.15.177:6809/88178553","cluster_addr":"172.21.15.177:6811/88178553","heartbeat_back_addr":"172.21.15.177:6815/88178553","heartbeat_front_addr":"172.21.15.177:6813/88178553","state":["exists","up"]},{"osd":5,"uuid":"30a1e714-08ff-44e8-9264-9ef164604b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6809","nonce":964732891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6811","nonce":964732891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6815","nonce":964732891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6813","nonce":964732891}]},"public_addr":"172.21.15.178:6809/964732891","cluster_addr":"172.21.15.178:6811/964732891","heartbeat_back_addr":"172.21.15.178:6815/964732891","heartbeat_front_addr":"172.21.15.178:6813/964732891","state":["exists","up"]},{"osd":6,"uuid":"b1e1646e-f8a9-42b8-b62d-9d7b5d6a1ff4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6819","nonce":3251327001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6821","nonce":3251327001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6825","nonce":3251327001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6823","nonce":3251327001}]},"public_addr":"172.21.15.160:6819/3251327001","cluster_addr":"172.21.15.160:6821/3251327001","heartbeat_back_addr":"172.21.15.160:6825/3251327001","heartbeat_front_addr":"172.21.15.160:6823/3251327001","state":["exists","up"]},{"osd":7,"uuid":"e66b400b-9747-4f6d-b05b-db22468e2750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6817","nonce":1175500850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6819","nonce":1175500850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6823","nonce":1175500850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6821","nonce":1175500850}]},"public_addr":"172.21.15.177:6817/1175500850","cluster_addr":"172.21.15.177:6819/1175500850","heartbeat_back_addr":"172.21.15.177:6823/1175500850","heartbeat_front_addr":"172.21.15.177:6821/1175500850","state":["exists","up"]},{"osd":8,"uuid":"3ee0f2c4-fca4-49fc-a98b-07aa0fcba0e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6817","nonce":688015531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6819","nonce":688015531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6823","nonce":688015531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6821","nonce":688015531}]},"public_addr":"172.21.15.178:6817/688015531","cluster_addr":"172.21.15.178:6819/688015531","heartbeat_back_addr":"172.21.15.178:6823/688015531","heartbeat_front_addr":"172.21.15.178:6821/688015531","state":["exists","up"]},{"osd":9,"uuid":"856039d7-f7ee-4dbd-b019-0073e705ce0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6827","nonce":3265797851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6829","nonce":3265797851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6833","nonce":3265797851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6831","nonce":3265797851}]},"public_addr":"172.21.15.160:6827/3265797851","cluster_addr":"172.21.15.160:6829/3265797851","heartbeat_back_addr":"172.21.15.160:6833/3265797851","heartbeat_front_addr":"172.21.15.160:6831/3265797851","state":["exists","up"]},{"osd":10,"uuid":"bdc9899d-d720-4429-ad53-4c388a4dbe75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6825","nonce":2867473412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6827","nonce":2867473412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6831","nonce":2867473412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6829","nonce":2867473412}]},"public_addr":"172.21.15.177:6825/2867473412","cluster_addr":"172.21.15.177:6827/2867473412","heartbeat_back_addr":"172.21.15.177:6831/2867473412","heartbeat_front_addr":"172.21.15.177:6829/2867473412","state":["exists","up"]},{"osd":11,"uuid":"cc9adb54-ef49-47c6-90e3-fc03bc0f5cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6825","nonce":4068724731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6827","nonce":4068724731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6831","nonce":4068724731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6829","nonce":4068724731}]},"public_addr":"172.21.15.178:6825/4068724731","cluster_addr":"172.21.15.178:6827/4068724731","heartbeat_back_addr":"172.21.15.178:6831/4068724731","heartbeat_front_addr":"172.21.15.178:6829/4068724731","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:33:44.141585+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:07.685585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:31.044386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:56.308925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:35:21.582097+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:35:45.983540+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:36:11.530242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:36:36.844681+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:02.303437+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:27.629867+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:53.289495+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:38:18.536169+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/3027450612":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/874709722":"2024-03-08T07:32:08.190837+0000","172.21.15.160:6800/1130675225":"2024-03-08T07:31:41.166858+0000","172.21.15.160:0/3857193980":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/3861968297":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6800/3859174315":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/3013534743":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6801/3859174315":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/1310937667":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6801/1130675225":"2024-03-08T07:31:41.166858+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T07:41:03.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.394+0000 7fa39df1d640 1 -- 172.21.15.160:0/2049381411 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa3840a1d80 msgr2=0x7fa3840a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:03.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.394+0000 7fa39df1d640 1 --2- 172.21.15.160:0/2049381411 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa3840a1d80 0x7fa3840a4240 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa388000c00 tx=0x7fa388005fd0 comp rx=0 tx=0).stop 2024-03-07T07:41:03.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.394+0000 7fa39df1d640 1 -- 172.21.15.160:0/2049381411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa398101770 msgr2=0x7fa398196de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:03.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.394+0000 7fa39df1d640 1 --2- 172.21.15.160:0/2049381411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa398101770 0x7fa398196de0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa38000e9f0 tx=0x7fa38000eeb0 comp rx=0 tx=0).stop 2024-03-07T07:41:03.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.394+0000 7fa39df1d640 1 -- 172.21.15.160:0/2049381411 shutdown_connections 2024-03-07T07:41:03.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.394+0000 7fa39df1d640 1 --2- 172.21.15.160:0/2049381411 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa3840a1d80 0x7fa3840a4240 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:03.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.394+0000 7fa39df1d640 1 --2- 172.21.15.160:0/2049381411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa398102130 0x7fa39819de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:03.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.394+0000 7fa39df1d640 1 --2- 172.21.15.160:0/2049381411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa398101770 0x7fa398196de0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:03.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.394+0000 7fa39df1d640 1 --2- 172.21.15.160:0/2049381411 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa398100580 0x7fa3981968a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:03.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.394+0000 7fa39df1d640 1 -- 172.21.15.160:0/2049381411 >> 172.21.15.160:0/2049381411 conn(0x7fa3980fbaf0 msgr2=0x7fa398104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:03.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.394+0000 7fa39df1d640 1 -- 172.21.15.160:0/2049381411 shutdown_connections 2024-03-07T07:41:03.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.394+0000 7fa39df1d640 1 -- 172.21.15.160:0/2049381411 wait complete. 2024-03-07T07:41:03.407 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-07T07:34:34.529424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-07T07:39:12.075432+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-07T07:39:13.367820+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-07T07:41:03.408 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-07T07:41:03.519 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.517+0000 7f21ab978640 1 -- 172.21.15.160:0/75217298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21a410dc40 msgr2=0x7f21a41144d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:03.519 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.517+0000 7f21ab978640 1 --2- 172.21.15.160:0/75217298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21a410dc40 0x7f21a41144d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f21a0009900 tx=0x7f21a002bc30 comp rx=0 tx=0).stop 2024-03-07T07:41:03.519 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.517+0000 7f21ab978640 1 -- 172.21.15.160:0/75217298 shutdown_connections 2024-03-07T07:41:03.519 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.517+0000 7f21ab978640 1 --2- 172.21.15.160:0/75217298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21a410dc40 0x7f21a41144d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:03.519 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.517+0000 7f21ab978640 1 --2- 172.21.15.160:0/75217298 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21a410d280 0x7f21a410d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:03.519 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.517+0000 7f21ab978640 1 --2- 172.21.15.160:0/75217298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21a410c090 0x7f21a410c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:03.519 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.517+0000 7f21ab978640 1 -- 172.21.15.160:0/75217298 >> 172.21.15.160:0/75217298 conn(0x7f21a406c040 msgr2=0x7f21a406c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:03.519 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.517+0000 7f21ab978640 1 -- 172.21.15.160:0/75217298 shutdown_connections 2024-03-07T07:41:03.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.517+0000 7f21ab978640 1 -- 172.21.15.160:0/75217298 wait complete. 2024-03-07T07:41:03.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.517+0000 7f21ab978640 1 Processor -- start 2024-03-07T07:41:03.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.517+0000 7f21ab978640 1 -- start start 2024-03-07T07:41:03.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.517+0000 7f21ab978640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21a410c090 0x7f21a41a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:03.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.517+0000 7f21ab978640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21a410d280 0x7f21a41a2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:03.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.517+0000 7f21ab978640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21a410dc40 0x7f21a41a98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:03.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.517+0000 7f21ab978640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a4117d20 con 0x7f21a410d280 2024-03-07T07:41:03.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.517+0000 7f21ab978640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a4117ba0 con 0x7f21a410dc40 2024-03-07T07:41:03.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.518+0000 7f21ab978640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a4117ea0 con 0x7f21a410c090 2024-03-07T07:41:03.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.518+0000 7f21a8f13640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21a410d280 0x7f21a41a2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:03.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.518+0000 7f21a8f13640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21a410d280 0x7f21a41a2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57794/0 (socket says 172.21.15.160:57794) 2024-03-07T07:41:03.521 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.518+0000 7f21a8f13640 1 -- 172.21.15.160:0/287497829 learned_addr learned my addr 172.21.15.160:0/287497829 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:41:03.521 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.518+0000 7f21a9f15640 1 --2- 172.21.15.160:0/287497829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21a410dc40 0x7f21a41a98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:03.521 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.518+0000 7f21a9714640 1 --2- 172.21.15.160:0/287497829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21a410c090 0x7f21a41a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:03.521 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.518+0000 7f21a9f15640 1 -- 172.21.15.160:0/287497829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21a410c090 msgr2=0x7f21a41a2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:03.521 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.518+0000 7f21a9f15640 1 --2- 172.21.15.160:0/287497829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21a410c090 0x7f21a41a2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:03.521 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.518+0000 7f21a9f15640 1 -- 172.21.15.160:0/287497829 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21a410d280 msgr2=0x7f21a41a2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:03.521 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.518+0000 7f21a9f15640 1 --2- 172.21.15.160:0/287497829 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21a410d280 0x7f21a41a2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:03.521 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.518+0000 7f21a9f15640 1 -- 172.21.15.160:0/287497829 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a41a9f30 con 0x7f21a410dc40 2024-03-07T07:41:03.521 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.518+0000 7f21a8f13640 1 --2- 172.21.15.160:0/287497829 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21a410d280 0x7f21a41a2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:41:03.521 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.518+0000 7f21a9f15640 1 --2- 172.21.15.160:0/287497829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21a410dc40 0x7f21a41a98c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f21a002c170 tx=0x7f21a002c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:03.521 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.518+0000 7f219a7fc640 1 -- 172.21.15.160:0/287497829 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a0041070 con 0x7f21a410dc40 2024-03-07T07:41:03.521 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.518+0000 7f219a7fc640 1 -- 172.21.15.160:0/287497829 <== mon.2 v2:172.21.15.177:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f21a0004450 con 0x7f21a410dc40 2024-03-07T07:41:03.521 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.518+0000 7f21ab978640 1 -- 172.21.15.160:0/287497829 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21a41aa1c0 con 0x7f21a410dc40 2024-03-07T07:41:03.521 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.518+0000 7f219a7fc640 1 -- 172.21.15.160:0/287497829 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a0034440 con 0x7f21a410dc40 2024-03-07T07:41:03.521 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.518+0000 7f21ab978640 1 -- 172.21.15.160:0/287497829 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21a41aa680 con 0x7f21a410dc40 2024-03-07T07:41:03.522 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.519+0000 7f21ab978640 1 -- 172.21.15.160:0/287497829 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21a4113350 con 0x7f21a410dc40 2024-03-07T07:41:03.522 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.520+0000 7f219a7fc640 1 -- 172.21.15.160:0/287497829 <== mon.2 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 18) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f21a0044070 con 0x7f21a410dc40 2024-03-07T07:41:03.522 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.521+0000 7f219a7fc640 1 --2- 172.21.15.160:0/287497829 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f21900a1d80 0x7f21900a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:03.523 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.521+0000 7f219a7fc640 1 -- 172.21.15.160:0/287497829 <== mon.2 v2:172.21.15.177:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f21a00ef8d0 con 0x7f21a410dc40 2024-03-07T07:41:03.523 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.521+0000 7f21a9714640 1 --2- 172.21.15.160:0/287497829 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f21900a1d80 0x7f21900a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:03.523 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.521+0000 7f21a9714640 1 --2- 172.21.15.160:0/287497829 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f21900a1d80 0x7f21900a4240 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f21a410d0e0 tx=0x7f219c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:03.525 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.525+0000 7f219a7fc640 1 -- 172.21.15.160:0/287497829 <== mon.2 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f21a00bc6b0 con 0x7f21a410dc40 2024-03-07T07:41:03.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:03 smithi160 ceph-mon[31963]: pgmap v344: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-07T07:41:03.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:03 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/1671649614' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-07T07:41:03.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:03 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/3941428160' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-07T07:41:03.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:03 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/756136389' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:41:03.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.708+0000 7f21ab978640 1 -- 172.21.15.160:0/287497829 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f21a402d030 con 0x7f21a410dc40 2024-03-07T07:41:03.709 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.709+0000 7f219a7fc640 1 -- 172.21.15.160:0/287497829 <== mon.2 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f21a00c0750 con 0x7f21a410dc40 2024-03-07T07:41:03.709 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 1 2024-03-07T07:41:03.711 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.711+0000 7f21ab978640 1 -- 172.21.15.160:0/287497829 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f21900a1d80 msgr2=0x7f21900a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:03.711 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.711+0000 7f21ab978640 1 --2- 172.21.15.160:0/287497829 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f21900a1d80 0x7f21900a4240 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f21a410d0e0 tx=0x7f219c009500 comp rx=0 tx=0).stop 2024-03-07T07:41:03.711 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.711+0000 7f21ab978640 1 -- 172.21.15.160:0/287497829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21a410dc40 msgr2=0x7f21a41a98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:03.711 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.711+0000 7f21ab978640 1 --2- 172.21.15.160:0/287497829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21a410dc40 0x7f21a41a98c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f21a002c170 tx=0x7f21a002c6c0 comp rx=0 tx=0).stop 2024-03-07T07:41:03.711 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.711+0000 7f21ab978640 1 -- 172.21.15.160:0/287497829 shutdown_connections 2024-03-07T07:41:03.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.711+0000 7f21ab978640 1 --2- 172.21.15.160:0/287497829 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f21900a1d80 0x7f21900a4240 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:03.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.711+0000 7f21ab978640 1 --2- 172.21.15.160:0/287497829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21a410dc40 0x7f21a41a98c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:03.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.711+0000 7f21ab978640 1 --2- 172.21.15.160:0/287497829 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21a410d280 0x7f21a41a2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:03.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.711+0000 7f21ab978640 1 --2- 172.21.15.160:0/287497829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21a410c090 0x7f21a41a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:03.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.711+0000 7f21ab978640 1 -- 172.21.15.160:0/287497829 >> 172.21.15.160:0/287497829 conn(0x7f21a406c040 msgr2=0x7f21a4110520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:03.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.711+0000 7f21ab978640 1 -- 172.21.15.160:0/287497829 shutdown_connections 2024-03-07T07:41:03.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.711+0000 7f21ab978640 1 -- 172.21.15.160:0/287497829 wait complete. 2024-03-07T07:41:03.723 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-07T07:41:03.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:03 smithi178 ceph-mon[36650]: pgmap v344: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-07T07:41:03.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:03 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/1671649614' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-07T07:41:03.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:03 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/3941428160' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-07T07:41:03.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:03 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/756136389' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:41:03.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.831+0000 7fe160307640 1 -- 172.21.15.160:0/3042166676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe158101750 msgr2=0x7fe158101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:03.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.831+0000 7fe160307640 1 --2- 172.21.15.160:0/3042166676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe158101750 0x7fe158101bd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe148009860 tx=0x7fe14802bb70 comp rx=0 tx=0).stop 2024-03-07T07:41:03.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.831+0000 7fe160307640 1 -- 172.21.15.160:0/3042166676 shutdown_connections 2024-03-07T07:41:03.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.831+0000 7fe160307640 1 --2- 172.21.15.160:0/3042166676 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe158102110 0x7fe1581089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:03.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.831+0000 7fe160307640 1 --2- 172.21.15.160:0/3042166676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe158101750 0x7fe158101bd0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:03.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.831+0000 7fe160307640 1 --2- 172.21.15.160:0/3042166676 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe158100560 0x7fe158100980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:03.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.831+0000 7fe160307640 1 -- 172.21.15.160:0/3042166676 >> 172.21.15.160:0/3042166676 conn(0x7fe1580fbaf0 msgr2=0x7fe1580fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:03.833 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.831+0000 7fe160307640 1 -- 172.21.15.160:0/3042166676 shutdown_connections 2024-03-07T07:41:03.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.831+0000 7fe160307640 1 -- 172.21.15.160:0/3042166676 wait complete. 2024-03-07T07:41:03.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.831+0000 7fe160307640 1 Processor -- start 2024-03-07T07:41:03.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.831+0000 7fe160307640 1 -- start start 2024-03-07T07:41:03.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.831+0000 7fe160307640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe158100560 0x7fe1581968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:03.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.831+0000 7fe160307640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe158101750 0x7fe158196df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:03.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.831+0000 7fe160307640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe158102110 0x7fe15819de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:03.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.831+0000 7fe160307640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15810c1f0 con 0x7fe158102110 2024-03-07T07:41:03.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.831+0000 7fe160307640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15810c070 con 0x7fe158101750 2024-03-07T07:41:03.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.831+0000 7fe160307640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15810c370 con 0x7fe158100560 2024-03-07T07:41:03.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.832+0000 7fe15e8a4640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe158102110 0x7fe15819de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:03.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.832+0000 7fe15e0a3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe158100560 0x7fe1581968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:03.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.832+0000 7fe15e8a4640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe158102110 0x7fe15819de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57802/0 (socket says 172.21.15.160:57802) 2024-03-07T07:41:03.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.832+0000 7fe15d8a2640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe158101750 0x7fe158196df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:03.834 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.832+0000 7fe15e8a4640 1 -- 172.21.15.160:0/168179578 learned_addr learned my addr 172.21.15.160:0/168179578 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:41:03.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.832+0000 7fe15e0a3640 1 -- 172.21.15.160:0/168179578 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe158101750 msgr2=0x7fe158196df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:03.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.832+0000 7fe15e0a3640 1 --2- 172.21.15.160:0/168179578 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe158101750 0x7fe158196df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:03.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.832+0000 7fe15e0a3640 1 -- 172.21.15.160:0/168179578 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe158102110 msgr2=0x7fe15819de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:03.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.832+0000 7fe15e0a3640 1 --2- 172.21.15.160:0/168179578 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe158102110 0x7fe15819de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:03.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.832+0000 7fe15e0a3640 1 -- 172.21.15.160:0/168179578 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe15819e4d0 con 0x7fe158100560 2024-03-07T07:41:03.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.832+0000 7fe15e8a4640 1 --2- 172.21.15.160:0/168179578 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe158102110 0x7fe15819de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:41:03.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.832+0000 7fe15d8a2640 1 --2- 172.21.15.160:0/168179578 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe158101750 0x7fe158196df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:41:03.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.832+0000 7fe15e0a3640 1 --2- 172.21.15.160:0/168179578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe158100560 0x7fe1581968b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe15000ead0 tx=0x7fe15000ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:03.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.832+0000 7fe14f7fe640 1 -- 172.21.15.160:0/168179578 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15001c070 con 0x7fe158100560 2024-03-07T07:41:03.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.832+0000 7fe14f7fe640 1 -- 172.21.15.160:0/168179578 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe15000bba0 con 0x7fe158100560 2024-03-07T07:41:03.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.832+0000 7fe160307640 1 -- 172.21.15.160:0/168179578 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe158068820 con 0x7fe158100560 2024-03-07T07:41:03.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.832+0000 7fe14f7fe640 1 -- 172.21.15.160:0/168179578 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1500165d0 con 0x7fe158100560 2024-03-07T07:41:03.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.832+0000 7fe160307640 1 -- 172.21.15.160:0/168179578 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe158068d40 con 0x7fe158100560 2024-03-07T07:41:03.836 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.833+0000 7fe160307640 1 -- 172.21.15.160:0/168179578 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe158068450 con 0x7fe158100560 2024-03-07T07:41:03.836 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.834+0000 7fe14f7fe640 1 -- 172.21.15.160:0/168179578 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136754+0+0 (secure 0 0 0) 0x7fe15001f070 con 0x7fe158100560 2024-03-07T07:41:03.836 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.835+0000 7fe14f7fe640 1 --2- 172.21.15.160:0/168179578 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fe1440a1d80 0x7fe1440a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:03.836 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.835+0000 7fe15d8a2640 1 --2- 172.21.15.160:0/168179578 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fe1440a1d80 0x7fe1440a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:03.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.835+0000 7fe14f7fe640 1 -- 172.21.15.160:0/168179578 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe1500cf1f0 con 0x7fe158100560 2024-03-07T07:41:03.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.835+0000 7fe15d8a2640 1 --2- 172.21.15.160:0/168179578 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fe1440a1d80 0x7fe1440a4240 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe14802c180 tx=0x7fe14802e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:03.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:03.838+0000 7fe14f7fe640 1 -- 172.21.15.160:0/168179578 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fe15009bfd0 con 0x7fe158100560 2024-03-07T07:41:03.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:03 smithi177 ceph-mon[36578]: pgmap v344: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-07T07:41:03.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:03 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/1671649614' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-07T07:41:03.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:03 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/3941428160' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-07T07:41:03.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:03 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/756136389' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:41:04.022 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.021+0000 7fe160307640 1 -- 172.21.15.160:0/168179578 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe158107390 con 0x7fe158100560 2024-03-07T07:41:04.022 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.022+0000 7fe14f7fe640 1 -- 172.21.15.160:0/168179578 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe1500a0070 con 0x7fe158100560 2024-03-07T07:41:04.022 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 32 2024-03-07T07:41:04.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.024+0000 7fe160307640 1 -- 172.21.15.160:0/168179578 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fe1440a1d80 msgr2=0x7fe1440a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:04.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.024+0000 7fe160307640 1 --2- 172.21.15.160:0/168179578 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fe1440a1d80 0x7fe1440a4240 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe14802c180 tx=0x7fe14802e040 comp rx=0 tx=0).stop 2024-03-07T07:41:04.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.024+0000 7fe160307640 1 -- 172.21.15.160:0/168179578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe158100560 msgr2=0x7fe1581968b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:04.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.024+0000 7fe160307640 1 --2- 172.21.15.160:0/168179578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe158100560 0x7fe1581968b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe15000ead0 tx=0x7fe15000ef90 comp rx=0 tx=0).stop 2024-03-07T07:41:04.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.024+0000 7fe160307640 1 -- 172.21.15.160:0/168179578 shutdown_connections 2024-03-07T07:41:04.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.024+0000 7fe160307640 1 --2- 172.21.15.160:0/168179578 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fe1440a1d80 0x7fe1440a4240 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:04.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.024+0000 7fe160307640 1 --2- 172.21.15.160:0/168179578 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe158102110 0x7fe15819de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:04.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.024+0000 7fe160307640 1 --2- 172.21.15.160:0/168179578 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe158101750 0x7fe158196df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:04.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.024+0000 7fe160307640 1 --2- 172.21.15.160:0/168179578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe158100560 0x7fe1581968b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:04.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.024+0000 7fe160307640 1 -- 172.21.15.160:0/168179578 >> 172.21.15.160:0/168179578 conn(0x7fe1580fbaf0 msgr2=0x7fe158104af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:04.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.024+0000 7fe160307640 1 -- 172.21.15.160:0/168179578 shutdown_connections 2024-03-07T07:41:04.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.024+0000 7fe160307640 1 -- 172.21.15.160:0/168179578 wait complete. 2024-03-07T07:41:04.035 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-07T07:41:04.144 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.142+0000 7f2e3ecdd640 1 -- 172.21.15.160:0/1560171518 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e381073e0 msgr2=0x7f2e381097d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:04.144 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.142+0000 7f2e3ecdd640 1 --2- 172.21.15.160:0/1560171518 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e381073e0 0x7f2e381097d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2e30009be0 tx=0x7f2e3002bef0 comp rx=0 tx=0).stop 2024-03-07T07:41:04.144 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.142+0000 7f2e3ecdd640 1 -- 172.21.15.160:0/1560171518 shutdown_connections 2024-03-07T07:41:04.144 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.142+0000 7f2e3ecdd640 1 --2- 172.21.15.160:0/1560171518 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e381073e0 0x7f2e381097d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:04.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.142+0000 7f2e3ecdd640 1 --2- 172.21.15.160:0/1560171518 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e38104ab0 0x7f2e38106ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:04.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.142+0000 7f2e3ecdd640 1 --2- 172.21.15.160:0/1560171518 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e38102180 0x7f2e38104570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:04.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.142+0000 7f2e3ecdd640 1 -- 172.21.15.160:0/1560171518 >> 172.21.15.160:0/1560171518 conn(0x7f2e380fbaf0 msgr2=0x7f2e380fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:04.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.142+0000 7f2e3ecdd640 1 -- 172.21.15.160:0/1560171518 shutdown_connections 2024-03-07T07:41:04.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.142+0000 7f2e3ecdd640 1 -- 172.21.15.160:0/1560171518 wait complete. 2024-03-07T07:41:04.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.143+0000 7f2e3ecdd640 1 Processor -- start 2024-03-07T07:41:04.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.143+0000 7f2e3ecdd640 1 -- start start 2024-03-07T07:41:04.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.143+0000 7f2e3ecdd640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e38102180 0x7f2e38197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:04.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.143+0000 7f2e3ecdd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e38104ab0 0x7f2e38197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:04.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.143+0000 7f2e3ecdd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e381073e0 0x7f2e3819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:04.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.143+0000 7f2e3ecdd640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3810d2f0 con 0x7f2e38104ab0 2024-03-07T07:41:04.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.143+0000 7f2e3ecdd640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3810d170 con 0x7f2e38102180 2024-03-07T07:41:04.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.143+0000 7f2e3ecdd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3810d470 con 0x7f2e381073e0 2024-03-07T07:41:04.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.143+0000 7f2e37fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e38104ab0 0x7f2e38197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:04.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.143+0000 7f2e37fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e38104ab0 0x7f2e38197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57826/0 (socket says 172.21.15.160:57826) 2024-03-07T07:41:04.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.143+0000 7f2e37fff640 1 -- 172.21.15.160:0/2204807383 learned_addr learned my addr 172.21.15.160:0/2204807383 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:41:04.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.143+0000 7f2e3ca79640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e38102180 0x7f2e38197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:04.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.143+0000 7f2e3d27a640 1 --2- 172.21.15.160:0/2204807383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e381073e0 0x7f2e3819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:04.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.143+0000 7f2e37fff640 1 -- 172.21.15.160:0/2204807383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e381073e0 msgr2=0x7f2e3819ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:04.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.143+0000 7f2e37fff640 1 --2- 172.21.15.160:0/2204807383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e381073e0 0x7f2e3819ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:04.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.143+0000 7f2e37fff640 1 -- 172.21.15.160:0/2204807383 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e38102180 msgr2=0x7f2e38197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:04.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.143+0000 7f2e37fff640 1 --2- 172.21.15.160:0/2204807383 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e38102180 0x7f2e38197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:04.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.143+0000 7f2e37fff640 1 -- 172.21.15.160:0/2204807383 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e3819f590 con 0x7f2e38104ab0 2024-03-07T07:41:04.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.143+0000 7f2e3ca79640 1 --2- 172.21.15.160:0/2204807383 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e38102180 0x7f2e38197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:41:04.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.143+0000 7f2e37fff640 1 --2- 172.21.15.160:0/2204807383 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e38104ab0 0x7f2e38197eb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2e2400e9f0 tx=0x7f2e2400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:04.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.144+0000 7f2e35ffb640 1 -- 172.21.15.160:0/2204807383 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2400ccc0 con 0x7f2e38104ab0 2024-03-07T07:41:04.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.144+0000 7f2e3ecdd640 1 -- 172.21.15.160:0/2204807383 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e3819f880 con 0x7f2e38104ab0 2024-03-07T07:41:04.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.144+0000 7f2e35ffb640 1 -- 172.21.15.160:0/2204807383 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2e24005500 con 0x7f2e38104ab0 2024-03-07T07:41:04.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.144+0000 7f2e35ffb640 1 -- 172.21.15.160:0/2204807383 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e24010690 con 0x7f2e38104ab0 2024-03-07T07:41:04.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.144+0000 7f2e3ecdd640 1 -- 172.21.15.160:0/2204807383 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e3819fdd0 con 0x7f2e38104ab0 2024-03-07T07:41:04.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.145+0000 7f2e3ecdd640 1 -- 172.21.15.160:0/2204807383 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e380fd5e0 con 0x7f2e38104ab0 2024-03-07T07:41:04.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.146+0000 7f2e35ffb640 1 -- 172.21.15.160:0/2204807383 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f2e240046c0 con 0x7f2e38104ab0 2024-03-07T07:41:04.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.146+0000 7f2e35ffb640 1 --2- 172.21.15.160:0/2204807383 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2e280a1d80 0x7f2e280a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:04.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.146+0000 7f2e3ca79640 1 --2- 172.21.15.160:0/2204807383 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2e280a1d80 0x7f2e280a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:04.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.146+0000 7f2e35ffb640 1 -- 172.21.15.160:0/2204807383 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2e240ce130 con 0x7f2e38104ab0 2024-03-07T07:41:04.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.147+0000 7f2e3ca79640 1 --2- 172.21.15.160:0/2204807383 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2e280a1d80 0x7f2e280a4240 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c0096c0 tx=0x7f2e2c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:04.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.151+0000 7f2e35ffb640 1 -- 172.21.15.160:0/2204807383 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f2e2409b560 con 0x7f2e38104ab0 2024-03-07T07:41:04.330 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.330+0000 7f2e3ecdd640 1 -- 172.21.15.160:0/2204807383 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2e38062260 con 0x7f2e38104ab0 2024-03-07T07:41:04.330 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.330+0000 7f2e35ffb640 1 -- 172.21.15.160:0/2204807383 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2e2409ae60 con 0x7f2e38104ab0 2024-03-07T07:41:04.331 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 64 2024-03-07T07:41:04.333 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.332+0000 7f2e3ecdd640 1 -- 172.21.15.160:0/2204807383 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2e280a1d80 msgr2=0x7f2e280a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:04.333 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.332+0000 7f2e3ecdd640 1 --2- 172.21.15.160:0/2204807383 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2e280a1d80 0x7f2e280a4240 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c0096c0 tx=0x7f2e2c009260 comp rx=0 tx=0).stop 2024-03-07T07:41:04.333 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.332+0000 7f2e3ecdd640 1 -- 172.21.15.160:0/2204807383 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e38104ab0 msgr2=0x7f2e38197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:04.333 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.332+0000 7f2e3ecdd640 1 --2- 172.21.15.160:0/2204807383 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e38104ab0 0x7f2e38197eb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2e2400e9f0 tx=0x7f2e2400eeb0 comp rx=0 tx=0).stop 2024-03-07T07:41:04.333 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.332+0000 7f2e3ecdd640 1 -- 172.21.15.160:0/2204807383 shutdown_connections 2024-03-07T07:41:04.333 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.332+0000 7f2e3ecdd640 1 --2- 172.21.15.160:0/2204807383 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2e280a1d80 0x7f2e280a4240 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:04.333 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.332+0000 7f2e3ecdd640 1 --2- 172.21.15.160:0/2204807383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e381073e0 0x7f2e3819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:04.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.332+0000 7f2e3ecdd640 1 --2- 172.21.15.160:0/2204807383 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e38104ab0 0x7f2e38197eb0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:04.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.332+0000 7f2e3ecdd640 1 --2- 172.21.15.160:0/2204807383 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e38102180 0x7f2e38197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:04.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.333+0000 7f2e3ecdd640 1 -- 172.21.15.160:0/2204807383 >> 172.21.15.160:0/2204807383 conn(0x7f2e380fbaf0 msgr2=0x7f2e38102ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:04.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.333+0000 7f2e3ecdd640 1 -- 172.21.15.160:0/2204807383 shutdown_connections 2024-03-07T07:41:04.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.333+0000 7f2e3ecdd640 1 -- 172.21.15.160:0/2204807383 wait complete. 2024-03-07T07:41:04.344 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T07:41:04.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.447+0000 7f2bf2f5a640 1 -- 172.21.15.160:0/3565137031 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2bec107440 msgr2=0x7f2bec109830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:04.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.447+0000 7f2bf2f5a640 1 --2- 172.21.15.160:0/3565137031 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2bec107440 0x7f2bec109830 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2be4009ca0 tx=0x7f2be402c0f0 comp rx=0 tx=0).stop 2024-03-07T07:41:04.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.447+0000 7f2bf2f5a640 1 -- 172.21.15.160:0/3565137031 shutdown_connections 2024-03-07T07:41:04.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.447+0000 7f2bf2f5a640 1 --2- 172.21.15.160:0/3565137031 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2bec107440 0x7f2bec109830 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:04.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.447+0000 7f2bf2f5a640 1 --2- 172.21.15.160:0/3565137031 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bec104b10 0x7f2bec106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:04.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.447+0000 7f2bf2f5a640 1 --2- 172.21.15.160:0/3565137031 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bec1021e0 0x7f2bec1045d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:04.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.447+0000 7f2bf2f5a640 1 -- 172.21.15.160:0/3565137031 >> 172.21.15.160:0/3565137031 conn(0x7f2bec0fbb90 msgr2=0x7f2bec0fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:04.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.447+0000 7f2bf2f5a640 1 -- 172.21.15.160:0/3565137031 shutdown_connections 2024-03-07T07:41:04.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.447+0000 7f2bf2f5a640 1 -- 172.21.15.160:0/3565137031 wait complete. 2024-03-07T07:41:04.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.448+0000 7f2bf2f5a640 1 Processor -- start 2024-03-07T07:41:04.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.448+0000 7f2bf2f5a640 1 -- start start 2024-03-07T07:41:04.451 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.448+0000 7f2bf2f5a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bec1021e0 0x7f2bec197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:04.451 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.448+0000 7f2bf2f5a640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bec104b10 0x7f2bec197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:04.451 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.448+0000 7f2bf2f5a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2bec107440 0x7f2bec19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:04.451 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.448+0000 7f2bf2f5a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec10d330 con 0x7f2bec107440 2024-03-07T07:41:04.451 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.448+0000 7f2bf2f5a640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec10d1b0 con 0x7f2bec104b10 2024-03-07T07:41:04.451 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.448+0000 7f2bf2f5a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec10d4b0 con 0x7f2bec1021e0 2024-03-07T07:41:04.451 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.448+0000 7f2bf14f7640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2bec107440 0x7f2bec19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:04.451 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.448+0000 7f2bf14f7640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2bec107440 0x7f2bec19ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57838/0 (socket says 172.21.15.160:57838) 2024-03-07T07:41:04.451 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.448+0000 7f2bf14f7640 1 -- 172.21.15.160:0/4200001009 learned_addr learned my addr 172.21.15.160:0/4200001009 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:41:04.451 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.448+0000 7f2bebfff640 1 --2- 172.21.15.160:0/4200001009 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bec104b10 0x7f2bec197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:04.451 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.448+0000 7f2bf0cf6640 1 --2- 172.21.15.160:0/4200001009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bec1021e0 0x7f2bec197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:04.451 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.448+0000 7f2bf14f7640 1 -- 172.21.15.160:0/4200001009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bec1021e0 msgr2=0x7f2bec197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:04.451 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.448+0000 7f2bf14f7640 1 --2- 172.21.15.160:0/4200001009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bec1021e0 0x7f2bec197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:04.451 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.448+0000 7f2bf14f7640 1 -- 172.21.15.160:0/4200001009 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bec104b10 msgr2=0x7f2bec197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:04.452 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.448+0000 7f2bf14f7640 1 --2- 172.21.15.160:0/4200001009 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bec104b10 0x7f2bec197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:04.452 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.448+0000 7f2bf14f7640 1 -- 172.21.15.160:0/4200001009 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bec19f590 con 0x7f2bec107440 2024-03-07T07:41:04.452 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.448+0000 7f2bf14f7640 1 --2- 172.21.15.160:0/4200001009 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2bec107440 0x7f2bec19ef20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2be4037040 tx=0x7f2be4032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:04.452 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.448+0000 7f2be9ffb640 1 -- 172.21.15.160:0/4200001009 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be4041020 con 0x7f2bec107440 2024-03-07T07:41:04.452 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.448+0000 7f2be9ffb640 1 -- 172.21.15.160:0/4200001009 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2be4034cd0 con 0x7f2bec107440 2024-03-07T07:41:04.452 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.448+0000 7f2bf2f5a640 1 -- 172.21.15.160:0/4200001009 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bec19f820 con 0x7f2bec107440 2024-03-07T07:41:04.452 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.448+0000 7f2be9ffb640 1 -- 172.21.15.160:0/4200001009 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be4039040 con 0x7f2bec107440 2024-03-07T07:41:04.452 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.449+0000 7f2bf2f5a640 1 -- 172.21.15.160:0/4200001009 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bec19fd10 con 0x7f2bec107440 2024-03-07T07:41:04.452 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.450+0000 7f2bf2f5a640 1 -- 172.21.15.160:0/4200001009 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bec0fd680 con 0x7f2bec107440 2024-03-07T07:41:04.453 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.451+0000 7f2be9ffb640 1 -- 172.21.15.160:0/4200001009 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f2be4044020 con 0x7f2bec107440 2024-03-07T07:41:04.453 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.451+0000 7f2be9ffb640 1 --2- 172.21.15.160:0/4200001009 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2bdc0a1d80 0x7f2bdc0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:04.453 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.451+0000 7f2bf0cf6640 1 --2- 172.21.15.160:0/4200001009 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2bdc0a1d80 0x7f2bdc0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:04.453 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.451+0000 7f2be9ffb640 1 -- 172.21.15.160:0/4200001009 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2be40effd0 con 0x7f2bec107440 2024-03-07T07:41:04.453 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.452+0000 7f2bf0cf6640 1 --2- 172.21.15.160:0/4200001009 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2bdc0a1d80 0x7f2bdc0a4240 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2be0009bd0 tx=0x7f2be0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:04.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.455+0000 7f2be9ffb640 1 -- 172.21.15.160:0/4200001009 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f2be404b020 con 0x7f2bec107440 2024-03-07T07:41:04.667 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.667+0000 7f2bf2f5a640 1 -- 172.21.15.160:0/4200001009 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2bec02d030 con 0x7f2bec107440 2024-03-07T07:41:04.667 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.667+0000 7f2be9ffb640 1 -- 172.21.15.160:0/4200001009 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+11411 (secure 0 0 0) 0x7f2be40bd490 con 0x7f2bec107440 2024-03-07T07:41:04.668 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:41:04.668 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/2615961083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2615961083},{"type":"v1","addr":"172.21.15.160:6835","nonce":2615961083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1759006033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1759006033},{"type":"v1","addr":"172.21.15.160:6837","nonce":1759006033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6839/3763944721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":3763944721},{"type":"v1","addr":"172.21.15.160:6839","nonce":3763944721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6841/3611426637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6840","nonce":3611426637},{"type":"v1","addr":"172.21.15.160:6841","nonce":3611426637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1457364761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1457364761},{"type":"v1","addr":"172.21.15.177:6833","nonce":1457364761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/170428292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":170428292},{"type":"v1","addr":"172.21.15.177:6835","nonce":170428292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-07T07:39:14.164243+0000","modified":"2024-03-07T07:39:18.861903+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24511,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/645365846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":645365846},{"type":"v1","addr":"172.21.15.178:6833","nonce":645365846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6837/4088036516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":4088036516},{"type":"v1","addr":"172.21.15.177:6837","nonce":4088036516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/647840547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":647840547},{"type":"v1","addr":"172.21.15.178:6835","nonce":647840547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.178:6837/1247829154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1247829154},{"type":"v1","addr":"172.21.15.178:6837","nonce":1247829154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24497":{"gid":24497,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/3329773838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3329773838},{"type":"v1","addr":"172.21.15.177:6839","nonce":3329773838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24511":{"gid":24511,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.178:6839/2947354024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":2947354024},{"type":"v1","addr":"172.21.15.178:6839","nonce":2947354024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24511,"qdb_cluster":[24487,24511,24497]},"id":1}]} 2024-03-07T07:41:04.668 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 21 2024-03-07T07:41:04.670 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.669+0000 7f2bf2f5a640 1 -- 172.21.15.160:0/4200001009 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2bdc0a1d80 msgr2=0x7f2bdc0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:04.670 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.669+0000 7f2bf2f5a640 1 --2- 172.21.15.160:0/4200001009 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2bdc0a1d80 0x7f2bdc0a4240 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2be0009bd0 tx=0x7f2be0005f90 comp rx=0 tx=0).stop 2024-03-07T07:41:04.670 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.669+0000 7f2bf2f5a640 1 -- 172.21.15.160:0/4200001009 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2bec107440 msgr2=0x7f2bec19ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:04.670 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.669+0000 7f2bf2f5a640 1 --2- 172.21.15.160:0/4200001009 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2bec107440 0x7f2bec19ef20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2be4037040 tx=0x7f2be4032f40 comp rx=0 tx=0).stop 2024-03-07T07:41:04.670 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.670+0000 7f2bf2f5a640 1 -- 172.21.15.160:0/4200001009 shutdown_connections 2024-03-07T07:41:04.670 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.670+0000 7f2bf2f5a640 1 --2- 172.21.15.160:0/4200001009 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2bdc0a1d80 0x7f2bdc0a4240 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:04.670 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.670+0000 7f2bf2f5a640 1 --2- 172.21.15.160:0/4200001009 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2bec107440 0x7f2bec19ef20 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:04.670 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.670+0000 7f2bf2f5a640 1 --2- 172.21.15.160:0/4200001009 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bec104b10 0x7f2bec197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:04.671 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.670+0000 7f2bf2f5a640 1 --2- 172.21.15.160:0/4200001009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bec1021e0 0x7f2bec197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:04.671 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.670+0000 7f2bf2f5a640 1 -- 172.21.15.160:0/4200001009 >> 172.21.15.160:0/4200001009 conn(0x7f2bec0fbb90 msgr2=0x7f2bec102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:04.671 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.670+0000 7f2bf2f5a640 1 -- 172.21.15.160:0/4200001009 shutdown_connections 2024-03-07T07:41:04.671 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.670+0000 7f2bf2f5a640 1 -- 172.21.15.160:0/4200001009 wait complete. 2024-03-07T07:41:04.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:04 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/2049381411' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:41:04.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:04 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/287497829' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-07T07:41:04.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:04 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/168179578' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-07T07:41:04.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:04 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/2204807383' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-07T07:41:04.681 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T07:41:04.785 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.783+0000 7f2f41378640 1 -- 172.21.15.160:0/3094695341 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f3c100590 msgr2=0x7f2f3c1009b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:04.785 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.783+0000 7f2f41378640 1 --2- 172.21.15.160:0/3094695341 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f3c100590 0x7f2f3c1009b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2f30009900 tx=0x7f2f3002bc30 comp rx=0 tx=0).stop 2024-03-07T07:41:04.785 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.783+0000 7f2f41378640 1 -- 172.21.15.160:0/3094695341 shutdown_connections 2024-03-07T07:41:04.785 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.783+0000 7f2f41378640 1 --2- 172.21.15.160:0/3094695341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f3c102140 0x7f2f3c1089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:04.785 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.783+0000 7f2f41378640 1 --2- 172.21.15.160:0/3094695341 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2f3c101780 0x7f2f3c101c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:04.785 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.783+0000 7f2f41378640 1 --2- 172.21.15.160:0/3094695341 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f3c100590 0x7f2f3c1009b0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:04.786 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.783+0000 7f2f41378640 1 -- 172.21.15.160:0/3094695341 >> 172.21.15.160:0/3094695341 conn(0x7f2f3c0fbb40 msgr2=0x7f2f3c0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:04.786 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.783+0000 7f2f41378640 1 -- 172.21.15.160:0/3094695341 shutdown_connections 2024-03-07T07:41:04.786 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.783+0000 7f2f41378640 1 -- 172.21.15.160:0/3094695341 wait complete. 2024-03-07T07:41:04.786 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.783+0000 7f2f41378640 1 Processor -- start 2024-03-07T07:41:04.786 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.783+0000 7f2f41378640 1 -- start start 2024-03-07T07:41:04.786 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.784+0000 7f2f41378640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2f3c100590 0x7f2f3c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:04.786 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.784+0000 7f2f41378640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f3c101780 0x7f2f3c19c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:04.786 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.784+0000 7f2f41378640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f3c102140 0x7f2f3c19ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:04.786 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.784+0000 7f2f41378640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c10c220 con 0x7f2f3c101780 2024-03-07T07:41:04.786 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.784+0000 7f2f41378640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c10c0a0 con 0x7f2f3c100590 2024-03-07T07:41:04.786 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.784+0000 7f2f41378640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c10c3a0 con 0x7f2f3c102140 2024-03-07T07:41:04.786 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.784+0000 7f2f3a7fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f3c101780 0x7f2f3c19c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:04.786 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.784+0000 7f2f3a7fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f3c101780 0x7f2f3c19c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57852/0 (socket says 172.21.15.160:57852) 2024-03-07T07:41:04.786 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.784+0000 7f2f3a7fc640 1 -- 172.21.15.160:0/353281160 learned_addr learned my addr 172.21.15.160:0/353281160 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:41:04.787 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.784+0000 7f2f3affd640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2f3c100590 0x7f2f3c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:04.787 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.784+0000 7f2f3b7fe640 1 --2- 172.21.15.160:0/353281160 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f3c102140 0x7f2f3c19ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:04.787 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.784+0000 7f2f3a7fc640 1 -- 172.21.15.160:0/353281160 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f3c102140 msgr2=0x7f2f3c19ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:04.787 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.784+0000 7f2f3a7fc640 1 --2- 172.21.15.160:0/353281160 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f3c102140 0x7f2f3c19ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:04.787 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.784+0000 7f2f3a7fc640 1 -- 172.21.15.160:0/353281160 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2f3c100590 msgr2=0x7f2f3c06a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:04.787 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.784+0000 7f2f3a7fc640 1 --2- 172.21.15.160:0/353281160 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2f3c100590 0x7f2f3c06a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:04.787 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.784+0000 7f2f3a7fc640 1 -- 172.21.15.160:0/353281160 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f3c19f610 con 0x7f2f3c101780 2024-03-07T07:41:04.787 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.784+0000 7f2f3a7fc640 1 --2- 172.21.15.160:0/353281160 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f3c101780 0x7f2f3c19c6e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00ea60 tx=0x7f2f2c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:04.787 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.784+0000 7f2f23fff640 1 -- 172.21.15.160:0/353281160 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c00cde0 con 0x7f2f3c101780 2024-03-07T07:41:04.787 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.784+0000 7f2f23fff640 1 -- 172.21.15.160:0/353281160 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2f2c0035f0 con 0x7f2f3c101780 2024-03-07T07:41:04.787 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.784+0000 7f2f41378640 1 -- 172.21.15.160:0/353281160 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f3c19f8a0 con 0x7f2f3c101780 2024-03-07T07:41:04.787 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.784+0000 7f2f23fff640 1 -- 172.21.15.160:0/353281160 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c010670 con 0x7f2f3c101780 2024-03-07T07:41:04.787 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.784+0000 7f2f41378640 1 -- 172.21.15.160:0/353281160 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f3c19fdf0 con 0x7f2f3c101780 2024-03-07T07:41:04.788 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.785+0000 7f2f41378640 1 -- 172.21.15.160:0/353281160 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f3c107930 con 0x7f2f3c101780 2024-03-07T07:41:04.788 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.786+0000 7f2f23fff640 1 -- 172.21.15.160:0/353281160 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f2f2c0037a0 con 0x7f2f3c101780 2024-03-07T07:41:04.789 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.787+0000 7f2f23fff640 1 --2- 172.21.15.160:0/353281160 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2f240a1d80 0x7f2f240a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:04.789 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.787+0000 7f2f3affd640 1 --2- 172.21.15.160:0/353281160 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2f240a1d80 0x7f2f240a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:04.789 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.787+0000 7f2f23fff640 1 -- 172.21.15.160:0/353281160 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2f2c0cdfc0 con 0x7f2f3c101780 2024-03-07T07:41:04.789 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.787+0000 7f2f3affd640 1 --2- 172.21.15.160:0/353281160 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2f240a1d80 0x7f2f240a4240 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2f30000c00 tx=0x7f2f30005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:04.791 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.791+0000 7f2f23fff640 1 -- 172.21.15.160:0/353281160 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f2f2c09b490 con 0x7f2f3c101780 2024-03-07T07:41:04.805 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:04 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/2049381411' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:41:04.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:04 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/287497829' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-07T07:41:04.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:04 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/168179578' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-07T07:41:04.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:04 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/2204807383' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-07T07:41:04.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:04 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/2049381411' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:41:04.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:04 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/287497829' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-07T07:41:04.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:04 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/168179578' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-07T07:41:04.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:04 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/2204807383' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-07T07:41:04.968 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.968+0000 7f2f41378640 1 -- 172.21.15.160:0/353281160 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f3c062260 con 0x7f2f3c101780 2024-03-07T07:41:04.969 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.969+0000 7f2f23fff640 1 -- 172.21.15.160:0/353281160 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22721 (secure 0 0 0) 0x7f2f2c09ad90 con 0x7f2f3c101780 2024-03-07T07:41:04.970 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:41:04.970 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":87,"fsid":"826f6582-dc54-11ee-95c6-87774f69a715","created":"2024-03-07T07:31:14.229288+0000","modified":"2024-03-07T07:40:09.354265+0000","last_up_change":"2024-03-07T07:38:20.776225+0000","last_in_change":"2024-03-07T07:38:00.508917+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T07:34:34.529424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T07:39:12.075432+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T07:39:13.367820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bee66c67-fb6e-4717-9685-ca62587d5f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6803","nonce":1368482926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6805","nonce":1368482926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6809","nonce":1368482926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6807","nonce":1368482926}]},"public_addr":"172.21.15.160:6803/1368482926","cluster_addr":"172.21.15.160:6805/1368482926","heartbeat_back_addr":"172.21.15.160:6809/1368482926","heartbeat_front_addr":"172.21.15.160:6807/1368482926","state":["exists","up"]},{"osd":1,"uuid":"49236251-f750-424e-992b-bddf55762556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6801","nonce":2922391884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6803","nonce":2922391884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6807","nonce":2922391884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6805","nonce":2922391884}]},"public_addr":"172.21.15.177:6801/2922391884","cluster_addr":"172.21.15.177:6803/2922391884","heartbeat_back_addr":"172.21.15.177:6807/2922391884","heartbeat_front_addr":"172.21.15.177:6805/2922391884","state":["exists","up"]},{"osd":2,"uuid":"7a432b28-f7ee-43fe-9196-7aaeccfecf77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6801","nonce":1217901551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6803","nonce":1217901551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6807","nonce":1217901551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6805","nonce":1217901551}]},"public_addr":"172.21.15.178:6801/1217901551","cluster_addr":"172.21.15.178:6803/1217901551","heartbeat_back_addr":"172.21.15.178:6807/1217901551","heartbeat_front_addr":"172.21.15.178:6805/1217901551","state":["exists","up"]},{"osd":3,"uuid":"4ad22a20-f1be-4002-a48d-ecd3953e63f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6811","nonce":42543989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6813","nonce":42543989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6817","nonce":42543989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6815","nonce":42543989}]},"public_addr":"172.21.15.160:6811/42543989","cluster_addr":"172.21.15.160:6813/42543989","heartbeat_back_addr":"172.21.15.160:6817/42543989","heartbeat_front_addr":"172.21.15.160:6815/42543989","state":["exists","up"]},{"osd":4,"uuid":"2b65b104-ac82-4ce3-b6c5-11a399103045","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6809","nonce":88178553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6811","nonce":88178553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6815","nonce":88178553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6813","nonce":88178553}]},"public_addr":"172.21.15.177:6809/88178553","cluster_addr":"172.21.15.177:6811/88178553","heartbeat_back_addr":"172.21.15.177:6815/88178553","heartbeat_front_addr":"172.21.15.177:6813/88178553","state":["exists","up"]},{"osd":5,"uuid":"30a1e714-08ff-44e8-9264-9ef164604b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6809","nonce":964732891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6811","nonce":964732891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6815","nonce":964732891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6813","nonce":964732891}]},"public_addr":"172.21.15.178:6809/964732891","cluster_addr":"172.21.15.178:6811/964732891","heartbeat_back_addr":"172.21.15.178:6815/964732891","heartbeat_front_addr":"172.21.15.178:6813/964732891","state":["exists","up"]},{"osd":6,"uuid":"b1e1646e-f8a9-42b8-b62d-9d7b5d6a1ff4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6819","nonce":3251327001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6821","nonce":3251327001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6825","nonce":3251327001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6823","nonce":3251327001}]},"public_addr":"172.21.15.160:6819/3251327001","cluster_addr":"172.21.15.160:6821/3251327001","heartbeat_back_addr":"172.21.15.160:6825/3251327001","heartbeat_front_addr":"172.21.15.160:6823/3251327001","state":["exists","up"]},{"osd":7,"uuid":"e66b400b-9747-4f6d-b05b-db22468e2750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6817","nonce":1175500850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6819","nonce":1175500850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6823","nonce":1175500850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6821","nonce":1175500850}]},"public_addr":"172.21.15.177:6817/1175500850","cluster_addr":"172.21.15.177:6819/1175500850","heartbeat_back_addr":"172.21.15.177:6823/1175500850","heartbeat_front_addr":"172.21.15.177:6821/1175500850","state":["exists","up"]},{"osd":8,"uuid":"3ee0f2c4-fca4-49fc-a98b-07aa0fcba0e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6817","nonce":688015531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6819","nonce":688015531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6823","nonce":688015531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6821","nonce":688015531}]},"public_addr":"172.21.15.178:6817/688015531","cluster_addr":"172.21.15.178:6819/688015531","heartbeat_back_addr":"172.21.15.178:6823/688015531","heartbeat_front_addr":"172.21.15.178:6821/688015531","state":["exists","up"]},{"osd":9,"uuid":"856039d7-f7ee-4dbd-b019-0073e705ce0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6827","nonce":3265797851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6829","nonce":3265797851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6833","nonce":3265797851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6831","nonce":3265797851}]},"public_addr":"172.21.15.160:6827/3265797851","cluster_addr":"172.21.15.160:6829/3265797851","heartbeat_back_addr":"172.21.15.160:6833/3265797851","heartbeat_front_addr":"172.21.15.160:6831/3265797851","state":["exists","up"]},{"osd":10,"uuid":"bdc9899d-d720-4429-ad53-4c388a4dbe75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6825","nonce":2867473412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6827","nonce":2867473412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6831","nonce":2867473412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6829","nonce":2867473412}]},"public_addr":"172.21.15.177:6825/2867473412","cluster_addr":"172.21.15.177:6827/2867473412","heartbeat_back_addr":"172.21.15.177:6831/2867473412","heartbeat_front_addr":"172.21.15.177:6829/2867473412","state":["exists","up"]},{"osd":11,"uuid":"cc9adb54-ef49-47c6-90e3-fc03bc0f5cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6825","nonce":4068724731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6827","nonce":4068724731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6831","nonce":4068724731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6829","nonce":4068724731}]},"public_addr":"172.21.15.178:6825/4068724731","cluster_addr":"172.21.15.178:6827/4068724731","heartbeat_back_addr":"172.21.15.178:6831/4068724731","heartbeat_front_addr":"172.21.15.178:6829/4068724731","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:33:44.141585+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:07.685585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:31.044386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:56.308925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:35:21.582097+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:35:45.983540+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:36:11.530242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:36:36.844681+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:02.303437+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:27.629867+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:53.289495+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:38:18.536169+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/3027450612":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/874709722":"2024-03-08T07:32:08.190837+0000","172.21.15.160:6800/1130675225":"2024-03-08T07:31:41.166858+0000","172.21.15.160:0/3857193980":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/3861968297":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6800/3859174315":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/3013534743":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6801/3859174315":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/1310937667":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6801/1130675225":"2024-03-08T07:31:41.166858+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T07:41:04.972 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.971+0000 7f2f41378640 1 -- 172.21.15.160:0/353281160 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2f240a1d80 msgr2=0x7f2f240a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:04.972 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.971+0000 7f2f41378640 1 --2- 172.21.15.160:0/353281160 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2f240a1d80 0x7f2f240a4240 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2f30000c00 tx=0x7f2f30005fd0 comp rx=0 tx=0).stop 2024-03-07T07:41:04.972 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.971+0000 7f2f41378640 1 -- 172.21.15.160:0/353281160 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f3c101780 msgr2=0x7f2f3c19c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:04.972 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.971+0000 7f2f41378640 1 --2- 172.21.15.160:0/353281160 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f3c101780 0x7f2f3c19c6e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00ea60 tx=0x7f2f2c00ef20 comp rx=0 tx=0).stop 2024-03-07T07:41:04.972 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.972+0000 7f2f41378640 1 -- 172.21.15.160:0/353281160 shutdown_connections 2024-03-07T07:41:04.972 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.972+0000 7f2f41378640 1 --2- 172.21.15.160:0/353281160 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2f240a1d80 0x7f2f240a4240 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:04.973 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.972+0000 7f2f41378640 1 --2- 172.21.15.160:0/353281160 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f3c102140 0x7f2f3c19ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:04.973 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.972+0000 7f2f41378640 1 --2- 172.21.15.160:0/353281160 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f3c101780 0x7f2f3c19c6e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:04.973 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.972+0000 7f2f41378640 1 --2- 172.21.15.160:0/353281160 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2f3c100590 0x7f2f3c06a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:04.973 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.972+0000 7f2f41378640 1 -- 172.21.15.160:0/353281160 >> 172.21.15.160:0/353281160 conn(0x7f2f3c0fbb40 msgr2=0x7f2f3c104b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:04.973 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.972+0000 7f2f41378640 1 -- 172.21.15.160:0/353281160 shutdown_connections 2024-03-07T07:41:04.973 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:04.972+0000 7f2f41378640 1 -- 172.21.15.160:0/353281160 wait complete. 2024-03-07T07:41:04.983 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-07T07:39:14.164243+0000', 'modified': '2024-03-07T07:39:18.861903+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24511, 'mds_1': 24497, 'mds_2': 24487}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24436': {'gid': 24436, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.178:6833/645365846', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6832', 'nonce': 645365846}, {'type': 'v1', 'addr': '172.21.15.178:6833', 'nonce': 645365846}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24458': {'gid': 24458, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.177:6837/4088036516', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6836', 'nonce': 4088036516}, {'type': 'v1', 'addr': '172.21.15.177:6837', 'nonce': 4088036516}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24475': {'gid': 24475, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.178:6835/647840547', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6834', 'nonce': 647840547}, {'type': 'v1', 'addr': '172.21.15.178:6835', 'nonce': 647840547}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24487': {'gid': 24487, 'name': 'c', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.178:6837/1247829154', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6836', 'nonce': 1247829154}, {'type': 'v1', 'addr': '172.21.15.178:6837', 'nonce': 1247829154}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24497': {'gid': 24497, 'name': 'b', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.177:6839/3329773838', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6838', 'nonce': 3329773838}, {'type': 'v1', 'addr': '172.21.15.177:6839', 'nonce': 3329773838}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24511': {'gid': 24511, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.178:6839/2947354024', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6838', 'nonce': 2947354024}, {'type': 'v1', 'addr': '172.21.15.178:6839', 'nonce': 2947354024}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24511, 'qdb_cluster': [24487, 24511, 24497]} 2024-03-07T07:41:04.984 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-03-07T07:41:04.984 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-07T07:41:05.088 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.085+0000 7f1bced14640 1 -- 172.21.15.160:0/2454939106 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1bc8102860 msgr2=0x7f1bc8102ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:05.088 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.085+0000 7f1bced14640 1 --2- 172.21.15.160:0/2454939106 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1bc8102860 0x7f1bc8102ce0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4009900 tx=0x7f1bb402bc30 comp rx=0 tx=0).stop 2024-03-07T07:41:05.088 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.085+0000 7f1bced14640 1 -- 172.21.15.160:0/2454939106 shutdown_connections 2024-03-07T07:41:05.088 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.085+0000 7f1bced14640 1 --2- 172.21.15.160:0/2454939106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1bc8103220 0x7f1bc8109ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.088 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.085+0000 7f1bced14640 1 --2- 172.21.15.160:0/2454939106 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1bc8102860 0x7f1bc8102ce0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.088 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.085+0000 7f1bced14640 1 --2- 172.21.15.160:0/2454939106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1bc8101670 0x7f1bc8101a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.088 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.085+0000 7f1bced14640 1 -- 172.21.15.160:0/2454939106 >> 172.21.15.160:0/2454939106 conn(0x7f1bc80fcc00 msgr2=0x7f1bc80ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:05.089 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.086+0000 7f1bced14640 1 -- 172.21.15.160:0/2454939106 shutdown_connections 2024-03-07T07:41:05.089 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.086+0000 7f1bced14640 1 -- 172.21.15.160:0/2454939106 wait complete. 2024-03-07T07:41:05.089 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.086+0000 7f1bced14640 1 Processor -- start 2024-03-07T07:41:05.089 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.086+0000 7f1bced14640 1 -- start start 2024-03-07T07:41:05.089 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.086+0000 7f1bced14640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1bc8101670 0x7f1bc8075410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.089 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.086+0000 7f1bced14640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1bc8102860 0x7f1bc8073a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.089 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.086+0000 7f1bced14640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1bc8103220 0x7f1bc8073f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.089 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.086+0000 7f1bced14640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc810d2e0 con 0x7f1bc8101670 2024-03-07T07:41:05.089 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.086+0000 7f1bced14640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc810d160 con 0x7f1bc8102860 2024-03-07T07:41:05.089 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.086+0000 7f1bced14640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc810d460 con 0x7f1bc8103220 2024-03-07T07:41:05.089 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.086+0000 7f1bccab0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1bc8101670 0x7f1bc8075410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.089 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.086+0000 7f1bccab0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1bc8101670 0x7f1bc8075410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57878/0 (socket says 172.21.15.160:57878) 2024-03-07T07:41:05.089 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.086+0000 7f1bccab0640 1 -- 172.21.15.160:0/3488984247 learned_addr learned my addr 172.21.15.160:0/3488984247 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:41:05.090 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.086+0000 7f1bc7fff640 1 --2- 172.21.15.160:0/3488984247 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1bc8102860 0x7f1bc8073a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.090 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.086+0000 7f1bcd2b1640 1 --2- 172.21.15.160:0/3488984247 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1bc8103220 0x7f1bc8073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.090 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.087+0000 7f1bccab0640 1 -- 172.21.15.160:0/3488984247 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1bc8103220 msgr2=0x7f1bc8073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:05.090 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.087+0000 7f1bccab0640 1 --2- 172.21.15.160:0/3488984247 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1bc8103220 0x7f1bc8073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.090 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.087+0000 7f1bccab0640 1 -- 172.21.15.160:0/3488984247 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1bc8102860 msgr2=0x7f1bc8073a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:05.090 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.087+0000 7f1bccab0640 1 --2- 172.21.15.160:0/3488984247 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1bc8102860 0x7f1bc8073a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.090 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.087+0000 7f1bccab0640 1 -- 172.21.15.160:0/3488984247 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bc8074820 con 0x7f1bc8101670 2024-03-07T07:41:05.090 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.087+0000 7f1bc7fff640 1 --2- 172.21.15.160:0/3488984247 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1bc8102860 0x7f1bc8073a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:41:05.090 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.087+0000 7f1bccab0640 1 --2- 172.21.15.160:0/3488984247 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1bc8101670 0x7f1bc8075410 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc00e9c0 tx=0x7f1bbc00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:05.090 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.087+0000 7f1bc5ffb640 1 -- 172.21.15.160:0/3488984247 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc00cd80 con 0x7f1bc8101670 2024-03-07T07:41:05.090 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.087+0000 7f1bc5ffb640 1 -- 172.21.15.160:0/3488984247 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1bbc010910 con 0x7f1bc8101670 2024-03-07T07:41:05.090 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.087+0000 7f1bced14640 1 -- 172.21.15.160:0/3488984247 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bc81a1b00 con 0x7f1bc8101670 2024-03-07T07:41:05.090 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.087+0000 7f1bc5ffb640 1 -- 172.21.15.160:0/3488984247 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc018950 con 0x7f1bc8101670 2024-03-07T07:41:05.090 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.087+0000 7f1bced14640 1 -- 172.21.15.160:0/3488984247 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bc81a2020 con 0x7f1bc8101670 2024-03-07T07:41:05.097 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.089+0000 7f1bc5ffb640 1 -- 172.21.15.160:0/3488984247 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f1bbc010ac0 con 0x7f1bc8101670 2024-03-07T07:41:05.097 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.090+0000 7f1bd1bd7740 1 Processor -- start 2024-03-07T07:41:05.097 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.090+0000 7f1bd1bd7740 1 -- start start 2024-03-07T07:41:05.097 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.090+0000 7f1bd1bd7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b29bbe20b0 0x55b29bbe24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.097 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.090+0000 7f1bd1bd7740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b29bbe2aa0 0x55b29bbe75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.098 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.090+0000 7f1bc5ffb640 1 --2- 172.21.15.160:0/3488984247 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f1bb80a1d80 0x7f1bb80a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.098 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.090+0000 7f1bd1bd7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b29bbe7c10 0x55b29bbea0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.098 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.090+0000 7f1bd1bd7740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29bb47790 con 0x55b29bbe2aa0 2024-03-07T07:41:05.098 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.090+0000 7f1bd1bd7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29baf0b20 con 0x55b29bbe7c10 2024-03-07T07:41:05.098 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.090+0000 7f1bd1bd7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29bb43830 con 0x55b29bbe20b0 2024-03-07T07:41:05.098 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.090+0000 7f1bccab0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b29bbe2aa0 0x55b29bbe75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.098 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.090+0000 7f1bc7fff640 1 --2- 172.21.15.160:0/3488984247 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f1bb80a1d80 0x7f1bb80a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.098 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.090+0000 7f1bcd2b1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b29bbe20b0 0x55b29bbe24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.098 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.090+0000 7f1bccab0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b29bbe2aa0 0x55b29bbe75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57880/0 (socket says 172.21.15.160:57880) 2024-03-07T07:41:05.098 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.090+0000 7f1bccab0640 1 -- 172.21.15.160:0/76889876 learned_addr learned my addr 172.21.15.160:0/76889876 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:41:05.098 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.090+0000 7f1bcd2b1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b29bbe20b0 0x55b29bbe24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.160:47384/0 (socket says 172.21.15.160:47384) 2024-03-07T07:41:05.098 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.090+0000 7f1bc5ffb640 1 -- 172.21.15.160:0/3488984247 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1bbc0ce1f0 con 0x7f1bc8101670 2024-03-07T07:41:05.098 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.090+0000 7f1bcd2b1640 1 --2- 172.21.15.160:0/76889876 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b29bbe7c10 0x55b29bbea0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.098 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.090+0000 7f1bcd2b1640 1 -- 172.21.15.160:0/76889876 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b29bbe7c10 msgr2=0x55b29bbea0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:41:05.098 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.090+0000 7f1bcd2b1640 1 --2- 172.21.15.160:0/76889876 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b29bbe7c10 0x55b29bbea0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.099 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.090+0000 7f1bcd2b1640 1 -- 172.21.15.160:0/76889876 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b29bbe2aa0 msgr2=0x55b29bbe75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:41:05.099 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.090+0000 7f1bcd2b1640 1 --2- 172.21.15.160:0/76889876 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b29bbe2aa0 0x55b29bbe75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.099 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.090+0000 7f1bcd2b1640 1 -- 172.21.15.160:0/76889876 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b29bb04970 con 0x55b29bbe20b0 2024-03-07T07:41:05.099 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.091+0000 7f1bc7fff640 1 --2- 172.21.15.160:0/3488984247 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f1bb80a1d80 0x7f1bb80a4240 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1bb402c7b0 tx=0x7f1bb4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:05.099 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.091+0000 7f1bcd2b1640 1 --2- 172.21.15.160:0/76889876 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b29bbe20b0 0x55b29bbe24d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0011a70 tx=0x7f1bc0004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b8f9a32936c20ac server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:05.099 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.091+0000 7f1bb2ffd640 1 -- 172.21.15.160:0/76889876 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc0004b20 con 0x55b29bbe20b0 2024-03-07T07:41:05.099 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.091+0000 7f1bb2ffd640 1 -- 172.21.15.160:0/76889876 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1bc000fd60 con 0x55b29bbe20b0 2024-03-07T07:41:05.099 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.091+0000 7f1bb2ffd640 1 -- 172.21.15.160:0/76889876 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc00054a0 con 0x55b29bbe20b0 2024-03-07T07:41:05.099 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.091+0000 7f1bd1bd7740 1 -- 172.21.15.160:0/76889876 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b29bbe20b0 msgr2=0x55b29bbe24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:05.099 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.091+0000 7f1bd1bd7740 1 --2- 172.21.15.160:0/76889876 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b29bbe20b0 0x55b29bbe24d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0011a70 tx=0x7f1bc0004ab0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.099 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.091+0000 7f1bd1bd7740 1 -- 172.21.15.160:0/76889876 shutdown_connections 2024-03-07T07:41:05.099 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.091+0000 7f1bd1bd7740 1 --2- 172.21.15.160:0/76889876 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b29bbe7c10 0x55b29bbea0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.099 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.091+0000 7f1bd1bd7740 1 --2- 172.21.15.160:0/76889876 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b29bbe2aa0 0x55b29bbe75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.099 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.091+0000 7f1bd1bd7740 1 --2- 172.21.15.160:0/76889876 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b29bbe20b0 0x55b29bbe24d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.091+0000 7f1bd1bd7740 1 -- 172.21.15.160:0/76889876 >> 172.21.15.160:0/76889876 conn(0x55b29bbaa3d0 msgr2=0x55b29bbdda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:05.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.092+0000 7f1bd1bd7740 1 -- 172.21.15.160:0/76889876 shutdown_connections 2024-03-07T07:41:05.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.092+0000 7f1bd1bd7740 1 -- 172.21.15.160:0/76889876 wait complete. 2024-03-07T07:41:05.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.092+0000 7f1bd1bd7740 1 -- start start 2024-03-07T07:41:05.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.092+0000 7f1bd1bd7740 1 Processor -- start 2024-03-07T07:41:05.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.092+0000 7f1bd1bd7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b29bbe20b0 0x55b29bbec690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.092+0000 7f1bd1bd7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b29bbe2aa0 0x55b29bbecbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.092+0000 7f1bd1bd7740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b29bbe7c10 0x55b29bbf3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.092+0000 7f1bd1bd7740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29bbeabc0 con 0x55b29bbe7c10 2024-03-07T07:41:05.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.092+0000 7f1bd1bd7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29bb03f60 con 0x55b29bbe20b0 2024-03-07T07:41:05.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.092+0000 7f1bd1bd7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29bbead40 con 0x55b29bbe2aa0 2024-03-07T07:41:05.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.092+0000 7f1bd1bd7740 10 client.0 _ll_register_callbacks cb 0x55b29bb3f3d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-07T07:41:05.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.092+0000 7f1bcd2b1640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b29bbe7c10 0x55b29bbf3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.092+0000 7f1bccab0640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b29bbe20b0 0x55b29bbec690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.092+0000 7f1bcd2b1640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b29bbe7c10 0x55b29bbf3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57896/0 (socket says 172.21.15.160:57896) 2024-03-07T07:41:05.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.092+0000 7f1bcd2b1640 1 -- 172.21.15.160:0/1923093443 learned_addr learned my addr 172.21.15.160:0/1923093443 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:41:05.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.092+0000 7f1bc7fff640 1 --2- 172.21.15.160:0/1923093443 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b29bbe2aa0 0x55b29bbecbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.093+0000 7f1bccab0640 1 -- 172.21.15.160:0/1923093443 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b29bbe2aa0 msgr2=0x55b29bbecbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:41:05.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.093+0000 7f1bccab0640 1 --2- 172.21.15.160:0/1923093443 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b29bbe2aa0 0x55b29bbecbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.093+0000 7f1bccab0640 1 -- 172.21.15.160:0/1923093443 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b29bbe7c10 msgr2=0x55b29bbf3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:41:05.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.093+0000 7f1bccab0640 1 --2- 172.21.15.160:0/1923093443 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b29bbe7c10 0x55b29bbf3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.093+0000 7f1bccab0640 1 -- 172.21.15.160:0/1923093443 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b29b85c2d0 con 0x55b29bbe20b0 2024-03-07T07:41:05.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.093+0000 7f1bcd2b1640 1 --2- 172.21.15.160:0/1923093443 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b29bbe7c10 0x55b29bbf3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:41:05.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.093+0000 7f1bccab0640 1 --2- 172.21.15.160:0/1923093443 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b29bbe20b0 0x55b29bbec690 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x55b29bbe3510 tx=0x7f1bbc01d7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb841ca1aedaaee6 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:05.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.093+0000 7f1ba7fff640 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-03-07T07:41:05.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.093+0000 7f1ba7fff640 1 -- 172.21.15.160:0/1923093443 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc0cd070 con 0x55b29bbe20b0 2024-03-07T07:41:05.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.093+0000 7f1ba7fff640 1 -- 172.21.15.160:0/1923093443 <== mon.2 v2:172.21.15.177:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1bbc0a31f0 con 0x55b29bbe20b0 2024-03-07T07:41:05.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.093+0000 7f1bd1bd7740 1 -- 172.21.15.160:0/1923093443 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1bc000fd60 con 0x55b29bbe20b0 2024-03-07T07:41:05.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.093+0000 7f1ba7fff640 1 -- 172.21.15.160:0/1923093443 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bbc09b5e0 con 0x55b29bbe20b0 2024-03-07T07:41:05.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.093+0000 7f1bd1bd7740 10 client.25043 fetch_fsmap learned FSMap version 21 2024-03-07T07:41:05.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.093+0000 7f1bd1bd7740 1 -- 172.21.15.160:0/1923093443 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55b29b9266b0 con 0x55b29bbe20b0 2024-03-07T07:41:05.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.093+0000 7f1ba7fff640 1 -- 172.21.15.160:0/1923093443 <== mon.2 v2:172.21.15.177:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1bc000fd60 con 0x55b29bbe20b0 2024-03-07T07:41:05.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.094+0000 7f1ba7fff640 1 -- 172.21.15.160:0/1923093443 <== mon.2 v2:172.21.15.177:3300/0 5 ==== fsmap(e 21) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f1bbc09a330 con 0x55b29bbe20b0 2024-03-07T07:41:05.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.094+0000 7f1bd1bd7740 10 client.25043 fetch_fsmap finished waiting for FSMap version 21 2024-03-07T07:41:05.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.094+0000 7f1bd1bd7740 10 client.25043 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-07T07:41:05.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.094+0000 7f1bd1bd7740 20 client.25043 populate_metadata read hostname 'smithi160' 2024-03-07T07:41:05.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.094+0000 7f1bd1bd7740 1 --2- 172.21.15.160:0/1923093443 >> [v2:172.21.15.178:6836/1247829154,v1:172.21.15.178:6837/1247829154] conn(0x55b29bbf9140 0x55b29bbf81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.094+0000 7f1bd1bd7740 4 client.25043 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-07T07:41:05.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.094+0000 7f1bd1bd7740 1 -- 172.21.15.160:0/1923093443 --> [v2:172.21.15.178:6836/1247829154,v1:172.21.15.178:6837/1247829154] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b29bb54cd0 con 0x55b29bbf9140 2024-03-07T07:41:05.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.094+0000 7f1bc7fff640 1 --2- 172.21.15.160:0/1923093443 >> [v2:172.21.15.178:6836/1247829154,v1:172.21.15.178:6837/1247829154] conn(0x55b29bbf9140 0x55b29bbf81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.094+0000 7f1ba7fff640 10 client.25043 ms_handle_connect on v2:172.21.15.178:6836/1247829154 2024-03-07T07:41:05.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.094+0000 7f1bc7fff640 1 --2- 172.21.15.160:0/1923093443 >> [v2:172.21.15.178:6836/1247829154,v1:172.21.15.178:6837/1247829154] conn(0x55b29bbf9140 0x55b29bbf81c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=eb2b3b8d99fc62c3 server_cookie=59f6f85362792dd in_seq=0 out_seq=0 2024-03-07T07:41:05.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.095+0000 7f1ba7fff640 1 -- 172.21.15.160:0/1923093443 <== mds.2 v2:172.21.15.178:6836/1247829154 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b29bb54cd0 con 0x55b29bbf9140 2024-03-07T07:41:05.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.095+0000 7f1ba7fff640 10 client.25043 handle_command_reply: tid=0 2024-03-07T07:41:05.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.095+0000 7f1bd1bd7740 1 client.25043 shutdown 2024-03-07T07:41:05.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.095+0000 7f1bd1bd7740 1 -- 172.21.15.160:0/1923093443 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b29bbe20b0 msgr2=0x55b29bbec690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:05.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.095+0000 7f1bd1bd7740 1 --2- 172.21.15.160:0/1923093443 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b29bbe20b0 0x55b29bbec690 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x55b29bbe3510 tx=0x7f1bbc01d7c0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.095+0000 7f1bd1bd7740 1 -- 172.21.15.160:0/1923093443 shutdown_connections 2024-03-07T07:41:05.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.095+0000 7f1bd1bd7740 1 --2- 172.21.15.160:0/1923093443 >> [v2:172.21.15.178:6836/1247829154,v1:172.21.15.178:6837/1247829154] conn(0x55b29bbf9140 0x55b29bbf81c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.095+0000 7f1bd1bd7740 1 --2- 172.21.15.160:0/1923093443 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b29bbe7c10 0x55b29bbf3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.095+0000 7f1ba7fff640 0 client.25043 ms_handle_reset on v2:172.21.15.178:6836/1247829154 2024-03-07T07:41:05.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.095+0000 7f1bd1bd7740 1 --2- 172.21.15.160:0/1923093443 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b29bbe2aa0 0x55b29bbecbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.095+0000 7f1bd1bd7740 1 --2- 172.21.15.160:0/1923093443 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b29bbe20b0 0x55b29bbec690 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.095+0000 7f1bd1bd7740 1 -- 172.21.15.160:0/1923093443 >> 172.21.15.160:0/1923093443 conn(0x55b29bbaa3d0 msgr2=0x55b29bbdcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:05.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.095+0000 7f1bd1bd7740 1 -- 172.21.15.160:0/1923093443 shutdown_connections 2024-03-07T07:41:05.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.095+0000 7f1bd1bd7740 1 -- 172.21.15.160:0/1923093443 wait complete. 2024-03-07T07:41:05.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.095+0000 7f1bd1bd7740 20 client.25043 trim_cache size 0 max 16384 2024-03-07T07:41:05.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.112+0000 7f1bd1bd7740 1 Processor -- start 2024-03-07T07:41:05.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.112+0000 7f1bd1bd7740 1 -- start start 2024-03-07T07:41:05.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.112+0000 7f1bd1bd7740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b29bbe20b0 0x55b29bbeb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.112+0000 7f1bd1bd7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b29bbe2aa0 0x55b29bbf3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.112+0000 7f1bd1bd7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b29bbe7c10 0x55b29bbe4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.112+0000 7f1bd1bd7740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29bbf8960 con 0x55b29bbe20b0 2024-03-07T07:41:05.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.112+0000 7f1bd1bd7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29bbf8fc0 con 0x55b29bbe2aa0 2024-03-07T07:41:05.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.112+0000 7f1bd1bd7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29bbf9620 con 0x55b29bbe7c10 2024-03-07T07:41:05.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.112+0000 7f1bccab0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b29bbe20b0 0x55b29bbeb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.112+0000 7f1bccab0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b29bbe20b0 0x55b29bbeb140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57900/0 (socket says 172.21.15.160:57900) 2024-03-07T07:41:05.119 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.112+0000 7f1bccab0640 1 -- 172.21.15.160:0/162218422 learned_addr learned my addr 172.21.15.160:0/162218422 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:41:05.119 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.112+0000 7f1bcd2b1640 1 --2- 172.21.15.160:0/162218422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b29bbe7c10 0x55b29bbe4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.119 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.112+0000 7f1bc7fff640 1 --2- 172.21.15.160:0/162218422 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b29bbe2aa0 0x55b29bbf3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.119 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.113+0000 7f1bccab0640 1 -- 172.21.15.160:0/162218422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b29bbe7c10 msgr2=0x55b29bbe4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:41:05.119 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.113+0000 7f1bccab0640 1 --2- 172.21.15.160:0/162218422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b29bbe7c10 0x55b29bbe4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.119 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.113+0000 7f1bccab0640 1 -- 172.21.15.160:0/162218422 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b29bbe2aa0 msgr2=0x55b29bbf3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:41:05.119 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.113+0000 7f1bccab0640 1 --2- 172.21.15.160:0/162218422 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b29bbe2aa0 0x55b29bbf3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.119 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.113+0000 7f1bccab0640 1 -- 172.21.15.160:0/162218422 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b29bbe2e80 con 0x55b29bbe20b0 2024-03-07T07:41:05.119 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.113+0000 7f1bccab0640 1 --2- 172.21.15.160:0/162218422 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b29bbe20b0 0x55b29bbeb140 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc09ae70 tx=0x7f1bbc018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3181b75fb080698 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:05.119 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.113+0000 7f1bb27fc640 1 -- 172.21.15.160:0/162218422 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc0cd070 con 0x55b29bbe20b0 2024-03-07T07:41:05.119 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.113+0000 7f1bb27fc640 1 -- 172.21.15.160:0/162218422 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1bbc0a3680 con 0x55b29bbe20b0 2024-03-07T07:41:05.119 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.113+0000 7f1bb27fc640 1 -- 172.21.15.160:0/162218422 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc0a3430 con 0x55b29bbe20b0 2024-03-07T07:41:05.119 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.113+0000 7f1bd1bd7740 1 -- 172.21.15.160:0/162218422 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b29bbe20b0 msgr2=0x55b29bbeb140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:05.119 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.113+0000 7f1bd1bd7740 1 --2- 172.21.15.160:0/162218422 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b29bbe20b0 0x55b29bbeb140 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc09ae70 tx=0x7f1bbc018420 comp rx=0 tx=0).stop 2024-03-07T07:41:05.120 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.113+0000 7f1bd1bd7740 1 -- 172.21.15.160:0/162218422 shutdown_connections 2024-03-07T07:41:05.120 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.113+0000 7f1bd1bd7740 1 --2- 172.21.15.160:0/162218422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b29bbe7c10 0x55b29bbe4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.120 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.113+0000 7f1bd1bd7740 1 --2- 172.21.15.160:0/162218422 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b29bbe2aa0 0x55b29bbf3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.120 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.113+0000 7f1bd1bd7740 1 --2- 172.21.15.160:0/162218422 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b29bbe20b0 0x55b29bbeb140 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.120 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.113+0000 7f1bd1bd7740 1 -- 172.21.15.160:0/162218422 >> 172.21.15.160:0/162218422 conn(0x55b29bbaa3d0 msgr2=0x55b29bbe3540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:05.120 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.113+0000 7f1bd1bd7740 1 -- 172.21.15.160:0/162218422 shutdown_connections 2024-03-07T07:41:05.120 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.113+0000 7f1bd1bd7740 1 -- 172.21.15.160:0/162218422 wait complete. 2024-03-07T07:41:05.120 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.114+0000 7f1bd1bd7740 1 -- start start 2024-03-07T07:41:05.120 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.114+0000 7f1bd1bd7740 1 Processor -- start 2024-03-07T07:41:05.120 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.114+0000 7f1bd1bd7740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b29bbe20b0 0x55b29bbf3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.120 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.114+0000 7f1bd1bd7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b29bbe2aa0 0x55b29bbf3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.120 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.114+0000 7f1bd1bd7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b29bbe7c10 0x55b29bbf01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.120 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.114+0000 7f1bd1bd7740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29bbfa2e0 con 0x55b29bbe20b0 2024-03-07T07:41:05.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.114+0000 7f1bd1bd7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29bbf9c80 con 0x55b29bbe2aa0 2024-03-07T07:41:05.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.114+0000 7f1bd1bd7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29bbfa940 con 0x55b29bbe7c10 2024-03-07T07:41:05.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.114+0000 7f1bd1bd7740 10 client.0 _ll_register_callbacks cb 0x7f1bbc01ff70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-07T07:41:05.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.114+0000 7f1bccab0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b29bbe20b0 0x55b29bbf3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.114+0000 7f1bccab0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b29bbe20b0 0x55b29bbf3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57916/0 (socket says 172.21.15.160:57916) 2024-03-07T07:41:05.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.114+0000 7f1bccab0640 1 -- 172.21.15.160:0/2573567339 learned_addr learned my addr 172.21.15.160:0/2573567339 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:41:05.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.114+0000 7f1bcd2b1640 1 --2- 172.21.15.160:0/2573567339 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b29bbe7c10 0x55b29bbf01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.114+0000 7f1bc7fff640 1 --2- 172.21.15.160:0/2573567339 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b29bbe2aa0 0x55b29bbf3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.114+0000 7f1bccab0640 1 -- 172.21.15.160:0/2573567339 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b29bbe7c10 msgr2=0x55b29bbf01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:41:05.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.114+0000 7f1bccab0640 1 --2- 172.21.15.160:0/2573567339 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b29bbe7c10 0x55b29bbf01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.114+0000 7f1bccab0640 1 -- 172.21.15.160:0/2573567339 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b29bbe2aa0 msgr2=0x55b29bbf3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:41:05.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.114+0000 7f1bccab0640 1 --2- 172.21.15.160:0/2573567339 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b29bbe2aa0 0x55b29bbf3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.114+0000 7f1bccab0640 1 -- 172.21.15.160:0/2573567339 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b29bbf09e0 con 0x55b29bbe20b0 2024-03-07T07:41:05.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.115+0000 7f1bccab0640 1 --2- 172.21.15.160:0/2573567339 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b29bbe20b0 0x55b29bbf3620 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc00f010 tx=0x7f1bbc0903c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52cd5d358610ad7f server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:05.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.115+0000 7f1ba7fff640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-03-07T07:41:05.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.115+0000 7f1ba7fff640 1 -- 172.21.15.160:0/2573567339 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc064070 con 0x55b29bbe20b0 2024-03-07T07:41:05.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.115+0000 7f1ba7fff640 1 -- 172.21.15.160:0/2573567339 <== mon.0 v2:172.21.15.160:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1bbc0bc0b0 con 0x55b29bbe20b0 2024-03-07T07:41:05.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.115+0000 7f1bd1bd7740 1 -- 172.21.15.160:0/2573567339 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1bbc0a3680 con 0x55b29bbe20b0 2024-03-07T07:41:05.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.115+0000 7f1ba7fff640 1 -- 172.21.15.160:0/2573567339 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bbc0cd070 con 0x55b29bbe20b0 2024-03-07T07:41:05.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.115+0000 7f1bd1bd7740 10 client.15213 fetch_fsmap learned FSMap version 21 2024-03-07T07:41:05.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.115+0000 7f1bd1bd7740 1 -- 172.21.15.160:0/2573567339 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55b29bc55000 con 0x55b29bbe20b0 2024-03-07T07:41:05.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.115+0000 7f1ba7fff640 1 -- 172.21.15.160:0/2573567339 <== mon.0 v2:172.21.15.160:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1bbc0a3680 con 0x55b29bbe20b0 2024-03-07T07:41:05.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.115+0000 7f1ba7fff640 1 -- 172.21.15.160:0/2573567339 <== mon.0 v2:172.21.15.160:3300/0 5 ==== fsmap(e 21) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f1bbc096d50 con 0x55b29bbe20b0 2024-03-07T07:41:05.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.115+0000 7f1bd1bd7740 10 client.15213 fetch_fsmap finished waiting for FSMap version 21 2024-03-07T07:41:05.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.115+0000 7f1bd1bd7740 10 client.15213 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-07T07:41:05.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.115+0000 7f1bd1bd7740 20 client.15213 populate_metadata read hostname 'smithi160' 2024-03-07T07:41:05.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.115+0000 7f1bd1bd7740 1 --2- 172.21.15.160:0/2573567339 >> [v2:172.21.15.178:6836/1247829154,v1:172.21.15.178:6837/1247829154] conn(0x55b29bbf9140 0x55b29bc554c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.115+0000 7f1bd1bd7740 4 client.15213 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-07T07:41:05.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.115+0000 7f1bd1bd7740 1 -- 172.21.15.160:0/2573567339 --> [v2:172.21.15.178:6836/1247829154,v1:172.21.15.178:6837/1247829154] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b29bb4f020 con 0x55b29bbf9140 2024-03-07T07:41:05.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.115+0000 7f1bc7fff640 1 --2- 172.21.15.160:0/2573567339 >> [v2:172.21.15.178:6836/1247829154,v1:172.21.15.178:6837/1247829154] conn(0x55b29bbf9140 0x55b29bc554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.116+0000 7f1ba7fff640 10 client.15213 ms_handle_connect on v2:172.21.15.178:6836/1247829154 2024-03-07T07:41:05.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.116+0000 7f1bc7fff640 1 --2- 172.21.15.160:0/2573567339 >> [v2:172.21.15.178:6836/1247829154,v1:172.21.15.178:6837/1247829154] conn(0x55b29bbf9140 0x55b29bc554c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8a896b645dafd233 server_cookie=253b72a12ed82f07 in_seq=0 out_seq=0 2024-03-07T07:41:05.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.116+0000 7f1ba7fff640 1 -- 172.21.15.160:0/2573567339 <== mds.2 v2:172.21.15.178:6836/1247829154 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55b29bb4f020 con 0x55b29bbf9140 2024-03-07T07:41:05.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.116+0000 7f1ba7fff640 10 client.15213 handle_command_reply: tid=0 2024-03-07T07:41:05.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.116+0000 7f1bd1bd7740 1 client.15213 shutdown 2024-03-07T07:41:05.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.116+0000 7f1bd1bd7740 1 -- 172.21.15.160:0/2573567339 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b29bbe20b0 msgr2=0x55b29bbf3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:05.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.116+0000 7f1bd1bd7740 1 --2- 172.21.15.160:0/2573567339 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b29bbe20b0 0x55b29bbf3620 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc00f010 tx=0x7f1bbc0903c0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.116+0000 7f1bd1bd7740 1 -- 172.21.15.160:0/2573567339 shutdown_connections 2024-03-07T07:41:05.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.116+0000 7f1bd1bd7740 1 --2- 172.21.15.160:0/2573567339 >> [v2:172.21.15.178:6836/1247829154,v1:172.21.15.178:6837/1247829154] conn(0x55b29bbf9140 0x55b29bc554c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.116+0000 7f1bd1bd7740 1 --2- 172.21.15.160:0/2573567339 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b29bbe7c10 0x55b29bbf01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.116+0000 7f1bd1bd7740 1 --2- 172.21.15.160:0/2573567339 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b29bbe2aa0 0x55b29bbf3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.116+0000 7f1ba7fff640 0 client.15213 ms_handle_reset on v2:172.21.15.178:6836/1247829154 2024-03-07T07:41:05.124 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.116+0000 7f1bd1bd7740 1 --2- 172.21.15.160:0/2573567339 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b29bbe20b0 0x55b29bbf3620 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.124 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.116+0000 7f1bd1bd7740 1 -- 172.21.15.160:0/2573567339 >> 172.21.15.160:0/2573567339 conn(0x55b29bbaa3d0 msgr2=0x55b29bbdd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:05.124 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.116+0000 7f1bd1bd7740 1 -- 172.21.15.160:0/2573567339 shutdown_connections 2024-03-07T07:41:05.124 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.116+0000 7f1bd1bd7740 1 -- 172.21.15.160:0/2573567339 wait complete. 2024-03-07T07:41:05.124 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.116+0000 7f1bd1bd7740 20 client.15213 trim_cache size 0 max 16384 2024-03-07T07:41:05.124 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-03-07T07:41:05.124 INFO:teuthology.orchestra.run.smithi160.stdout: "cluster_fsid": "826f6582-dc54-11ee-95c6-87774f69a715", 2024-03-07T07:41:05.124 INFO:teuthology.orchestra.run.smithi160.stdout: "whoami": 2, 2024-03-07T07:41:05.124 INFO:teuthology.orchestra.run.smithi160.stdout: "id": 24487, 2024-03-07T07:41:05.124 INFO:teuthology.orchestra.run.smithi160.stdout: "want_state": "up:active", 2024-03-07T07:41:05.125 INFO:teuthology.orchestra.run.smithi160.stdout: "state": "up:active", 2024-03-07T07:41:05.125 INFO:teuthology.orchestra.run.smithi160.stdout: "fs_name": "cephfs", 2024-03-07T07:41:05.125 INFO:teuthology.orchestra.run.smithi160.stdout: "rank_uptime": 107.263636455, 2024-03-07T07:41:05.125 INFO:teuthology.orchestra.run.smithi160.stdout: "mdsmap_epoch": 21, 2024-03-07T07:41:05.125 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_epoch": 87, 2024-03-07T07:41:05.125 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_epoch_barrier": 85, 2024-03-07T07:41:05.125 INFO:teuthology.orchestra.run.smithi160.stdout: "uptime": 131.43392743000001 2024-03-07T07:41:05.125 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-03-07T07:41:05.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.117+0000 7f1bced14640 1 -- 172.21.15.160:0/3488984247 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f1bb80a1d80 msgr2=0x7f1bb80a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:05.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.117+0000 7f1bced14640 1 --2- 172.21.15.160:0/3488984247 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f1bb80a1d80 0x7f1bb80a4240 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1bb402c7b0 tx=0x7f1bb4005fd0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.117+0000 7f1bced14640 1 -- 172.21.15.160:0/3488984247 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1bc8101670 msgr2=0x7f1bc8075410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:05.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.117+0000 7f1bced14640 1 --2- 172.21.15.160:0/3488984247 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1bc8101670 0x7f1bc8075410 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc00e9c0 tx=0x7f1bbc00edc0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.117+0000 7f1bced14640 1 -- 172.21.15.160:0/3488984247 shutdown_connections 2024-03-07T07:41:05.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.117+0000 7f1bced14640 1 --2- 172.21.15.160:0/3488984247 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f1bb80a1d80 0x7f1bb80a4240 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.117+0000 7f1bced14640 1 --2- 172.21.15.160:0/3488984247 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1bc8103220 0x7f1bc8073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.117+0000 7f1bced14640 1 --2- 172.21.15.160:0/3488984247 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1bc8102860 0x7f1bc8073a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.117+0000 7f1bced14640 1 --2- 172.21.15.160:0/3488984247 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1bc8101670 0x7f1bc8075410 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.117+0000 7f1bced14640 1 -- 172.21.15.160:0/3488984247 >> 172.21.15.160:0/3488984247 conn(0x7f1bc80fcc00 msgr2=0x7f1bc8105be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:05.127 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.117+0000 7f1bced14640 1 -- 172.21.15.160:0/3488984247 shutdown_connections 2024-03-07T07:41:05.127 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.117+0000 7f1bced14640 1 -- 172.21.15.160:0/3488984247 wait complete. 2024-03-07T07:41:05.128 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-07T07:41:05.234 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.231+0000 7fa941c1e640 1 -- 172.21.15.160:0/2607304064 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa93c100570 msgr2=0x7fa93c100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:05.234 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.231+0000 7fa941c1e640 1 --2- 172.21.15.160:0/2607304064 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa93c100570 0x7fa93c100990 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa92c009900 tx=0x7fa92c02bc30 comp rx=0 tx=0).stop 2024-03-07T07:41:05.234 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.231+0000 7fa941c1e640 1 -- 172.21.15.160:0/2607304064 shutdown_connections 2024-03-07T07:41:05.234 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.231+0000 7fa941c1e640 1 --2- 172.21.15.160:0/2607304064 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa93c102120 0x7fa93c1089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.235 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.231+0000 7fa941c1e640 1 --2- 172.21.15.160:0/2607304064 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa93c101760 0x7fa93c101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.235 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.231+0000 7fa941c1e640 1 --2- 172.21.15.160:0/2607304064 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa93c100570 0x7fa93c100990 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.235 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.231+0000 7fa941c1e640 1 -- 172.21.15.160:0/2607304064 >> 172.21.15.160:0/2607304064 conn(0x7fa93c0fbb00 msgr2=0x7fa93c0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:05.235 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.231+0000 7fa941c1e640 1 -- 172.21.15.160:0/2607304064 shutdown_connections 2024-03-07T07:41:05.235 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.231+0000 7fa941c1e640 1 -- 172.21.15.160:0/2607304064 wait complete. 2024-03-07T07:41:05.235 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.232+0000 7fa941c1e640 1 Processor -- start 2024-03-07T07:41:05.235 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.232+0000 7fa941c1e640 1 -- start start 2024-03-07T07:41:05.235 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.232+0000 7fa941c1e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa93c100570 0x7fa93c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.235 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.232+0000 7fa941c1e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa93c101760 0x7fa93c19c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.235 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.232+0000 7fa941c1e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa93c102120 0x7fa93c19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.235 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.232+0000 7fa941c1e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa93c10c230 con 0x7fa93c100570 2024-03-07T07:41:05.236 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.232+0000 7fa941c1e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa93c10c0b0 con 0x7fa93c102120 2024-03-07T07:41:05.236 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.232+0000 7fa941c1e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa93c10c3b0 con 0x7fa93c101760 2024-03-07T07:41:05.236 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.232+0000 7fa93b7fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa93c100570 0x7fa93c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.236 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.232+0000 7fa93b7fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa93c100570 0x7fa93c06a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57924/0 (socket says 172.21.15.160:57924) 2024-03-07T07:41:05.236 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.232+0000 7fa93affd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa93c101760 0x7fa93c19c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.236 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.232+0000 7fa93b7fe640 1 -- 172.21.15.160:0/1681665051 learned_addr learned my addr 172.21.15.160:0/1681665051 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:41:05.236 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.232+0000 7fa93bfff640 1 --2- 172.21.15.160:0/1681665051 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa93c102120 0x7fa93c19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.236 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.232+0000 7fa93b7fe640 1 -- 172.21.15.160:0/1681665051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa93c101760 msgr2=0x7fa93c19c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:05.236 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.232+0000 7fa93b7fe640 1 --2- 172.21.15.160:0/1681665051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa93c101760 0x7fa93c19c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.236 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.232+0000 7fa93b7fe640 1 -- 172.21.15.160:0/1681665051 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa93c102120 msgr2=0x7fa93c19ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:05.236 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.232+0000 7fa93b7fe640 1 --2- 172.21.15.160:0/1681665051 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa93c102120 0x7fa93c19ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.236 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.232+0000 7fa93b7fe640 1 -- 172.21.15.160:0/1681665051 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa93c19f670 con 0x7fa93c100570 2024-03-07T07:41:05.236 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.232+0000 7fa93affd640 1 --2- 172.21.15.160:0/1681665051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa93c101760 0x7fa93c19c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:41:05.237 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.232+0000 7fa93bfff640 1 --2- 172.21.15.160:0/1681665051 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa93c102120 0x7fa93c19ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:41:05.237 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.233+0000 7fa93b7fe640 1 --2- 172.21.15.160:0/1681665051 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa93c100570 0x7fa93c06a7a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa92c000c00 tx=0x7fa92c02c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:05.237 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.233+0000 7fa938ff9640 1 -- 172.21.15.160:0/1681665051 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92c043020 con 0x7fa93c100570 2024-03-07T07:41:05.237 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.233+0000 7fa938ff9640 1 -- 172.21.15.160:0/1681665051 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa92c0054e0 con 0x7fa93c100570 2024-03-07T07:41:05.237 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.233+0000 7fa941c1e640 1 -- 172.21.15.160:0/1681665051 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa93c19f8a0 con 0x7fa93c100570 2024-03-07T07:41:05.237 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.233+0000 7fa938ff9640 1 -- 172.21.15.160:0/1681665051 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92c003bd0 con 0x7fa93c100570 2024-03-07T07:41:05.237 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.233+0000 7fa941c1e640 1 -- 172.21.15.160:0/1681665051 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa93c19fdf0 con 0x7fa93c100570 2024-03-07T07:41:05.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.235+0000 7fa938ff9640 1 -- 172.21.15.160:0/1681665051 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 136754+0+0 (secure 0 0 0) 0x7fa92c046020 con 0x7fa93c100570 2024-03-07T07:41:05.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.235+0000 7fa944a0a740 1 Processor -- start 2024-03-07T07:41:05.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.235+0000 7fa938ff9640 1 --2- 172.21.15.160:0/1681665051 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa9240a1d80 0x7fa9240a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.235+0000 7fa944a0a740 1 -- start start 2024-03-07T07:41:05.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.236+0000 7fa93bfff640 1 --2- 172.21.15.160:0/1681665051 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa9240a1d80 0x7fa9240a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.236+0000 7fa944a0a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5636d2b584c0 0x5636d2b588e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.236+0000 7fa944a0a740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5636d2b58eb0 0x5636d2b5d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.236+0000 7fa944a0a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5636d2b5e020 0x5636d2b604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.236+0000 7fa938ff9640 1 -- 172.21.15.160:0/1681665051 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa92c0efdc0 con 0x7fa93c100570 2024-03-07T07:41:05.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.236+0000 7fa944a0a740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636d2ab9790 con 0x5636d2b58eb0 2024-03-07T07:41:05.248 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.236+0000 7fa944a0a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636d2a62b20 con 0x5636d2b584c0 2024-03-07T07:41:05.248 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.236+0000 7fa944a0a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636d2ab5830 con 0x5636d2b5e020 2024-03-07T07:41:05.248 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.236+0000 7fa93affd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5636d2b58eb0 0x5636d2b5d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.248 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.236+0000 7fa93b7fe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5636d2b584c0 0x5636d2b588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.248 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.236+0000 7fa93affd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5636d2b58eb0 0x5636d2b5d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57926/0 (socket says 172.21.15.160:57926) 2024-03-07T07:41:05.248 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.236+0000 7fa93b7fe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5636d2b584c0 0x5636d2b588e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.160:38472/0 (socket says 172.21.15.160:38472) 2024-03-07T07:41:05.248 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.236+0000 7fa93affd640 1 -- 172.21.15.160:0/865950223 learned_addr learned my addr 172.21.15.160:0/865950223 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:41:05.248 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.236+0000 7fa93bfff640 1 --2- 172.21.15.160:0/865950223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5636d2b5e020 0x5636d2b604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.248 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.236+0000 7fa93b7fe640 1 -- 172.21.15.160:0/865950223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5636d2b5e020 msgr2=0x5636d2b604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:41:05.248 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.236+0000 7fa93b7fe640 1 --2- 172.21.15.160:0/865950223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5636d2b5e020 0x5636d2b604b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.248 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.236+0000 7fa93b7fe640 1 -- 172.21.15.160:0/865950223 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5636d2b58eb0 msgr2=0x5636d2b5d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:41:05.248 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.236+0000 7fa93b7fe640 1 --2- 172.21.15.160:0/865950223 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5636d2b58eb0 0x5636d2b5d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.248 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.236+0000 7fa93b7fe640 1 -- 172.21.15.160:0/865950223 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636d2a76970 con 0x5636d2b584c0 2024-03-07T07:41:05.248 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.236+0000 7fa93bfff640 1 --2- 172.21.15.160:0/1681665051 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa9240a1d80 0x7fa9240a4240 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa934005fd0 tx=0x7fa93400ff40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:05.249 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.236+0000 7fa93b7fe640 1 --2- 172.21.15.160:0/865950223 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5636d2b584c0 0x5636d2b588e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa92c006010 tx=0x7fa92c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=506e82024c7a3553 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:05.249 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.236+0000 7fa921ffb640 1 -- 172.21.15.160:0/865950223 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92c0d5070 con 0x5636d2b584c0 2024-03-07T07:41:05.249 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.236+0000 7fa921ffb640 1 -- 172.21.15.160:0/865950223 <== mon.2 v2:172.21.15.177:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa92c005690 con 0x5636d2b584c0 2024-03-07T07:41:05.249 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.236+0000 7fa921ffb640 1 -- 172.21.15.160:0/865950223 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92c003d70 con 0x5636d2b584c0 2024-03-07T07:41:05.249 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.237+0000 7fa944a0a740 1 -- 172.21.15.160:0/865950223 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5636d2b584c0 msgr2=0x5636d2b588e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:05.249 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.237+0000 7fa944a0a740 1 --2- 172.21.15.160:0/865950223 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5636d2b584c0 0x5636d2b588e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa92c006010 tx=0x7fa92c009260 comp rx=0 tx=0).stop 2024-03-07T07:41:05.249 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.237+0000 7fa944a0a740 1 -- 172.21.15.160:0/865950223 shutdown_connections 2024-03-07T07:41:05.249 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.237+0000 7fa944a0a740 1 --2- 172.21.15.160:0/865950223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5636d2b5e020 0x5636d2b604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.249 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.237+0000 7fa944a0a740 1 --2- 172.21.15.160:0/865950223 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5636d2b58eb0 0x5636d2b5d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.249 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.237+0000 7fa944a0a740 1 --2- 172.21.15.160:0/865950223 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5636d2b584c0 0x5636d2b588e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.249 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.237+0000 7fa944a0a740 1 -- 172.21.15.160:0/865950223 >> 172.21.15.160:0/865950223 conn(0x5636d2b1c3d0 msgr2=0x5636d2b4fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:05.249 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.237+0000 7fa944a0a740 1 -- 172.21.15.160:0/865950223 shutdown_connections 2024-03-07T07:41:05.249 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.237+0000 7fa944a0a740 1 -- 172.21.15.160:0/865950223 wait complete. 2024-03-07T07:41:05.249 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.237+0000 7fa944a0a740 1 -- start start 2024-03-07T07:41:05.250 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.238+0000 7fa944a0a740 1 Processor -- start 2024-03-07T07:41:05.250 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.238+0000 7fa944a0a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5636d2b584c0 0x5636d2b62720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.250 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.238+0000 7fa944a0a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5636d2b58eb0 0x5636d2b62c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.250 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.238+0000 7fa944a0a740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5636d2b5e020 0x5636d2b69ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.250 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.238+0000 7fa944a0a740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636d2b60d90 con 0x5636d2b5e020 2024-03-07T07:41:05.250 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.238+0000 7fa944a0a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636d2a75f60 con 0x5636d2b58eb0 2024-03-07T07:41:05.250 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.238+0000 7fa944a0a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636d2b60f10 con 0x5636d2b584c0 2024-03-07T07:41:05.250 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.238+0000 7fa944a0a740 10 client.0 _ll_register_callbacks cb 0x5636d2ab13d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-07T07:41:05.250 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.238+0000 7fa93b7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5636d2b584c0 0x5636d2b62720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.250 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.238+0000 7fa93bfff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5636d2b5e020 0x5636d2b69ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.250 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.238+0000 7fa93b7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5636d2b584c0 0x5636d2b62720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.160:47440/0 (socket says 172.21.15.160:47440) 2024-03-07T07:41:05.250 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.238+0000 7fa93b7fe640 1 -- 172.21.15.160:0/1596794171 learned_addr learned my addr 172.21.15.160:0/1596794171 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:41:05.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.238+0000 7fa93affd640 1 --2- 172.21.15.160:0/1596794171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5636d2b58eb0 0x5636d2b62c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.238+0000 7fa93bfff640 1 -- 172.21.15.160:0/1596794171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5636d2b584c0 msgr2=0x5636d2b62720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:41:05.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.238+0000 7fa93bfff640 1 --2- 172.21.15.160:0/1596794171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5636d2b584c0 0x5636d2b62720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.238+0000 7fa93bfff640 1 -- 172.21.15.160:0/1596794171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5636d2b58eb0 msgr2=0x5636d2b62c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:41:05.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.238+0000 7fa93bfff640 1 --2- 172.21.15.160:0/1596794171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5636d2b58eb0 0x5636d2b62c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.238+0000 7fa93bfff640 1 -- 172.21.15.160:0/1596794171 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5636d27ce2d0 con 0x5636d2b5e020 2024-03-07T07:41:05.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.238+0000 7fa93b7fe640 1 --2- 172.21.15.160:0/1596794171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5636d2b584c0 0x5636d2b62720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:41:05.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.238+0000 7fa93bfff640 1 --2- 172.21.15.160:0/1596794171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5636d2b5e020 0x5636d2b69ca0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa9340099a0 tx=0x7fa934004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=255335b8de8b9a6a server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:05.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.238+0000 7fa91affd640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-03-07T07:41:05.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.238+0000 7fa91affd640 1 -- 172.21.15.160:0/1596794171 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93401a040 con 0x5636d2b5e020 2024-03-07T07:41:05.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.238+0000 7fa91affd640 1 -- 172.21.15.160:0/1596794171 <== mon.0 v2:172.21.15.160:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa934004d20 con 0x5636d2b5e020 2024-03-07T07:41:05.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.238+0000 7fa944a0a740 1 -- 172.21.15.160:0/1596794171 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5636d2ac6cd0 con 0x5636d2b5e020 2024-03-07T07:41:05.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.239+0000 7fa91affd640 1 -- 172.21.15.160:0/1596794171 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa934020660 con 0x5636d2b5e020 2024-03-07T07:41:05.251 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.239+0000 7fa944a0a740 10 client.15225 fetch_fsmap learned FSMap version 21 2024-03-07T07:41:05.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.239+0000 7fa944a0a740 1 -- 172.21.15.160:0/1596794171 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x5636d28986b0 con 0x5636d2b5e020 2024-03-07T07:41:05.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.239+0000 7fa91affd640 1 -- 172.21.15.160:0/1596794171 <== mon.0 v2:172.21.15.160:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5636d2ac6cd0 con 0x5636d2b5e020 2024-03-07T07:41:05.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.239+0000 7fa91affd640 1 -- 172.21.15.160:0/1596794171 <== mon.0 v2:172.21.15.160:3300/0 5 ==== fsmap(e 21) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fa93401d030 con 0x5636d2b5e020 2024-03-07T07:41:05.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.239+0000 7fa944a0a740 10 client.15225 fetch_fsmap finished waiting for FSMap version 21 2024-03-07T07:41:05.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.239+0000 7fa944a0a740 10 client.15225 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-07T07:41:05.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.239+0000 7fa944a0a740 20 client.15225 populate_metadata read hostname 'smithi160' 2024-03-07T07:41:05.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.239+0000 7fa944a0a740 1 --2- 172.21.15.160:0/1596794171 >> [v2:172.21.15.177:6838/3329773838,v1:172.21.15.177:6839/3329773838] conn(0x5636d2b6f280 0x5636d2b6e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.239+0000 7fa944a0a740 4 client.15225 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-07T07:41:05.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.239+0000 7fa944a0a740 1 -- 172.21.15.160:0/1596794171 --> [v2:172.21.15.177:6838/3329773838,v1:172.21.15.177:6839/3329773838] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5636d2ac1020 con 0x5636d2b6f280 2024-03-07T07:41:05.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.239+0000 7fa93b7fe640 1 --2- 172.21.15.160:0/1596794171 >> [v2:172.21.15.177:6838/3329773838,v1:172.21.15.177:6839/3329773838] conn(0x5636d2b6f280 0x5636d2b6e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.240+0000 7fa91affd640 10 client.15225 ms_handle_connect on v2:172.21.15.177:6838/3329773838 2024-03-07T07:41:05.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.240+0000 7fa93b7fe640 1 --2- 172.21.15.160:0/1596794171 >> [v2:172.21.15.177:6838/3329773838,v1:172.21.15.177:6839/3329773838] conn(0x5636d2b6f280 0x5636d2b6e370 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4cc4e39a71632f81 server_cookie=504d9716eac361c in_seq=0 out_seq=0 2024-03-07T07:41:05.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.240+0000 7fa91affd640 1 -- 172.21.15.160:0/1596794171 <== mds.1 v2:172.21.15.177:6838/3329773838 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5636d2ac1020 con 0x5636d2b6f280 2024-03-07T07:41:05.252 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.240+0000 7fa91affd640 10 client.15225 handle_command_reply: tid=0 2024-03-07T07:41:05.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.240+0000 7fa944a0a740 1 client.15225 shutdown 2024-03-07T07:41:05.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.240+0000 7fa944a0a740 1 -- 172.21.15.160:0/1596794171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5636d2b5e020 msgr2=0x5636d2b69ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:05.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.240+0000 7fa944a0a740 1 --2- 172.21.15.160:0/1596794171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5636d2b5e020 0x5636d2b69ca0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa9340099a0 tx=0x7fa934004ab0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.240+0000 7fa944a0a740 1 -- 172.21.15.160:0/1596794171 shutdown_connections 2024-03-07T07:41:05.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.240+0000 7fa944a0a740 1 --2- 172.21.15.160:0/1596794171 >> [v2:172.21.15.177:6838/3329773838,v1:172.21.15.177:6839/3329773838] conn(0x5636d2b6f280 0x5636d2b6e370 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.240+0000 7fa944a0a740 1 --2- 172.21.15.160:0/1596794171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5636d2b5e020 0x5636d2b69ca0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.240+0000 7fa944a0a740 1 --2- 172.21.15.160:0/1596794171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5636d2b58eb0 0x5636d2b62c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.240+0000 7fa91affd640 0 client.15225 ms_handle_reset on v2:172.21.15.177:6838/3329773838 2024-03-07T07:41:05.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.240+0000 7fa944a0a740 1 --2- 172.21.15.160:0/1596794171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5636d2b584c0 0x5636d2b62720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.240+0000 7fa944a0a740 1 -- 172.21.15.160:0/1596794171 >> 172.21.15.160:0/1596794171 conn(0x5636d2b1c3d0 msgr2=0x5636d2b4ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:05.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.241+0000 7fa944a0a740 1 -- 172.21.15.160:0/1596794171 shutdown_connections 2024-03-07T07:41:05.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.241+0000 7fa944a0a740 1 -- 172.21.15.160:0/1596794171 wait complete. 2024-03-07T07:41:05.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.241+0000 7fa944a0a740 20 client.15225 trim_cache size 0 max 16384 2024-03-07T07:41:05.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.257+0000 7fa944a0a740 1 Processor -- start 2024-03-07T07:41:05.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.257+0000 7fa944a0a740 1 -- start start 2024-03-07T07:41:05.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.257+0000 7fa944a0a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5636d2b584c0 0x5636d2b61240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.257+0000 7fa944a0a740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5636d2b58eb0 0x5636d2b61780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.257+0000 7fa944a0a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5636d2b5e020 0x5636d2b69c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.257+0000 7fa944a0a740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636d2b6eaa0 con 0x5636d2b58eb0 2024-03-07T07:41:05.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.257+0000 7fa944a0a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636d2b6f100 con 0x5636d2b5e020 2024-03-07T07:41:05.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.257+0000 7fa944a0a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636d2b6f760 con 0x5636d2b584c0 2024-03-07T07:41:05.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.257+0000 7fa93affd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5636d2b58eb0 0x5636d2b61780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.268 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.257+0000 7fa93affd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5636d2b58eb0 0x5636d2b61780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57950/0 (socket says 172.21.15.160:57950) 2024-03-07T07:41:05.268 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.257+0000 7fa93affd640 1 -- 172.21.15.160:0/1698208641 learned_addr learned my addr 172.21.15.160:0/1698208641 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:41:05.268 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.257+0000 7fa93b7fe640 1 --2- 172.21.15.160:0/1698208641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5636d2b584c0 0x5636d2b61240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.268 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.257+0000 7fa93bfff640 1 --2- 172.21.15.160:0/1698208641 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5636d2b5e020 0x5636d2b69c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.268 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.257+0000 7fa93affd640 1 -- 172.21.15.160:0/1698208641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5636d2b584c0 msgr2=0x5636d2b61240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:41:05.268 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.258+0000 7fa93affd640 1 --2- 172.21.15.160:0/1698208641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5636d2b584c0 0x5636d2b61240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.268 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.258+0000 7fa93affd640 1 -- 172.21.15.160:0/1698208641 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5636d2b5e020 msgr2=0x5636d2b69c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:41:05.268 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.258+0000 7fa93affd640 1 --2- 172.21.15.160:0/1698208641 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5636d2b5e020 0x5636d2b69c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.268 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.258+0000 7fa93affd640 1 -- 172.21.15.160:0/1698208641 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636d2b5dd20 con 0x5636d2b58eb0 2024-03-07T07:41:05.268 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.258+0000 7fa93affd640 1 --2- 172.21.15.160:0/1698208641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5636d2b58eb0 0x5636d2b61780 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa9300127d0 tx=0x7fa930005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1011c928bf143157 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:05.268 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.258+0000 7fa9217fa640 1 -- 172.21.15.160:0/1698208641 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9300120d0 con 0x5636d2b58eb0 2024-03-07T07:41:05.268 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.258+0000 7fa9217fa640 1 -- 172.21.15.160:0/1698208641 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa930012270 con 0x5636d2b58eb0 2024-03-07T07:41:05.268 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.258+0000 7fa9217fa640 1 -- 172.21.15.160:0/1698208641 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9300047a0 con 0x5636d2b58eb0 2024-03-07T07:41:05.269 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.258+0000 7fa944a0a740 1 -- 172.21.15.160:0/1698208641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5636d2b58eb0 msgr2=0x5636d2b61780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:05.269 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.258+0000 7fa944a0a740 1 --2- 172.21.15.160:0/1698208641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5636d2b58eb0 0x5636d2b61780 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa9300127d0 tx=0x7fa930005f90 comp rx=0 tx=0).stop 2024-03-07T07:41:05.269 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.258+0000 7fa944a0a740 1 -- 172.21.15.160:0/1698208641 shutdown_connections 2024-03-07T07:41:05.269 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.258+0000 7fa944a0a740 1 --2- 172.21.15.160:0/1698208641 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5636d2b5e020 0x5636d2b69c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.269 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.258+0000 7fa944a0a740 1 --2- 172.21.15.160:0/1698208641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5636d2b58eb0 0x5636d2b61780 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.269 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.258+0000 7fa944a0a740 1 --2- 172.21.15.160:0/1698208641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5636d2b584c0 0x5636d2b61240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.269 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.258+0000 7fa944a0a740 1 -- 172.21.15.160:0/1698208641 >> 172.21.15.160:0/1698208641 conn(0x5636d2b1c3d0 msgr2=0x5636d2b5a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:05.269 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.258+0000 7fa944a0a740 1 -- 172.21.15.160:0/1698208641 shutdown_connections 2024-03-07T07:41:05.269 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.259+0000 7fa944a0a740 1 -- 172.21.15.160:0/1698208641 wait complete. 2024-03-07T07:41:05.269 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.259+0000 7fa944a0a740 1 -- start start 2024-03-07T07:41:05.269 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.259+0000 7fa944a0a740 1 Processor -- start 2024-03-07T07:41:05.269 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.259+0000 7fa944a0a740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5636d2b584c0 0x5636d2b65f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.269 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.259+0000 7fa944a0a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5636d2b58eb0 0x5636d2b66470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.269 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.259+0000 7fa944a0a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5636d2b5e020 0x5636d2b69490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.269 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.259+0000 7fa944a0a740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636d2b70420 con 0x5636d2b584c0 2024-03-07T07:41:05.270 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.259+0000 7fa944a0a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636d2b6fdc0 con 0x5636d2b5e020 2024-03-07T07:41:05.270 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.259+0000 7fa944a0a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636d2b70a80 con 0x5636d2b58eb0 2024-03-07T07:41:05.270 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.259+0000 7fa944a0a740 10 client.0 _ll_register_callbacks cb 0x7fa93400f820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-07T07:41:05.270 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.259+0000 7fa93b7fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5636d2b584c0 0x5636d2b65f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.270 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.259+0000 7fa93b7fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5636d2b584c0 0x5636d2b65f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57952/0 (socket says 172.21.15.160:57952) 2024-03-07T07:41:05.270 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.259+0000 7fa93b7fe640 1 -- 172.21.15.160:0/3776822452 learned_addr learned my addr 172.21.15.160:0/3776822452 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:41:05.270 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.259+0000 7fa93affd640 1 --2- 172.21.15.160:0/3776822452 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5636d2b58eb0 0x5636d2b66470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.270 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.259+0000 7fa93bfff640 1 --2- 172.21.15.160:0/3776822452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5636d2b5e020 0x5636d2b69490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.270 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.260+0000 7fa93b7fe640 1 -- 172.21.15.160:0/3776822452 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5636d2b58eb0 msgr2=0x5636d2b66470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:41:05.270 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.260+0000 7fa93b7fe640 1 --2- 172.21.15.160:0/3776822452 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5636d2b58eb0 0x5636d2b66470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.270 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.260+0000 7fa93b7fe640 1 -- 172.21.15.160:0/3776822452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5636d2b5e020 msgr2=0x5636d2b69490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:41:05.270 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.260+0000 7fa93b7fe640 1 --2- 172.21.15.160:0/3776822452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5636d2b5e020 0x5636d2b69490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.270 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.260+0000 7fa93b7fe640 1 -- 172.21.15.160:0/3776822452 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5636d2b69a80 con 0x5636d2b584c0 2024-03-07T07:41:05.270 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.260+0000 7fa93b7fe640 1 --2- 172.21.15.160:0/3776822452 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5636d2b584c0 0x5636d2b65f30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa92c0b9470 tx=0x7fa92c0c5290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d6ffd69510c9cd8 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:05.271 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.260+0000 7fa91affd640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-03-07T07:41:05.271 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.260+0000 7fa91affd640 1 -- 172.21.15.160:0/3776822452 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92c0d5040 con 0x5636d2b584c0 2024-03-07T07:41:05.271 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.260+0000 7fa91affd640 1 -- 172.21.15.160:0/3776822452 <== mon.0 v2:172.21.15.160:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa92c004c60 con 0x5636d2b584c0 2024-03-07T07:41:05.271 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.260+0000 7fa944a0a740 1 -- 172.21.15.160:0/3776822452 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa930012270 con 0x5636d2b584c0 2024-03-07T07:41:05.271 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.260+0000 7fa91affd640 1 -- 172.21.15.160:0/3776822452 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa92c0c0c40 con 0x5636d2b584c0 2024-03-07T07:41:05.271 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.260+0000 7fa944a0a740 10 client.15231 fetch_fsmap learned FSMap version 21 2024-03-07T07:41:05.271 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.260+0000 7fa944a0a740 1 -- 172.21.15.160:0/3776822452 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x5636d2bcade0 con 0x5636d2b584c0 2024-03-07T07:41:05.271 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.260+0000 7fa91affd640 1 -- 172.21.15.160:0/3776822452 <== mon.0 v2:172.21.15.160:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa930012270 con 0x5636d2b584c0 2024-03-07T07:41:05.271 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.260+0000 7fa91affd640 1 -- 172.21.15.160:0/3776822452 <== mon.0 v2:172.21.15.160:3300/0 5 ==== fsmap(e 21) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fa92c0c4e00 con 0x5636d2b584c0 2024-03-07T07:41:05.271 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.260+0000 7fa944a0a740 10 client.15231 fetch_fsmap finished waiting for FSMap version 21 2024-03-07T07:41:05.271 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.260+0000 7fa944a0a740 10 client.15231 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-07T07:41:05.271 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.260+0000 7fa944a0a740 20 client.15231 populate_metadata read hostname 'smithi160' 2024-03-07T07:41:05.271 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.260+0000 7fa944a0a740 1 --2- 172.21.15.160:0/3776822452 >> [v2:172.21.15.177:6838/3329773838,v1:172.21.15.177:6839/3329773838] conn(0x5636d2bcf100 0x5636d2bcae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.271 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.260+0000 7fa944a0a740 4 client.15231 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-07T07:41:05.271 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.260+0000 7fa944a0a740 1 -- 172.21.15.160:0/3776822452 --> [v2:172.21.15.177:6838/3329773838,v1:172.21.15.177:6839/3329773838] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5636d2a3f970 con 0x5636d2bcf100 2024-03-07T07:41:05.272 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.260+0000 7fa93affd640 1 --2- 172.21.15.160:0/3776822452 >> [v2:172.21.15.177:6838/3329773838,v1:172.21.15.177:6839/3329773838] conn(0x5636d2bcf100 0x5636d2bcae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.272 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.261+0000 7fa93affd640 1 --2- 172.21.15.160:0/3776822452 >> [v2:172.21.15.177:6838/3329773838,v1:172.21.15.177:6839/3329773838] conn(0x5636d2bcf100 0x5636d2bcae30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=69acd2527b96b7fd server_cookie=ad65707087ded112 in_seq=0 out_seq=0 2024-03-07T07:41:05.272 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.261+0000 7fa91affd640 10 client.15231 ms_handle_connect on v2:172.21.15.177:6838/3329773838 2024-03-07T07:41:05.272 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.261+0000 7fa91affd640 1 -- 172.21.15.160:0/3776822452 <== mds.1 v2:172.21.15.177:6838/3329773838 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5636d2a3f970 con 0x5636d2bcf100 2024-03-07T07:41:05.272 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.261+0000 7fa91affd640 10 client.15231 handle_command_reply: tid=0 2024-03-07T07:41:05.272 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.261+0000 7fa944a0a740 1 client.15231 shutdown 2024-03-07T07:41:05.272 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.261+0000 7fa944a0a740 1 -- 172.21.15.160:0/3776822452 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5636d2b584c0 msgr2=0x5636d2b65f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:05.272 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.261+0000 7fa944a0a740 1 --2- 172.21.15.160:0/3776822452 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5636d2b584c0 0x5636d2b65f30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa92c0b9470 tx=0x7fa92c0c5290 comp rx=0 tx=0).stop 2024-03-07T07:41:05.272 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.261+0000 7fa944a0a740 1 -- 172.21.15.160:0/3776822452 shutdown_connections 2024-03-07T07:41:05.272 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.261+0000 7fa944a0a740 1 --2- 172.21.15.160:0/3776822452 >> [v2:172.21.15.177:6838/3329773838,v1:172.21.15.177:6839/3329773838] conn(0x5636d2bcf100 0x5636d2bcae30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.272 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.261+0000 7fa944a0a740 1 --2- 172.21.15.160:0/3776822452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5636d2b5e020 0x5636d2b69490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.272 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.261+0000 7fa944a0a740 1 --2- 172.21.15.160:0/3776822452 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5636d2b58eb0 0x5636d2b66470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.272 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.261+0000 7fa91affd640 0 client.15231 ms_handle_reset on v2:172.21.15.177:6838/3329773838 2024-03-07T07:41:05.272 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.261+0000 7fa944a0a740 1 --2- 172.21.15.160:0/3776822452 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5636d2b584c0 0x5636d2b65f30 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.273 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.261+0000 7fa944a0a740 1 -- 172.21.15.160:0/3776822452 >> 172.21.15.160:0/3776822452 conn(0x5636d2b1c3d0 msgr2=0x5636d2b4f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:05.273 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.261+0000 7fa944a0a740 1 -- 172.21.15.160:0/3776822452 shutdown_connections 2024-03-07T07:41:05.273 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.261+0000 7fa944a0a740 1 -- 172.21.15.160:0/3776822452 wait complete. 2024-03-07T07:41:05.273 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.261+0000 7fa944a0a740 20 client.15231 trim_cache size 0 max 16384 2024-03-07T07:41:05.273 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-03-07T07:41:05.273 INFO:teuthology.orchestra.run.smithi160.stdout: "cluster_fsid": "826f6582-dc54-11ee-95c6-87774f69a715", 2024-03-07T07:41:05.273 INFO:teuthology.orchestra.run.smithi160.stdout: "whoami": 1, 2024-03-07T07:41:05.273 INFO:teuthology.orchestra.run.smithi160.stdout: "id": 24497, 2024-03-07T07:41:05.273 INFO:teuthology.orchestra.run.smithi160.stdout: "want_state": "up:active", 2024-03-07T07:41:05.273 INFO:teuthology.orchestra.run.smithi160.stdout: "state": "up:active", 2024-03-07T07:41:05.274 INFO:teuthology.orchestra.run.smithi160.stdout: "fs_name": "cephfs", 2024-03-07T07:41:05.274 INFO:teuthology.orchestra.run.smithi160.stdout: "rank_uptime": 108.413835632, 2024-03-07T07:41:05.274 INFO:teuthology.orchestra.run.smithi160.stdout: "mdsmap_epoch": 21, 2024-03-07T07:41:05.274 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_epoch": 87, 2024-03-07T07:41:05.274 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_epoch_barrier": 85, 2024-03-07T07:41:05.274 INFO:teuthology.orchestra.run.smithi160.stdout: "uptime": 117.52849806899999 2024-03-07T07:41:05.274 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-03-07T07:41:05.275 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.262+0000 7fa941c1e640 1 -- 172.21.15.160:0/1681665051 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa9240a1d80 msgr2=0x7fa9240a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:05.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.262+0000 7fa941c1e640 1 --2- 172.21.15.160:0/1681665051 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa9240a1d80 0x7fa9240a4240 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa934005fd0 tx=0x7fa93400ff40 comp rx=0 tx=0).stop 2024-03-07T07:41:05.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.262+0000 7fa941c1e640 1 -- 172.21.15.160:0/1681665051 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa93c100570 msgr2=0x7fa93c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:05.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.262+0000 7fa941c1e640 1 --2- 172.21.15.160:0/1681665051 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa93c100570 0x7fa93c06a7a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa92c000c00 tx=0x7fa92c02c650 comp rx=0 tx=0).stop 2024-03-07T07:41:05.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.262+0000 7fa941c1e640 1 -- 172.21.15.160:0/1681665051 shutdown_connections 2024-03-07T07:41:05.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.262+0000 7fa941c1e640 1 --2- 172.21.15.160:0/1681665051 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa9240a1d80 0x7fa9240a4240 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.262+0000 7fa941c1e640 1 --2- 172.21.15.160:0/1681665051 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa93c102120 0x7fa93c19ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.262+0000 7fa941c1e640 1 --2- 172.21.15.160:0/1681665051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa93c101760 0x7fa93c19c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.262+0000 7fa941c1e640 1 --2- 172.21.15.160:0/1681665051 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa93c100570 0x7fa93c06a7a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.262+0000 7fa941c1e640 1 -- 172.21.15.160:0/1681665051 >> 172.21.15.160:0/1681665051 conn(0x7fa93c0fbb00 msgr2=0x7fa93c104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:05.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.262+0000 7fa941c1e640 1 -- 172.21.15.160:0/1681665051 shutdown_connections 2024-03-07T07:41:05.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.262+0000 7fa941c1e640 1 -- 172.21.15.160:0/1681665051 wait complete. 2024-03-07T07:41:05.277 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-07T07:41:05.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.380+0000 7fe5e8683640 1 -- 172.21.15.160:0/352775666 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5e0102130 msgr2=0x7fe5e01089c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:05.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.380+0000 7fe5e8683640 1 --2- 172.21.15.160:0/352775666 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5e0102130 0x7fe5e01089c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8009ca0 tx=0x7fe5d802c0f0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.380+0000 7fe5e8683640 1 -- 172.21.15.160:0/352775666 shutdown_connections 2024-03-07T07:41:05.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.380+0000 7fe5e8683640 1 --2- 172.21.15.160:0/352775666 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5e0102130 0x7fe5e01089c0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.380+0000 7fe5e8683640 1 --2- 172.21.15.160:0/352775666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5e0101770 0x7fe5e0101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.380+0000 7fe5e8683640 1 --2- 172.21.15.160:0/352775666 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5e0100580 0x7fe5e01009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.380+0000 7fe5e8683640 1 -- 172.21.15.160:0/352775666 >> 172.21.15.160:0/352775666 conn(0x7fe5e00fbb10 msgr2=0x7fe5e00fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:05.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.380+0000 7fe5e8683640 1 -- 172.21.15.160:0/352775666 shutdown_connections 2024-03-07T07:41:05.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.380+0000 7fe5e8683640 1 -- 172.21.15.160:0/352775666 wait complete. 2024-03-07T07:41:05.383 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.381+0000 7fe5e8683640 1 Processor -- start 2024-03-07T07:41:05.383 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.381+0000 7fe5e8683640 1 -- start start 2024-03-07T07:41:05.383 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.381+0000 7fe5e8683640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5e0100580 0x7fe5e006a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.383 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.381+0000 7fe5e8683640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5e0101770 0x7fe5e019c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.383 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.381+0000 7fe5e8683640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5e0102130 0x7fe5e019ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.383 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.381+0000 7fe5e8683640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e010c1e0 con 0x7fe5e0101770 2024-03-07T07:41:05.383 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.381+0000 7fe5e8683640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e010c060 con 0x7fe5e0100580 2024-03-07T07:41:05.383 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.381+0000 7fe5e8683640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e010c360 con 0x7fe5e0102130 2024-03-07T07:41:05.383 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.381+0000 7fe5e5c1e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5e0101770 0x7fe5e019c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.383 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.381+0000 7fe5e641f640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5e0100580 0x7fe5e006a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.383 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.381+0000 7fe5e5c1e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5e0101770 0x7fe5e019c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57962/0 (socket says 172.21.15.160:57962) 2024-03-07T07:41:05.383 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.381+0000 7fe5e5c1e640 1 -- 172.21.15.160:0/3875036939 learned_addr learned my addr 172.21.15.160:0/3875036939 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:41:05.384 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.381+0000 7fe5e6c20640 1 --2- 172.21.15.160:0/3875036939 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5e0102130 0x7fe5e019ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.384 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.381+0000 7fe5e5c1e640 1 -- 172.21.15.160:0/3875036939 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5e0102130 msgr2=0x7fe5e019ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:05.384 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.381+0000 7fe5e5c1e640 1 --2- 172.21.15.160:0/3875036939 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5e0102130 0x7fe5e019ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.384 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.381+0000 7fe5e5c1e640 1 -- 172.21.15.160:0/3875036939 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5e0100580 msgr2=0x7fe5e006a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:05.384 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.381+0000 7fe5e5c1e640 1 --2- 172.21.15.160:0/3875036939 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5e0100580 0x7fe5e006a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.384 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.381+0000 7fe5e5c1e640 1 -- 172.21.15.160:0/3875036939 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e019f540 con 0x7fe5e0101770 2024-03-07T07:41:05.384 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.382+0000 7fe5e5c1e640 1 --2- 172.21.15.160:0/3875036939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5e0101770 0x7fe5e019c610 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe5d000e9f0 tx=0x7fe5d000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:05.385 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.384+0000 7fe5d77fe640 1 -- 172.21.15.160:0/3875036939 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d000cd80 con 0x7fe5e0101770 2024-03-07T07:41:05.385 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.384+0000 7fe5d77fe640 1 -- 172.21.15.160:0/3875036939 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe5d0010910 con 0x7fe5e0101770 2024-03-07T07:41:05.385 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.384+0000 7fe5d77fe640 1 -- 172.21.15.160:0/3875036939 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0018990 con 0x7fe5e0101770 2024-03-07T07:41:05.385 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.384+0000 7fe5e8683640 1 -- 172.21.15.160:0/3875036939 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5e019f830 con 0x7fe5e0101770 2024-03-07T07:41:05.385 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.384+0000 7fe5e8683640 1 -- 172.21.15.160:0/3875036939 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5e019fdd0 con 0x7fe5e0101770 2024-03-07T07:41:05.386 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.386+0000 7fe5d77fe640 1 -- 172.21.15.160:0/3875036939 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 136754+0+0 (secure 0 0 0) 0x7fe5d0010ac0 con 0x7fe5e0101770 2024-03-07T07:41:05.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.387+0000 7fe5eb465740 1 Processor -- start 2024-03-07T07:41:05.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.387+0000 7fe5d77fe640 1 --2- 172.21.15.160:0/3875036939 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fe5cc0a1e00 0x7fe5cc0a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.387+0000 7fe5e6c20640 1 --2- 172.21.15.160:0/3875036939 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fe5cc0a1e00 0x7fe5cc0a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.387+0000 7fe5d77fe640 1 -- 172.21.15.160:0/3875036939 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe5d00cdfb0 con 0x7fe5e0101770 2024-03-07T07:41:05.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.387+0000 7fe5eb465740 1 -- start start 2024-03-07T07:41:05.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.387+0000 7fe5eb465740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ea4ac10b0 0x562ea4ac14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.387+0000 7fe5eb465740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562ea4ac1aa0 0x562ea4ac65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.387+0000 7fe5eb465740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ea4ac6c10 0x562ea4ac90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.387+0000 7fe5eb465740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ea4a26790 con 0x562ea4ac1aa0 2024-03-07T07:41:05.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.387+0000 7fe5eb465740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ea49cfb20 con 0x562ea4ac6c10 2024-03-07T07:41:05.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.387+0000 7fe5eb465740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ea4a22830 con 0x562ea4ac10b0 2024-03-07T07:41:05.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.387+0000 7fe5e5c1e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562ea4ac1aa0 0x562ea4ac65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.387+0000 7fe5e5c1e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562ea4ac1aa0 0x562ea4ac65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57978/0 (socket says 172.21.15.160:57978) 2024-03-07T07:41:05.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.387+0000 7fe5e641f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ea4ac10b0 0x562ea4ac14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.387+0000 7fe5e5c1e640 1 -- 172.21.15.160:0/2564661909 learned_addr learned my addr 172.21.15.160:0/2564661909 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:41:05.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.387+0000 7fe5e6c20640 1 --2- 172.21.15.160:0/3875036939 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fe5cc0a1e00 0x7fe5cc0a42c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8005e30 tx=0x7fe5d8032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:05.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.387+0000 7fe5e6c20640 1 --2- 172.21.15.160:0/2564661909 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ea4ac6c10 0x562ea4ac90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.388+0000 7fe5e641f640 1 -- 172.21.15.160:0/2564661909 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ea4ac6c10 msgr2=0x562ea4ac90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:41:05.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.388+0000 7fe5e641f640 1 --2- 172.21.15.160:0/2564661909 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ea4ac6c10 0x562ea4ac90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.388+0000 7fe5e641f640 1 -- 172.21.15.160:0/2564661909 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562ea4ac1aa0 msgr2=0x562ea4ac65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:41:05.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.388+0000 7fe5e641f640 1 --2- 172.21.15.160:0/2564661909 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562ea4ac1aa0 0x562ea4ac65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.388+0000 7fe5e641f640 1 -- 172.21.15.160:0/2564661909 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ea49e3970 con 0x562ea4ac10b0 2024-03-07T07:41:05.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.388+0000 7fe5e5c1e640 1 --2- 172.21.15.160:0/2564661909 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562ea4ac1aa0 0x562ea4ac65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:41:05.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.388+0000 7fe5e641f640 1 --2- 172.21.15.160:0/2564661909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ea4ac10b0 0x562ea4ac14d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc009bf0 tx=0x7fe5dc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e4a6babf69cd680 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:05.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.388+0000 7fe5d4ff9640 1 -- 172.21.15.160:0/2564661909 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc004050 con 0x562ea4ac10b0 2024-03-07T07:41:05.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.388+0000 7fe5d4ff9640 1 -- 172.21.15.160:0/2564661909 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe5dc005500 con 0x562ea4ac10b0 2024-03-07T07:41:05.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.388+0000 7fe5d4ff9640 1 -- 172.21.15.160:0/2564661909 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc004d80 con 0x562ea4ac10b0 2024-03-07T07:41:05.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.389+0000 7fe5eb465740 1 -- 172.21.15.160:0/2564661909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ea4ac10b0 msgr2=0x562ea4ac14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:05.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.389+0000 7fe5eb465740 1 --2- 172.21.15.160:0/2564661909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ea4ac10b0 0x562ea4ac14d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc009bf0 tx=0x7fe5dc011810 comp rx=0 tx=0).stop 2024-03-07T07:41:05.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.389+0000 7fe5eb465740 1 -- 172.21.15.160:0/2564661909 shutdown_connections 2024-03-07T07:41:05.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.389+0000 7fe5eb465740 1 --2- 172.21.15.160:0/2564661909 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ea4ac6c10 0x562ea4ac90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.389+0000 7fe5eb465740 1 --2- 172.21.15.160:0/2564661909 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562ea4ac1aa0 0x562ea4ac65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.389+0000 7fe5eb465740 1 --2- 172.21.15.160:0/2564661909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ea4ac10b0 0x562ea4ac14d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.389+0000 7fe5eb465740 1 -- 172.21.15.160:0/2564661909 >> 172.21.15.160:0/2564661909 conn(0x562ea4a893d0 msgr2=0x562ea4abca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:05.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.389+0000 7fe5eb465740 1 -- 172.21.15.160:0/2564661909 shutdown_connections 2024-03-07T07:41:05.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.389+0000 7fe5eb465740 1 -- 172.21.15.160:0/2564661909 wait complete. 2024-03-07T07:41:05.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.389+0000 7fe5eb465740 1 -- start start 2024-03-07T07:41:05.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.389+0000 7fe5eb465740 1 Processor -- start 2024-03-07T07:41:05.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.389+0000 7fe5eb465740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ea4ac10b0 0x562ea4acb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.389+0000 7fe5eb465740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562ea4ac1aa0 0x562ea4acba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.389+0000 7fe5eb465740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ea4ac6c10 0x562ea4ad2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.389+0000 7fe5eb465740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ea4ac9bc0 con 0x562ea4ac1aa0 2024-03-07T07:41:05.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.389+0000 7fe5eb465740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ea49e2f60 con 0x562ea4ac6c10 2024-03-07T07:41:05.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.389+0000 7fe5eb465740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ea4ac9d40 con 0x562ea4ac10b0 2024-03-07T07:41:05.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.389+0000 7fe5e5c1e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562ea4ac1aa0 0x562ea4acba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.397 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.389+0000 7fe5eb465740 10 client.0 _ll_register_callbacks cb 0x562ea4a1e3d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-07T07:41:05.397 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.389+0000 7fe5e641f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ea4ac10b0 0x562ea4acb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.397 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.389+0000 7fe5e5c1e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562ea4ac1aa0 0x562ea4acba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:57994/0 (socket says 172.21.15.160:57994) 2024-03-07T07:41:05.397 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.389+0000 7fe5e5c1e640 1 -- 172.21.15.160:0/1058278822 learned_addr learned my addr 172.21.15.160:0/1058278822 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:41:05.397 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.390+0000 7fe5e6c20640 1 --2- 172.21.15.160:0/1058278822 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ea4ac6c10 0x562ea4ad2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.397 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.390+0000 7fe5e5c1e640 1 -- 172.21.15.160:0/1058278822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ea4ac10b0 msgr2=0x562ea4acb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:41:05.397 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.390+0000 7fe5e5c1e640 1 --2- 172.21.15.160:0/1058278822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ea4ac10b0 0x562ea4acb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.397 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.390+0000 7fe5e5c1e640 1 -- 172.21.15.160:0/1058278822 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ea4ac6c10 msgr2=0x562ea4ad2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:41:05.397 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.390+0000 7fe5e5c1e640 1 --2- 172.21.15.160:0/1058278822 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ea4ac6c10 0x562ea4ad2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.397 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.390+0000 7fe5e5c1e640 1 -- 172.21.15.160:0/1058278822 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ea473b2d0 con 0x562ea4ac1aa0 2024-03-07T07:41:05.397 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.390+0000 7fe5e5c1e640 1 --2- 172.21.15.160:0/1058278822 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562ea4ac1aa0 0x562ea4acba50 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0097240 tx=0x7fe5d009af70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad4cf1dff80649b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:05.397 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.390+0000 7fe5c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-03-07T07:41:05.397 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.390+0000 7fe5c9ffb640 1 -- 172.21.15.160:0/1058278822 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d00991f0 con 0x562ea4ac1aa0 2024-03-07T07:41:05.397 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.390+0000 7fe5c9ffb640 1 -- 172.21.15.160:0/1058278822 <== mon.0 v2:172.21.15.160:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe5d009a280 con 0x562ea4ac1aa0 2024-03-07T07:41:05.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.390+0000 7fe5eb465740 1 -- 172.21.15.160:0/1058278822 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe5dc005500 con 0x562ea4ac1aa0 2024-03-07T07:41:05.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.390+0000 7fe5c9ffb640 1 -- 172.21.15.160:0/1058278822 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5d009a480 con 0x562ea4ac1aa0 2024-03-07T07:41:05.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.390+0000 7fe5eb465740 10 client.15243 fetch_fsmap learned FSMap version 21 2024-03-07T07:41:05.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.390+0000 7fe5eb465740 1 -- 172.21.15.160:0/1058278822 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x562ea48056b0 con 0x562ea4ac1aa0 2024-03-07T07:41:05.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.390+0000 7fe5c9ffb640 1 -- 172.21.15.160:0/1058278822 <== mon.0 v2:172.21.15.160:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe5dc005500 con 0x562ea4ac1aa0 2024-03-07T07:41:05.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.391+0000 7fe5c9ffb640 1 -- 172.21.15.160:0/1058278822 <== mon.0 v2:172.21.15.160:3300/0 5 ==== fsmap(e 21) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fe5d00b3c40 con 0x562ea4ac1aa0 2024-03-07T07:41:05.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.391+0000 7fe5eb465740 10 client.15243 fetch_fsmap finished waiting for FSMap version 21 2024-03-07T07:41:05.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.391+0000 7fe5eb465740 10 client.15243 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-07T07:41:05.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.391+0000 7fe5eb465740 20 client.15243 populate_metadata read hostname 'smithi160' 2024-03-07T07:41:05.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.391+0000 7fe5eb465740 1 --2- 172.21.15.160:0/1058278822 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x562ea4ad80c0 0x562ea4ad7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.391+0000 7fe5eb465740 4 client.15243 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-07T07:41:05.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.391+0000 7fe5eb465740 1 -- 172.21.15.160:0/1058278822 --> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562ea4a33cd0 con 0x562ea4ad80c0 2024-03-07T07:41:05.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.391+0000 7fe5e641f640 1 --2- 172.21.15.160:0/1058278822 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x562ea4ad80c0 0x562ea4ad7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.391+0000 7fe5e641f640 1 --2- 172.21.15.160:0/1058278822 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x562ea4ad80c0 0x562ea4ad7160 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8606334d6a0dd1ec server_cookie=ce1654f67ff216a3 in_seq=0 out_seq=0 2024-03-07T07:41:05.399 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.391+0000 7fe5c9ffb640 10 client.15243 ms_handle_connect on v2:172.21.15.178:6838/2947354024 2024-03-07T07:41:05.399 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.392+0000 7fe5c9ffb640 1 -- 172.21.15.160:0/1058278822 <== mds.0 v2:172.21.15.178:6838/2947354024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562ea4a33cd0 con 0x562ea4ad80c0 2024-03-07T07:41:05.399 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.392+0000 7fe5c9ffb640 10 client.15243 handle_command_reply: tid=0 2024-03-07T07:41:05.399 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.392+0000 7fe5eb465740 1 client.15243 shutdown 2024-03-07T07:41:05.399 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.392+0000 7fe5eb465740 1 -- 172.21.15.160:0/1058278822 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562ea4ac1aa0 msgr2=0x562ea4acba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:05.399 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.392+0000 7fe5eb465740 1 --2- 172.21.15.160:0/1058278822 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562ea4ac1aa0 0x562ea4acba50 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0097240 tx=0x7fe5d009af70 comp rx=0 tx=0).stop 2024-03-07T07:41:05.399 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.392+0000 7fe5eb465740 1 -- 172.21.15.160:0/1058278822 shutdown_connections 2024-03-07T07:41:05.399 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.392+0000 7fe5eb465740 1 --2- 172.21.15.160:0/1058278822 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x562ea4ad80c0 0x562ea4ad7160 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.399 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.392+0000 7fe5eb465740 1 --2- 172.21.15.160:0/1058278822 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ea4ac6c10 0x562ea4ad2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.399 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.392+0000 7fe5eb465740 1 --2- 172.21.15.160:0/1058278822 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562ea4ac1aa0 0x562ea4acba50 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.399 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.392+0000 7fe5eb465740 1 --2- 172.21.15.160:0/1058278822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ea4ac10b0 0x562ea4acb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.399 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.392+0000 7fe5c9ffb640 0 client.15243 ms_handle_reset on v2:172.21.15.178:6838/2947354024 2024-03-07T07:41:05.399 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.392+0000 7fe5eb465740 1 -- 172.21.15.160:0/1058278822 >> 172.21.15.160:0/1058278822 conn(0x562ea4a893d0 msgr2=0x562ea4abbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:05.399 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.392+0000 7fe5eb465740 1 -- 172.21.15.160:0/1058278822 shutdown_connections 2024-03-07T07:41:05.399 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.392+0000 7fe5eb465740 1 -- 172.21.15.160:0/1058278822 wait complete. 2024-03-07T07:41:05.400 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.392+0000 7fe5eb465740 20 client.15243 trim_cache size 0 max 16384 2024-03-07T07:41:05.415 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.408+0000 7fe5eb465740 1 Processor -- start 2024-03-07T07:41:05.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.408+0000 7fe5eb465740 1 -- start start 2024-03-07T07:41:05.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.409+0000 7fe5eb465740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ea4ac10b0 0x562ea4acb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.409+0000 7fe5eb465740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562ea4ac1aa0 0x562ea4ad2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.409+0000 7fe5eb465740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ea4ac6c10 0x562ea4acba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.409+0000 7fe5eb465740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ea4ad78e0 con 0x562ea4ac1aa0 2024-03-07T07:41:05.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.409+0000 7fe5eb465740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ea4ad7f40 con 0x562ea4ac6c10 2024-03-07T07:41:05.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.409+0000 7fe5eb465740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ea4ad85a0 con 0x562ea4ac10b0 2024-03-07T07:41:05.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.409+0000 7fe5e5c1e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562ea4ac1aa0 0x562ea4ad2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.409+0000 7fe5e5c1e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562ea4ac1aa0 0x562ea4ad2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:58008/0 (socket says 172.21.15.160:58008) 2024-03-07T07:41:05.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.409+0000 7fe5e5c1e640 1 -- 172.21.15.160:0/3727727287 learned_addr learned my addr 172.21.15.160:0/3727727287 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:41:05.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.409+0000 7fe5e641f640 1 --2- 172.21.15.160:0/3727727287 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ea4ac10b0 0x562ea4acb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.409+0000 7fe5e6c20640 1 --2- 172.21.15.160:0/3727727287 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ea4ac6c10 0x562ea4acba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.409+0000 7fe5e5c1e640 1 -- 172.21.15.160:0/3727727287 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ea4ac10b0 msgr2=0x562ea4acb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:41:05.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.409+0000 7fe5e5c1e640 1 --2- 172.21.15.160:0/3727727287 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ea4ac10b0 0x562ea4acb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.409+0000 7fe5e5c1e640 1 -- 172.21.15.160:0/3727727287 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ea4ac6c10 msgr2=0x562ea4acba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:41:05.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.409+0000 7fe5e5c1e640 1 --2- 172.21.15.160:0/3727727287 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ea4ac6c10 0x562ea4acba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.409+0000 7fe5e5c1e640 1 -- 172.21.15.160:0/3727727287 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ea4ac1e80 con 0x562ea4ac1aa0 2024-03-07T07:41:05.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.409+0000 7fe5e6c20640 1 --2- 172.21.15.160:0/3727727287 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ea4ac6c10 0x562ea4acba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:41:05.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.409+0000 7fe5e5c1e640 1 --2- 172.21.15.160:0/3727727287 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562ea4ac1aa0 0x562ea4ad2a50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0097290 tx=0x7fe5d0018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1572cfed7396face server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:05.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.409+0000 7fe5cbfff640 1 -- 172.21.15.160:0/3727727287 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d00b91c0 con 0x562ea4ac1aa0 2024-03-07T07:41:05.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.409+0000 7fe5cbfff640 1 -- 172.21.15.160:0/3727727287 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe5d00b3070 con 0x562ea4ac1aa0 2024-03-07T07:41:05.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.409+0000 7fe5cbfff640 1 -- 172.21.15.160:0/3727727287 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0096b80 con 0x562ea4ac1aa0 2024-03-07T07:41:05.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.410+0000 7fe5eb465740 1 -- 172.21.15.160:0/3727727287 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562ea4ac1aa0 msgr2=0x562ea4ad2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:05.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.410+0000 7fe5eb465740 1 --2- 172.21.15.160:0/3727727287 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562ea4ac1aa0 0x562ea4ad2a50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0097290 tx=0x7fe5d0018420 comp rx=0 tx=0).stop 2024-03-07T07:41:05.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.410+0000 7fe5eb465740 1 -- 172.21.15.160:0/3727727287 shutdown_connections 2024-03-07T07:41:05.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.410+0000 7fe5eb465740 1 --2- 172.21.15.160:0/3727727287 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ea4ac6c10 0x562ea4acba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.410+0000 7fe5eb465740 1 --2- 172.21.15.160:0/3727727287 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562ea4ac1aa0 0x562ea4ad2a50 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.410+0000 7fe5eb465740 1 --2- 172.21.15.160:0/3727727287 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ea4ac10b0 0x562ea4acb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.410+0000 7fe5eb465740 1 -- 172.21.15.160:0/3727727287 >> 172.21.15.160:0/3727727287 conn(0x562ea4a893d0 msgr2=0x562ea4ac2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:05.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.410+0000 7fe5eb465740 1 -- 172.21.15.160:0/3727727287 shutdown_connections 2024-03-07T07:41:05.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.410+0000 7fe5eb465740 1 -- 172.21.15.160:0/3727727287 wait complete. 2024-03-07T07:41:05.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.410+0000 7fe5eb465740 1 -- start start 2024-03-07T07:41:05.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.410+0000 7fe5eb465740 1 Processor -- start 2024-03-07T07:41:05.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.410+0000 7fe5eb465740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ea4ac10b0 0x562ea4ad2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.410+0000 7fe5eb465740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562ea4ac1aa0 0x562ea4acf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.410+0000 7fe5eb465740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ea4ac6c10 0x562ea4acfa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.410+0000 7fe5eb465740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ea4ad9260 con 0x562ea4ac1aa0 2024-03-07T07:41:05.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.410+0000 7fe5eb465740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ea4ad8c00 con 0x562ea4ac10b0 2024-03-07T07:41:05.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.410+0000 7fe5eb465740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ea4ad98c0 con 0x562ea4ac6c10 2024-03-07T07:41:05.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.410+0000 7fe5eb465740 10 client.0 _ll_register_callbacks cb 0x7fe5d00cba90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-07T07:41:05.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.410+0000 7fe5e5c1e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562ea4ac1aa0 0x562ea4acf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.410+0000 7fe5e5c1e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562ea4ac1aa0 0x562ea4acf4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:58014/0 (socket says 172.21.15.160:58014) 2024-03-07T07:41:05.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.410+0000 7fe5e5c1e640 1 -- 172.21.15.160:0/1699904703 learned_addr learned my addr 172.21.15.160:0/1699904703 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:41:05.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.410+0000 7fe5e6c20640 1 --2- 172.21.15.160:0/1699904703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ea4ac6c10 0x562ea4acfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.411+0000 7fe5e641f640 1 --2- 172.21.15.160:0/1699904703 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ea4ac10b0 0x562ea4ad2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.411+0000 7fe5e5c1e640 1 -- 172.21.15.160:0/1699904703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ea4ac6c10 msgr2=0x562ea4acfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:41:05.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.411+0000 7fe5e5c1e640 1 --2- 172.21.15.160:0/1699904703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ea4ac6c10 0x562ea4acfa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.411+0000 7fe5e5c1e640 1 -- 172.21.15.160:0/1699904703 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ea4ac10b0 msgr2=0x562ea4ad2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:41:05.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.411+0000 7fe5e5c1e640 1 --2- 172.21.15.160:0/1699904703 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ea4ac10b0 0x562ea4ad2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.411+0000 7fe5e5c1e640 1 -- 172.21.15.160:0/1699904703 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ea4b33150 con 0x562ea4ac1aa0 2024-03-07T07:41:05.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.411+0000 7fe5e5c1e640 1 --2- 172.21.15.160:0/1699904703 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562ea4ac1aa0 0x562ea4acf4a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe5d000c6a0 tx=0x7fe5d009a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=330961758d96041d server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:05.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.411+0000 7fe5c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-03-07T07:41:05.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.411+0000 7fe5c9ffb640 1 -- 172.21.15.160:0/1699904703 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d009a050 con 0x562ea4ac1aa0 2024-03-07T07:41:05.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.411+0000 7fe5c9ffb640 1 -- 172.21.15.160:0/1699904703 <== mon.0 v2:172.21.15.160:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe5d0096b80 con 0x562ea4ac1aa0 2024-03-07T07:41:05.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.411+0000 7fe5eb465740 1 -- 172.21.15.160:0/1699904703 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe5d00b3070 con 0x562ea4ac1aa0 2024-03-07T07:41:05.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.411+0000 7fe5c9ffb640 1 -- 172.21.15.160:0/1699904703 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5d00991f0 con 0x562ea4ac1aa0 2024-03-07T07:41:05.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.411+0000 7fe5eb465740 10 client.15249 fetch_fsmap learned FSMap version 21 2024-03-07T07:41:05.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.411+0000 7fe5eb465740 1 -- 172.21.15.160:0/1699904703 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x562ea4b34450 con 0x562ea4ac1aa0 2024-03-07T07:41:05.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.411+0000 7fe5c9ffb640 1 -- 172.21.15.160:0/1699904703 <== mon.0 v2:172.21.15.160:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe5d00b3070 con 0x562ea4ac1aa0 2024-03-07T07:41:05.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.411+0000 7fe5c9ffb640 1 -- 172.21.15.160:0/1699904703 <== mon.0 v2:172.21.15.160:3300/0 5 ==== fsmap(e 21) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fe5d0085620 con 0x562ea4ac1aa0 2024-03-07T07:41:05.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.411+0000 7fe5eb465740 10 client.15249 fetch_fsmap finished waiting for FSMap version 21 2024-03-07T07:41:05.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.411+0000 7fe5eb465740 10 client.15249 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-07T07:41:05.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.411+0000 7fe5eb465740 20 client.15249 populate_metadata read hostname 'smithi160' 2024-03-07T07:41:05.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.411+0000 7fe5eb465740 1 --2- 172.21.15.160:0/1699904703 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x562ea4b38750 0x562ea4b34620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:05.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.411+0000 7fe5eb465740 4 client.15249 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-07T07:41:05.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.411+0000 7fe5eb465740 1 -- 172.21.15.160:0/1699904703 --> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562ea4a2e020 con 0x562ea4b38750 2024-03-07T07:41:05.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.412+0000 7fe5e641f640 1 --2- 172.21.15.160:0/1699904703 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x562ea4b38750 0x562ea4b34620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:05.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.412+0000 7fe5c9ffb640 10 client.15249 ms_handle_connect on v2:172.21.15.178:6838/2947354024 2024-03-07T07:41:05.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.412+0000 7fe5e641f640 1 --2- 172.21.15.160:0/1699904703 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x562ea4b38750 0x562ea4b34620 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdf918e861c4035b server_cookie=dd87fd1f1fd9bd55 in_seq=0 out_seq=0 2024-03-07T07:41:05.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.412+0000 7fe5c9ffb640 1 -- 172.21.15.160:0/1699904703 <== mds.0 v2:172.21.15.178:6838/2947354024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x562ea4a2e020 con 0x562ea4b38750 2024-03-07T07:41:05.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.412+0000 7fe5c9ffb640 10 client.15249 handle_command_reply: tid=0 2024-03-07T07:41:05.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.412+0000 7fe5eb465740 1 client.15249 shutdown 2024-03-07T07:41:05.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.412+0000 7fe5eb465740 1 -- 172.21.15.160:0/1699904703 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562ea4ac1aa0 msgr2=0x562ea4acf4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:05.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.412+0000 7fe5eb465740 1 --2- 172.21.15.160:0/1699904703 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562ea4ac1aa0 0x562ea4acf4a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe5d000c6a0 tx=0x7fe5d009a860 comp rx=0 tx=0).stop 2024-03-07T07:41:05.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.412+0000 7fe5eb465740 1 -- 172.21.15.160:0/1699904703 shutdown_connections 2024-03-07T07:41:05.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.412+0000 7fe5eb465740 1 --2- 172.21.15.160:0/1699904703 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x562ea4b38750 0x562ea4b34620 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.412+0000 7fe5eb465740 1 --2- 172.21.15.160:0/1699904703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ea4ac6c10 0x562ea4acfa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.412+0000 7fe5eb465740 1 --2- 172.21.15.160:0/1699904703 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562ea4ac1aa0 0x562ea4acf4a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.412+0000 7fe5c9ffb640 0 client.15249 ms_handle_reset on v2:172.21.15.178:6838/2947354024 2024-03-07T07:41:05.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.412+0000 7fe5eb465740 1 --2- 172.21.15.160:0/1699904703 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ea4ac10b0 0x562ea4ad2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.412+0000 7fe5eb465740 1 -- 172.21.15.160:0/1699904703 >> 172.21.15.160:0/1699904703 conn(0x562ea4a893d0 msgr2=0x562ea4acb510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:05.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.412+0000 7fe5eb465740 1 -- 172.21.15.160:0/1699904703 shutdown_connections 2024-03-07T07:41:05.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.412+0000 7fe5eb465740 1 -- 172.21.15.160:0/1699904703 wait complete. 2024-03-07T07:41:05.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.412+0000 7fe5eb465740 20 client.15249 trim_cache size 0 max 16384 2024-03-07T07:41:05.422 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-03-07T07:41:05.422 INFO:teuthology.orchestra.run.smithi160.stdout: "cluster_fsid": "826f6582-dc54-11ee-95c6-87774f69a715", 2024-03-07T07:41:05.422 INFO:teuthology.orchestra.run.smithi160.stdout: "whoami": 0, 2024-03-07T07:41:05.422 INFO:teuthology.orchestra.run.smithi160.stdout: "id": 24511, 2024-03-07T07:41:05.422 INFO:teuthology.orchestra.run.smithi160.stdout: "want_state": "up:active", 2024-03-07T07:41:05.422 INFO:teuthology.orchestra.run.smithi160.stdout: "state": "up:active", 2024-03-07T07:41:05.422 INFO:teuthology.orchestra.run.smithi160.stdout: "fs_name": "cephfs", 2024-03-07T07:41:05.423 INFO:teuthology.orchestra.run.smithi160.stdout: "rank_uptime": 110.582170968, 2024-03-07T07:41:05.423 INFO:teuthology.orchestra.run.smithi160.stdout: "mdsmap_epoch": 21, 2024-03-07T07:41:05.423 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_epoch": 87, 2024-03-07T07:41:05.423 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_epoch_barrier": 85, 2024-03-07T07:41:05.423 INFO:teuthology.orchestra.run.smithi160.stdout: "uptime": 126.075284303 2024-03-07T07:41:05.423 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-03-07T07:41:05.424 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.413+0000 7fe5e8683640 1 -- 172.21.15.160:0/3875036939 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fe5cc0a1e00 msgr2=0x7fe5cc0a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:05.424 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.413+0000 7fe5e8683640 1 --2- 172.21.15.160:0/3875036939 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fe5cc0a1e00 0x7fe5cc0a42c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8005e30 tx=0x7fe5d8032590 comp rx=0 tx=0).stop 2024-03-07T07:41:05.424 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.413+0000 7fe5e8683640 1 -- 172.21.15.160:0/3875036939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5e0101770 msgr2=0x7fe5e019c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:05.424 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.413+0000 7fe5e8683640 1 --2- 172.21.15.160:0/3875036939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5e0101770 0x7fe5e019c610 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe5d000e9f0 tx=0x7fe5d000eeb0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.413+0000 7fe5e8683640 1 -- 172.21.15.160:0/3875036939 shutdown_connections 2024-03-07T07:41:05.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.413+0000 7fe5e8683640 1 --2- 172.21.15.160:0/3875036939 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fe5cc0a1e00 0x7fe5cc0a42c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.413+0000 7fe5e8683640 1 --2- 172.21.15.160:0/3875036939 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5e0102130 0x7fe5e019ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.413+0000 7fe5e8683640 1 --2- 172.21.15.160:0/3875036939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5e0101770 0x7fe5e019c610 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.413+0000 7fe5e8683640 1 --2- 172.21.15.160:0/3875036939 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5e0100580 0x7fe5e006a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:05.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.413+0000 7fe5e8683640 1 -- 172.21.15.160:0/3875036939 >> 172.21.15.160:0/3875036939 conn(0x7fe5e00fbb10 msgr2=0x7fe5e0104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:05.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.413+0000 7fe5e8683640 1 -- 172.21.15.160:0/3875036939 shutdown_connections 2024-03-07T07:41:05.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:05.413+0000 7fe5e8683640 1 -- 172.21.15.160:0/3875036939 wait complete. 2024-03-07T07:41:05.425 INFO:teuthology.run_tasks:Running task kclient... 2024-03-07T07:41:05.457 INFO:tasks.kclient:Mounting kernel clients... 2024-03-07T07:41:05.457 INFO:tasks.kclient:config is {'client.0': {}} 2024-03-07T07:41:05.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:41:05.458 DEBUG:teuthology.orchestra.run.smithi160:> ip netns list 2024-03-07T07:41:05.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:41:05.473 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip link delete ceph-brx 2024-03-07T07:41:05.536 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:05 smithi160 ceph-mon[31963]: pgmap v345: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-07T07:41:05.536 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:05 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/4200001009' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:41:05.536 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:05 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/353281160' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:41:05.541 INFO:teuthology.orchestra.run.smithi160.stderr:Cannot find device "ceph-brx" 2024-03-07T07:41:05.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-07T07:41:05.543 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-03-07T07:41:05.543 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-07T07:41:05.544 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-07T07:41:05.544 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph fs ls 2024-03-07T07:41:05.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:05 smithi178 ceph-mon[36650]: pgmap v345: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-07T07:41:05.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:05 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/4200001009' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:41:05.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:05 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/353281160' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:41:05.825 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:41:05.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:05 smithi177 ceph-mon[36578]: pgmap v345: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-07T07:41:05.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:05 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/4200001009' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:41:05.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:05 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/353281160' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:41:06.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:06 smithi160 ceph-mon[31963]: pgmap v346: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-07T07:41:06.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:06 smithi178 ceph-mon[36650]: pgmap v346: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-07T07:41:06.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:06 smithi177 ceph-mon[36578]: pgmap v346: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-07T07:41:07.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.615+0000 7f91b3480700 1 -- 172.21.15.160:0/1623267914 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91ac102de0 msgr2=0x7f91ac106db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:07.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.615+0000 7f91b3480700 1 --2- 172.21.15.160:0/1623267914 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91ac102de0 0x7f91ac106db0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f91a80099c0 tx=0x7f91a8009cd0 comp rx=0 tx=0).stop 2024-03-07T07:41:07.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.616+0000 7f91b3480700 1 -- 172.21.15.160:0/1623267914 shutdown_connections 2024-03-07T07:41:07.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.616+0000 7f91b3480700 1 --2- 172.21.15.160:0/1623267914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91ac1074e0 0x7f91ac10f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:07.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.616+0000 7f91b3480700 1 --2- 172.21.15.160:0/1623267914 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91ac102430 0x7f91ac102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:07.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.616+0000 7f91b3480700 1 --2- 172.21.15.160:0/1623267914 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91ac102de0 0x7f91ac106db0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:07.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.616+0000 7f91b3480700 1 -- 172.21.15.160:0/1623267914 >> 172.21.15.160:0/1623267914 conn(0x7f91ac079270 msgr2=0x7f91ac079680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:07.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.616+0000 7f91b3480700 1 -- 172.21.15.160:0/1623267914 shutdown_connections 2024-03-07T07:41:07.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.616+0000 7f91b3480700 1 -- 172.21.15.160:0/1623267914 wait complete. 2024-03-07T07:41:07.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.616+0000 7f91b3480700 1 Processor -- start 2024-03-07T07:41:07.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.616+0000 7f91b3480700 1 -- start start 2024-03-07T07:41:07.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.616+0000 7f91b3480700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91ac102430 0x7f91ac074450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:07.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.616+0000 7f91b3480700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91ac102de0 0x7f91ac074990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:07.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.616+0000 7f91b3480700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91ac1074e0 0x7f91ac074ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:07.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.616+0000 7f91b3480700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ac112450 con 0x7f91ac1074e0 2024-03-07T07:41:07.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.616+0000 7f91b3480700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ac1122d0 con 0x7f91ac102de0 2024-03-07T07:41:07.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.617+0000 7f91b3480700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ac1125d0 con 0x7f91ac102430 2024-03-07T07:41:07.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.617+0000 7f91b121c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91ac102430 0x7f91ac074450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:07.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.617+0000 7f91b1a1d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91ac1074e0 0x7f91ac074ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:07.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.617+0000 7f91b121c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91ac102430 0x7f91ac074450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.160:47504/0 (socket says 172.21.15.160:47504) 2024-03-07T07:41:07.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.617+0000 7f91b121c700 1 -- 172.21.15.160:0/279357838 learned_addr learned my addr 172.21.15.160:0/279357838 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:41:07.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.617+0000 7f91b0a1b700 1 --2- 172.21.15.160:0/279357838 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91ac102de0 0x7f91ac074990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:07.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.617+0000 7f91b121c700 1 -- 172.21.15.160:0/279357838 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91ac102de0 msgr2=0x7f91ac074990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:07.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.617+0000 7f91b121c700 1 --2- 172.21.15.160:0/279357838 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91ac102de0 0x7f91ac074990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:07.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.617+0000 7f91b121c700 1 -- 172.21.15.160:0/279357838 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91ac1074e0 msgr2=0x7f91ac074ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:07.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.617+0000 7f91b121c700 1 --2- 172.21.15.160:0/279357838 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91ac1074e0 0x7f91ac074ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:07.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.617+0000 7f91b121c700 1 -- 172.21.15.160:0/279357838 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91ac1a6df0 con 0x7f91ac102430 2024-03-07T07:41:07.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.617+0000 7f91b121c700 1 --2- 172.21.15.160:0/279357838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91ac102430 0x7f91ac074450 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f919c00e9a0 tx=0x7f919c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:07.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.617+0000 7f91a27fc700 1 -- 172.21.15.160:0/279357838 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919c00ca10 con 0x7f91ac102430 2024-03-07T07:41:07.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.617+0000 7f91b3480700 1 -- 172.21.15.160:0/279357838 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91ac1a70e0 con 0x7f91ac102430 2024-03-07T07:41:07.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.617+0000 7f91a27fc700 1 -- 172.21.15.160:0/279357838 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f919c00cbb0 con 0x7f91ac102430 2024-03-07T07:41:07.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.617+0000 7f91a27fc700 1 -- 172.21.15.160:0/279357838 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919c018680 con 0x7f91ac102430 2024-03-07T07:41:07.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.617+0000 7f91b3480700 1 -- 172.21.15.160:0/279357838 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91ac1a7640 con 0x7f91ac102430 2024-03-07T07:41:07.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.618+0000 7f91b3480700 1 -- 172.21.15.160:0/279357838 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91ac068e10 con 0x7f91ac102430 2024-03-07T07:41:07.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.619+0000 7f91a27fc700 1 -- 172.21.15.160:0/279357838 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f919c010430 con 0x7f91ac102430 2024-03-07T07:41:07.622 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.620+0000 7f91a27fc700 1 --2- 172.21.15.160:0/279357838 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f91980a1e10 0x7f91980a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:07.622 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.620+0000 7f91b0a1b700 1 --2- 172.21.15.160:0/279357838 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f91980a1e10 0x7f91980a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:07.622 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.620+0000 7f91a27fc700 1 -- 172.21.15.160:0/279357838 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f919c0ce4c0 con 0x7f91ac102430 2024-03-07T07:41:07.622 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.620+0000 7f91b0a1b700 1 --2- 172.21.15.160:0/279357838 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f91980a1e10 0x7f91980a42d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f91a8009990 tx=0x7f91a8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:07.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.627+0000 7f91a27fc700 1 -- 172.21.15.160:0/279357838 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f919c09b8b0 con 0x7f91ac102430 2024-03-07T07:41:07.865 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.864+0000 7f91b3480700 1 -- 172.21.15.160:0/279357838 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f91ac02d9f0 con 0x7f91ac102430 2024-03-07T07:41:07.865 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.864+0000 7f91a27fc700 1 -- 172.21.15.160:0/279357838 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v21) v1 ==== 53+0+73 (secure 0 0 0) 0x7f919c09b1b0 con 0x7f91ac102430 2024-03-07T07:41:07.866 INFO:teuthology.orchestra.run.smithi160.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-07T07:41:07.866 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.866+0000 7f91b3480700 1 -- 172.21.15.160:0/279357838 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f91980a1e10 msgr2=0x7f91980a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:07.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.866+0000 7f91b3480700 1 --2- 172.21.15.160:0/279357838 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f91980a1e10 0x7f91980a42d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f91a8009990 tx=0x7f91a8005fb0 comp rx=0 tx=0).stop 2024-03-07T07:41:07.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.866+0000 7f91b3480700 1 -- 172.21.15.160:0/279357838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91ac102430 msgr2=0x7f91ac074450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:07.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.866+0000 7f91b3480700 1 --2- 172.21.15.160:0/279357838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91ac102430 0x7f91ac074450 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f919c00e9a0 tx=0x7f919c00ecb0 comp rx=0 tx=0).stop 2024-03-07T07:41:07.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.866+0000 7f91b3480700 1 -- 172.21.15.160:0/279357838 shutdown_connections 2024-03-07T07:41:07.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.866+0000 7f91b3480700 1 --2- 172.21.15.160:0/279357838 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91ac1074e0 0x7f91ac074ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:07.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.866+0000 7f91b3480700 1 --2- 172.21.15.160:0/279357838 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f91980a1e10 0x7f91980a42d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:07.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.866+0000 7f91b3480700 1 --2- 172.21.15.160:0/279357838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91ac102430 0x7f91ac074450 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:07.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.866+0000 7f91b3480700 1 --2- 172.21.15.160:0/279357838 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91ac102de0 0x7f91ac074990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:07.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.866+0000 7f91b3480700 1 -- 172.21.15.160:0/279357838 >> 172.21.15.160:0/279357838 conn(0x7f91ac079270 msgr2=0x7f91ac108610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:07.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.866+0000 7f91b3480700 1 -- 172.21.15.160:0/279357838 shutdown_connections 2024-03-07T07:41:07.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:07.866+0000 7f91b3480700 1 -- 172.21.15.160:0/279357838 wait complete. 2024-03-07T07:41:08.176 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:07 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/279357838' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-07T07:41:08.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:07 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/279357838' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-07T07:41:08.356 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:07 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/279357838' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-07T07:41:08.613 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-07T07:41:08.613 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-07T07:41:08.613 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi160.front.sepia.ceph.com 2024-03-07T07:41:08.613 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-07T07:41:08.613 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-07T07:41:08.613 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-07T07:41:08.613 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-07T07:41:08.613 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-07T07:41:08.613 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-07T07:41:08.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:41:08.614 DEBUG:teuthology.orchestra.run.smithi160:> ip addr 2024-03-07T07:41:08.630 INFO:teuthology.orchestra.run.smithi160.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-07T07:41:08.630 INFO:teuthology.orchestra.run.smithi160.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-07T07:41:08.630 INFO:teuthology.orchestra.run.smithi160.stdout: inet 127.0.0.1/8 scope host lo 2024-03-07T07:41:08.630 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft forever preferred_lft forever 2024-03-07T07:41:08.630 INFO:teuthology.orchestra.run.smithi160.stdout: inet6 ::1/128 scope host 2024-03-07T07:41:08.630 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft forever preferred_lft forever 2024-03-07T07:41:08.630 INFO:teuthology.orchestra.run.smithi160.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-07T07:41:08.630 INFO:teuthology.orchestra.run.smithi160.stdout: link/ether 0c:c4:7a:d9:82:80 brd ff:ff:ff:ff:ff:ff 2024-03-07T07:41:08.630 INFO:teuthology.orchestra.run.smithi160.stdout: altname enp7s0f0 2024-03-07T07:41:08.630 INFO:teuthology.orchestra.run.smithi160.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-07T07:41:08.631 INFO:teuthology.orchestra.run.smithi160.stdout: link/ether 0c:c4:7a:d9:82:81 brd ff:ff:ff:ff:ff:ff 2024-03-07T07:41:08.631 INFO:teuthology.orchestra.run.smithi160.stdout: altname enp7s0f1 2024-03-07T07:41:08.631 INFO:teuthology.orchestra.run.smithi160.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-07T07:41:08.631 INFO:teuthology.orchestra.run.smithi160.stdout: link/ether 0c:c4:7a:88:7f:5a brd ff:ff:ff:ff:ff:ff 2024-03-07T07:41:08.631 INFO:teuthology.orchestra.run.smithi160.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-07T07:41:08.631 INFO:teuthology.orchestra.run.smithi160.stdout: link/ether 0c:c4:7a:88:7f:5b brd ff:ff:ff:ff:ff:ff 2024-03-07T07:41:08.631 INFO:teuthology.orchestra.run.smithi160.stdout: inet 172.21.15.160/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-07T07:41:08.631 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft 42078sec preferred_lft 42078sec 2024-03-07T07:41:08.631 INFO:teuthology.orchestra.run.smithi160.stdout: inet6 fe80::ec4:7aff:fe88:7f5b/64 scope link 2024-03-07T07:41:08.631 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft forever preferred_lft forever 2024-03-07T07:41:08.632 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-07T07:41:08.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:41:08.632 DEBUG:teuthology.orchestra.run.smithi160:> (cd / && exec bash -c ' 2024-03-07T07:41:08.632 DEBUG:teuthology.orchestra.run.smithi160:> set -e 2024-03-07T07:41:08.632 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip link add name ceph-brx type bridge 2024-03-07T07:41:08.632 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip addr flush dev ceph-brx 2024-03-07T07:41:08.632 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip link set ceph-brx up 2024-03-07T07:41:08.632 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-07T07:41:08.632 DEBUG:teuthology.orchestra.run.smithi160:> ') 2024-03-07T07:41:08.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:41:08.807 DEBUG:teuthology.orchestra.run.smithi160:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-07T07:41:08.834 INFO:teuthology.orchestra.run.smithi160.stdout:1 2024-03-07T07:41:08.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:41:08.836 DEBUG:teuthology.orchestra.run.smithi160:> route 2024-03-07T07:41:08.902 INFO:teuthology.orchestra.run.smithi160.stdout:Kernel IP routing table 2024-03-07T07:41:08.902 INFO:teuthology.orchestra.run.smithi160.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-07T07:41:08.902 INFO:teuthology.orchestra.run.smithi160.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-07T07:41:08.902 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-07T07:41:08.903 INFO:teuthology.orchestra.run.smithi160.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-07T07:41:08.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:41:08.903 DEBUG:teuthology.orchestra.run.smithi160:> (cd / && exec bash -c ' 2024-03-07T07:41:08.903 DEBUG:teuthology.orchestra.run.smithi160:> set -e 2024-03-07T07:41:08.903 DEBUG:teuthology.orchestra.run.smithi160:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-07T07:41:08.903 DEBUG:teuthology.orchestra.run.smithi160:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-07T07:41:08.903 DEBUG:teuthology.orchestra.run.smithi160:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-07T07:41:08.903 DEBUG:teuthology.orchestra.run.smithi160:> ') 2024-03-07T07:41:09.178 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:08 smithi160 ceph-mon[31963]: pgmap v347: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-07T07:41:09.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:41:09.184 DEBUG:teuthology.orchestra.run.smithi160:> ip netns list 2024-03-07T07:41:09.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:41:09.199 DEBUG:teuthology.orchestra.run.smithi160:> ip netns list-id 2024-03-07T07:41:09.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:41:09.253 DEBUG:teuthology.orchestra.run.smithi160:> (cd / && exec bash -c ' 2024-03-07T07:41:09.253 DEBUG:teuthology.orchestra.run.smithi160:> set -e 2024-03-07T07:41:09.253 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-07T07:41:09.253 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-07T07:41:09.253 DEBUG:teuthology.orchestra.run.smithi160:> ') 2024-03-07T07:41:09.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:08 smithi178 ceph-mon[36650]: pgmap v347: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-07T07:41:09.336 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-07T07:41:09.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:41:09.336 DEBUG:teuthology.orchestra.run.smithi160:> (cd / && exec bash -c ' 2024-03-07T07:41:09.336 DEBUG:teuthology.orchestra.run.smithi160:> set -e 2024-03-07T07:41:09.337 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-07T07:41:09.337 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-07T07:41:09.337 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-07T07:41:09.337 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-07T07:41:09.337 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-07T07:41:09.337 DEBUG:teuthology.orchestra.run.smithi160:> ') 2024-03-07T07:41:09.356 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:08 smithi177 ceph-mon[36578]: pgmap v347: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-07T07:41:09.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:41:09.485 DEBUG:teuthology.orchestra.run.smithi160:> (cd / && exec bash -c ' 2024-03-07T07:41:09.485 DEBUG:teuthology.orchestra.run.smithi160:> set -e 2024-03-07T07:41:09.485 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip link set brx.0 up 2024-03-07T07:41:09.485 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip link set dev brx.0 master ceph-brx 2024-03-07T07:41:09.485 DEBUG:teuthology.orchestra.run.smithi160:> ') 2024-03-07T07:41:09.526 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-07T07:41:09.526 DEBUG:teuthology.orchestra.run.smithi160:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-07T07:41:09.582 INFO:teuthology.orchestra.run.smithi160.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-07T07:41:09.582 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-07T07:41:09.582 DEBUG:teuthology.orchestra.run.smithi160:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-07T07:41:09.636 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-03-07T07:41:09.636 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-03-07T07:41:09.636 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/be5ac2bd-d227-43bd-bead-10930cde3bb5 2024-03-07T07:41:09.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:41:09.637 DEBUG:teuthology.orchestra.run.smithi160:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/be5ac2bd-d227-43bd-bead-10930cde3bb5 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-03-07T07:41:09.720 INFO:teuthology.orchestra.run.smithi160.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-03-07T07:41:09.721 INFO:teuthology.orchestra.run.smithi160.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2024-03-07T07:41:09.834 INFO:teuthology.orchestra.run.smithi160.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-03-07T07:41:09.834 INFO:teuthology.orchestra.run.smithi160.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2024-03-07T07:41:09.834 INFO:teuthology.orchestra.run.smithi160.stdout:mount.ceph: resolved to: "172.21.15.160:3300,172.21.15.177:3300,172.21.15.178:3300" 2024-03-07T07:41:09.835 INFO:teuthology.orchestra.run.smithi160.stdout:mount.ceph: trying mount with new device syntax: 0@826f6582-dc54-11ee-95c6-87774f69a715.cephfs=/volumes/qa/sv_1/be5ac2bd-d227-43bd-bead-10930cde3bb5 2024-03-07T07:41:09.835 INFO:teuthology.orchestra.run.smithi160.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync,name=0,key=0,mon_addr=172.21.15.160:3300/172.21.15.177:3300/172.21.15.178:3300" will pass to kernel 2024-03-07T07:41:09.836 INFO:teuthology.orchestra.run.smithi160.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-07T07:41:09.836 INFO:teuthology.orchestra.run.smithi160.stdout: You just mounted a file system that supports labels which does not 2024-03-07T07:41:09.836 INFO:teuthology.orchestra.run.smithi160.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-07T07:41:09.836 INFO:teuthology.orchestra.run.smithi160.stdout: applications will generate AVC messages and not be allowed access to 2024-03-07T07:41:09.836 INFO:teuthology.orchestra.run.smithi160.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-07T07:41:09.838 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-07T07:41:09.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:41:09.838 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-07T07:41:09.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:41:09.909 DEBUG:teuthology.orchestra.run.smithi160:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-07T07:41:09.966 INFO:teuthology.orchestra.run.smithi160.stdout:client25094 2024-03-07T07:41:09.966 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-03-07T07:41:09.966 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T07:41:10.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:41:10.023 DEBUG:teuthology.orchestra.run.smithi160:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-07T07:41:10.079 INFO:teuthology.orchestra.run.smithi160.stdout:client25094 2024-03-07T07:41:10.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:41:10.079 DEBUG:teuthology.orchestra.run.smithi160:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-07T07:41:10.135 INFO:teuthology.orchestra.run.smithi160.stdout:client25094 2024-03-07T07:41:10.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:41:10.135 DEBUG:teuthology.orchestra.run.smithi160:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/826f6582-dc54-11ee-95c6-87774f69a715.client25094/status') 2024-03-07T07:41:10.204 INFO:teuthology.orchestra.run.smithi160.stdout:instance: client.25094 (3)172.21.15.160:0/619388284 2024-03-07T07:41:10.204 INFO:teuthology.orchestra.run.smithi160.stdout:blocklisted: false 2024-03-07T07:41:10.205 INFO:teuthology.orchestra.run.smithi160.stderr:0+1 records in 2024-03-07T07:41:10.205 INFO:teuthology.orchestra.run.smithi160.stderr:0+1 records out 2024-03-07T07:41:10.205 INFO:teuthology.orchestra.run.smithi160.stderr:71 bytes copied, 2.05e-05 s, 3.5 MB/s 2024-03-07T07:41:10.206 INFO:teuthology.run_tasks:Running task exec... 2024-03-07T07:41:10.217 INFO:teuthology.task.exec:Executing custom commands... 2024-03-07T07:41:10.217 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi160.front.sepia.ceph.com 2024-03-07T07:41:10.217 DEBUG:teuthology.orchestra.run.smithi160:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-03-07T07:41:10.353 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:10.349+0000 7fc13d083640 1 -- 172.21.15.160:0/3164947476 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1381018f0 msgr2=0x7fc138108180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:10.353 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:10.349+0000 7fc13d083640 1 --2- 172.21.15.160:0/3164947476 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1381018f0 0x7fc138108180 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc130009ca0 tx=0x7fc13002c0f0 comp rx=0 tx=0).stop 2024-03-07T07:41:10.353 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:10.349+0000 7fc13d083640 1 -- 172.21.15.160:0/3164947476 shutdown_connections 2024-03-07T07:41:10.353 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:10.349+0000 7fc13d083640 1 --2- 172.21.15.160:0/3164947476 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1381018f0 0x7fc138108180 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:10.353 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:10.349+0000 7fc13d083640 1 --2- 172.21.15.160:0/3164947476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc138100f30 0x7fc1381013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:10.354 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:10.349+0000 7fc13d083640 1 --2- 172.21.15.160:0/3164947476 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1380ffd40 0x7fc138100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:10.354 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:10.349+0000 7fc13d083640 1 -- 172.21.15.160:0/3164947476 >> 172.21.15.160:0/3164947476 conn(0x7fc1380fb2b0 msgr2=0x7fc1380fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:10.354 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:10.349+0000 7fc13d083640 1 -- 172.21.15.160:0/3164947476 shutdown_connections 2024-03-07T07:41:10.354 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:10.349+0000 7fc13d083640 1 -- 172.21.15.160:0/3164947476 wait complete. 2024-03-07T07:41:10.354 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:10.350+0000 7fc13d083640 1 Processor -- start 2024-03-07T07:41:10.354 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:10.350+0000 7fc13d083640 1 -- start start 2024-03-07T07:41:10.354 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:10.350+0000 7fc13d083640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1380ffd40 0x7fc1381978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:10.355 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:10.350+0000 7fc13d083640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc138100f30 0x7fc138197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:10.355 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:10.350+0000 7fc13d083640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1381018f0 0x7fc13819eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:10.355 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:10.350+0000 7fc13d083640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13810b150 con 0x7fc138100f30 2024-03-07T07:41:10.355 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:10.350+0000 7fc13d083640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13810afd0 con 0x7fc1380ffd40 2024-03-07T07:41:10.355 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:10.350+0000 7fc13d083640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13810b2d0 con 0x7fc1381018f0 2024-03-07T07:41:10.355 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:10.351+0000 7fc13659c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc138100f30 0x7fc138197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:10.356 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:10.351+0000 7fc136d9d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1380ffd40 0x7fc1381978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:10.356 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:10.351+0000 7fc136d9d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1380ffd40 0x7fc1381978f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.160:55408/0 (socket says 172.21.15.160:55408) 2024-03-07T07:41:10.356 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:10.351+0000 7fc13759e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1381018f0 0x7fc13819eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:10.356 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:10.351+0000 7fc136d9d640 1 -- 172.21.15.160:0/755117927 learned_addr learned my addr 172.21.15.160:0/755117927 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:41:10.356 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:10.351+0000 7fc13759e640 1 -- 172.21.15.160:0/755117927 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1380ffd40 msgr2=0x7fc1381978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:10.356 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:10.351+0000 7fc13759e640 1 --2- 172.21.15.160:0/755117927 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1380ffd40 0x7fc1381978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:10.357 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:10.351+0000 7fc13759e640 1 -- 172.21.15.160:0/755117927 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc138100f30 msgr2=0x7fc138197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:10.357 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:10.351+0000 7fc13759e640 1 --2- 172.21.15.160:0/755117927 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc138100f30 0x7fc138197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:10.357 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:10.351+0000 7fc13759e640 1 -- 172.21.15.160:0/755117927 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc13819f510 con 0x7fc1381018f0 2024-03-07T07:41:10.357 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:10.351+0000 7fc13759e640 1 --2- 172.21.15.160:0/755117927 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1381018f0 0x7fc13819eea0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc13002c630 tx=0x7fc130032f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:10.359 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:10.355+0000 7fc11ffff640 1 -- 172.21.15.160:0/755117927 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc130049070 con 0x7fc1381018f0 2024-03-07T07:41:10.359 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:10.355+0000 7fc11ffff640 1 -- 172.21.15.160:0/755117927 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc130003970 con 0x7fc1381018f0 2024-03-07T07:41:10.359 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:10.355+0000 7fc11ffff640 1 -- 172.21.15.160:0/755117927 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1300044a0 con 0x7fc1381018f0 2024-03-07T07:41:10.359 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:10.355+0000 7fc13d083640 1 -- 172.21.15.160:0/755117927 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc138102150 con 0x7fc1381018f0 2024-03-07T07:41:10.359 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:10.355+0000 7fc13d083640 1 -- 172.21.15.160:0/755117927 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc138102660 con 0x7fc1381018f0 2024-03-07T07:41:10.359 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:10.356+0000 7fc13d083640 1 -- 172.21.15.160:0/755117927 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1381028c0 con 0x7fc1381018f0 2024-03-07T07:41:10.359 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:10.357+0000 7fc11ffff640 1 -- 172.21.15.160:0/755117927 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136754+0+0 (secure 0 0 0) 0x7fc13004c070 con 0x7fc1381018f0 2024-03-07T07:41:10.359 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:10.357+0000 7fc11ffff640 1 --2- 172.21.15.160:0/755117927 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fc1200a1ea0 0x7fc1200a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:10.359 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:10.357+0000 7fc136d9d640 1 --2- 172.21.15.160:0/755117927 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fc1200a1ea0 0x7fc1200a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:10.360 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:10.357+0000 7fc11ffff640 1 -- 172.21.15.160:0/755117927 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc1300f36f0 con 0x7fc1381018f0 2024-03-07T07:41:10.360 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:10.357+0000 7fc136d9d640 1 --2- 172.21.15.160:0/755117927 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fc1200a1ea0 0x7fc1200a4360 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc1280058c0 tx=0x7fc128010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:10.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:10.366+0000 7fc11ffff640 1 -- 172.21.15.160:0/755117927 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc130041070 con 0x7fc1381018f0 2024-03-07T07:41:10.588 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:10.587+0000 7fc13d083640 1 -- 172.21.15.160:0/755117927 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) v1 -- 0x7fc1381030a0 con 0x7fc1381018f0 2024-03-07T07:41:11.356 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.356+0000 7fc11ffff640 1 -- 172.21.15.160:0/755117927 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v22) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc1300098b0 con 0x7fc1381018f0 2024-03-07T07:41:11.360 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.359+0000 7fc13d083640 1 -- 172.21.15.160:0/755117927 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fc1200a1ea0 msgr2=0x7fc1200a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:11.360 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.359+0000 7fc13d083640 1 --2- 172.21.15.160:0/755117927 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fc1200a1ea0 0x7fc1200a4360 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc1280058c0 tx=0x7fc128010040 comp rx=0 tx=0).stop 2024-03-07T07:41:11.360 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.359+0000 7fc13d083640 1 -- 172.21.15.160:0/755117927 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1381018f0 msgr2=0x7fc13819eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:11.360 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.359+0000 7fc13d083640 1 --2- 172.21.15.160:0/755117927 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1381018f0 0x7fc13819eea0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc13002c630 tx=0x7fc130032f20 comp rx=0 tx=0).stop 2024-03-07T07:41:11.360 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.359+0000 7fc13d083640 1 -- 172.21.15.160:0/755117927 shutdown_connections 2024-03-07T07:41:11.360 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.359+0000 7fc13d083640 1 --2- 172.21.15.160:0/755117927 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fc1200a1ea0 0x7fc1200a4360 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:11.360 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.359+0000 7fc13d083640 1 --2- 172.21.15.160:0/755117927 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1381018f0 0x7fc13819eea0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:11.360 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.359+0000 7fc13d083640 1 --2- 172.21.15.160:0/755117927 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc138100f30 0x7fc138197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:11.360 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.359+0000 7fc13d083640 1 --2- 172.21.15.160:0/755117927 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1380ffd40 0x7fc1381978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:11.360 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.359+0000 7fc13d083640 1 -- 172.21.15.160:0/755117927 >> 172.21.15.160:0/755117927 conn(0x7fc1380fb2b0 msgr2=0x7fc138106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:11.361 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.360+0000 7fc13d083640 1 -- 172.21.15.160:0/755117927 shutdown_connections 2024-03-07T07:41:11.361 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.360+0000 7fc13d083640 1 -- 172.21.15.160:0/755117927 wait complete. 2024-03-07T07:41:11.372 DEBUG:teuthology.orchestra.run.smithi160:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-03-07T07:41:11.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.478+0000 7fc182fa1640 1 -- 172.21.15.160:0/452398978 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc17c101fa0 msgr2=0x7fc17c108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:11.484 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.478+0000 7fc182fa1640 1 --2- 172.21.15.160:0/452398978 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc17c101fa0 0x7fc17c108830 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc174009be0 tx=0x7fc17402bef0 comp rx=0 tx=0).stop 2024-03-07T07:41:11.484 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.479+0000 7fc182fa1640 1 -- 172.21.15.160:0/452398978 shutdown_connections 2024-03-07T07:41:11.484 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.479+0000 7fc182fa1640 1 --2- 172.21.15.160:0/452398978 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc17c101fa0 0x7fc17c108830 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:11.484 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.479+0000 7fc182fa1640 1 --2- 172.21.15.160:0/452398978 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc17c1015e0 0x7fc17c101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:11.484 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.479+0000 7fc182fa1640 1 --2- 172.21.15.160:0/452398978 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc17c1003f0 0x7fc17c100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:11.484 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.479+0000 7fc182fa1640 1 -- 172.21.15.160:0/452398978 >> 172.21.15.160:0/452398978 conn(0x7fc17c0fb960 msgr2=0x7fc17c0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:11.484 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.479+0000 7fc182fa1640 1 -- 172.21.15.160:0/452398978 shutdown_connections 2024-03-07T07:41:11.484 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.479+0000 7fc182fa1640 1 -- 172.21.15.160:0/452398978 wait complete. 2024-03-07T07:41:11.484 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.479+0000 7fc182fa1640 1 Processor -- start 2024-03-07T07:41:11.484 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.479+0000 7fc182fa1640 1 -- start start 2024-03-07T07:41:11.484 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.479+0000 7fc182fa1640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc17c1003f0 0x7fc17c104380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:11.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.479+0000 7fc182fa1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc17c1015e0 0x7fc17c1029c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:11.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.479+0000 7fc182fa1640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc17c101fa0 0x7fc17c102f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:11.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.479+0000 7fc182fa1640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17c10c080 con 0x7fc17c101fa0 2024-03-07T07:41:11.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.479+0000 7fc182fa1640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17c10bf00 con 0x7fc17c1003f0 2024-03-07T07:41:11.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.479+0000 7fc182fa1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17c10c200 con 0x7fc17c1015e0 2024-03-07T07:41:11.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.479+0000 7fc1827a0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc17c101fa0 0x7fc17c102f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:11.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.479+0000 7fc18179e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc17c1015e0 0x7fc17c1029c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:11.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.479+0000 7fc1827a0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc17c101fa0 0x7fc17c102f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:35642/0 (socket says 172.21.15.160:35642) 2024-03-07T07:41:11.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.479+0000 7fc181f9f640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc17c1003f0 0x7fc17c104380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:11.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.479+0000 7fc1827a0640 1 -- 172.21.15.160:0/2192974032 learned_addr learned my addr 172.21.15.160:0/2192974032 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:41:11.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.480+0000 7fc1827a0640 1 -- 172.21.15.160:0/2192974032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc17c1015e0 msgr2=0x7fc17c1029c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:11.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.480+0000 7fc1827a0640 1 --2- 172.21.15.160:0/2192974032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc17c1015e0 0x7fc17c1029c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:11.486 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.480+0000 7fc1827a0640 1 -- 172.21.15.160:0/2192974032 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc17c1003f0 msgr2=0x7fc17c104380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:11.486 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.480+0000 7fc1827a0640 1 --2- 172.21.15.160:0/2192974032 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc17c1003f0 0x7fc17c104380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:11.486 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.480+0000 7fc1827a0640 1 -- 172.21.15.160:0/2192974032 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17c103790 con 0x7fc17c101fa0 2024-03-07T07:41:11.486 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.480+0000 7fc181f9f640 1 --2- 172.21.15.160:0/2192974032 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc17c1003f0 0x7fc17c104380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:41:11.486 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.480+0000 7fc1827a0640 1 --2- 172.21.15.160:0/2192974032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc17c101fa0 0x7fc17c102f00 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc174036040 tx=0x7fc1740337b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:11.486 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.480+0000 7fc172ffd640 1 -- 172.21.15.160:0/2192974032 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc174043020 con 0x7fc17c101fa0 2024-03-07T07:41:11.486 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.480+0000 7fc182fa1640 1 -- 172.21.15.160:0/2192974032 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc17c1a18c0 con 0x7fc17c101fa0 2024-03-07T07:41:11.486 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.480+0000 7fc172ffd640 1 -- 172.21.15.160:0/2192974032 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc17402ca60 con 0x7fc17c101fa0 2024-03-07T07:41:11.487 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.480+0000 7fc172ffd640 1 -- 172.21.15.160:0/2192974032 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc174034700 con 0x7fc17c101fa0 2024-03-07T07:41:11.487 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.480+0000 7fc182fa1640 1 -- 172.21.15.160:0/2192974032 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc17c1a1e10 con 0x7fc17c101fa0 2024-03-07T07:41:11.487 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.481+0000 7fc182fa1640 1 -- 172.21.15.160:0/2192974032 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc17c1a2040 con 0x7fc17c101fa0 2024-03-07T07:41:11.487 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.482+0000 7fc172ffd640 1 -- 172.21.15.160:0/2192974032 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) v1 ==== 136754+0+0 (secure 0 0 0) 0x7fc174046020 con 0x7fc17c101fa0 2024-03-07T07:41:11.488 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.483+0000 7fc172ffd640 1 --2- 172.21.15.160:0/2192974032 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fc1680a1d70 0x7fc1680a4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:11.488 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.483+0000 7fc181f9f640 1 --2- 172.21.15.160:0/2192974032 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fc1680a1d70 0x7fc1680a4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:11.488 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.483+0000 7fc172ffd640 1 -- 172.21.15.160:0/2192974032 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc1740f0bb0 con 0x7fc17c101fa0 2024-03-07T07:41:11.488 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.483+0000 7fc181f9f640 1 --2- 172.21.15.160:0/2192974032 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fc1680a1d70 0x7fc1680a4230 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc1780096c0 tx=0x7fc178009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:11.489 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.487+0000 7fc172ffd640 1 -- 172.21.15.160:0/2192974032 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc17404d020 con 0x7fc17c101fa0 2024-03-07T07:41:11.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:11 smithi160 ceph-mon[31963]: pgmap v348: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-07T07:41:11.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:11 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/755117927' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-07T07:41:11.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:11 smithi160 ceph-mon[31963]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-07T07:41:11.687 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.687+0000 7fc182fa1640 1 -- 172.21.15.160:0/2192974032 --> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) v1 -- 0x7fc17c0fdb40 con 0x7fc1680a1d70 2024-03-07T07:41:11.703 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.703+0000 7fc172ffd640 1 -- 172.21.15.160:0/2192974032 <== mgr.14152 v2:172.21.15.160:6800/419630619 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3 (secure 0 0 0) 0x7fc17c0fdb40 con 0x7fc1680a1d70 2024-03-07T07:41:11.703 INFO:teuthology.orchestra.run.smithi160.stdout:{} 2024-03-07T07:41:11.706 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.705+0000 7fc182fa1640 1 -- 172.21.15.160:0/2192974032 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fc1680a1d70 msgr2=0x7fc1680a4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:11.706 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.705+0000 7fc182fa1640 1 --2- 172.21.15.160:0/2192974032 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fc1680a1d70 0x7fc1680a4230 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc1780096c0 tx=0x7fc178009260 comp rx=0 tx=0).stop 2024-03-07T07:41:11.706 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.705+0000 7fc182fa1640 1 -- 172.21.15.160:0/2192974032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc17c101fa0 msgr2=0x7fc17c102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:11.706 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.705+0000 7fc182fa1640 1 --2- 172.21.15.160:0/2192974032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc17c101fa0 0x7fc17c102f00 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc174036040 tx=0x7fc1740337b0 comp rx=0 tx=0).stop 2024-03-07T07:41:11.706 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.705+0000 7fc182fa1640 1 -- 172.21.15.160:0/2192974032 shutdown_connections 2024-03-07T07:41:11.707 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.705+0000 7fc182fa1640 1 --2- 172.21.15.160:0/2192974032 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fc1680a1d70 0x7fc1680a4230 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:11.707 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.705+0000 7fc182fa1640 1 --2- 172.21.15.160:0/2192974032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc17c101fa0 0x7fc17c102f00 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:11.707 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.705+0000 7fc182fa1640 1 --2- 172.21.15.160:0/2192974032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc17c1015e0 0x7fc17c1029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:11.707 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.705+0000 7fc182fa1640 1 --2- 172.21.15.160:0/2192974032 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc17c1003f0 0x7fc17c104380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:11.707 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.705+0000 7fc182fa1640 1 -- 172.21.15.160:0/2192974032 >> 172.21.15.160:0/2192974032 conn(0x7fc17c0fb960 msgr2=0x7fc17c0fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:11.707 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.705+0000 7fc182fa1640 1 -- 172.21.15.160:0/2192974032 shutdown_connections 2024-03-07T07:41:11.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:41:11.705+0000 7fc182fa1640 1 -- 172.21.15.160:0/2192974032 wait complete. 2024-03-07T07:41:11.716 INFO:teuthology.run_tasks:Running task check-counter... 2024-03-07T07:41:11.729 INFO:teuthology.run_tasks:Running task background_exec... 2024-03-07T07:41:11.741 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi160.front.sepia.ceph.com 2024-03-07T07:41:11.741 DEBUG:teuthology.orchestra.run.smithi160:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-03-07T07:41:11.744 INFO:teuthology.run_tasks:Running task workunit... 2024-03-07T07:41:11.757 INFO:tasks.workunit:Pulling workunits from ref 8b896a9e145796119f0451201dc4d53ddac97db0 2024-03-07T07:41:11.758 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-07T07:41:11.758 INFO:tasks.workunit:timeout=3h 2024-03-07T07:41:11.758 INFO:tasks.workunit:cleanup=True 2024-03-07T07:41:11.758 DEBUG:teuthology.orchestra.run.smithi160:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-07T07:41:11.775 INFO:teuthology.orchestra.run.smithi160.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-07T07:41:11.775 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-03-07T07:41:11.775 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-03-07T07:41:11.775 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-07T07:41:11.775 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-03-07T07:41:11.775 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2024-03-07 07:39:20.236710995 +0000 2024-03-07T07:41:11.775 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2024-03-07 07:39:20.236710995 +0000 2024-03-07T07:41:11.775 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2024-03-07 07:41:09.906177532 +0000 2024-03-07T07:41:11.775 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: 2024-03-07 07:39:20.236710995 +0000 2024-03-07T07:41:11.775 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-07T07:41:11.776 DEBUG:teuthology.orchestra.run.smithi160:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-07T07:41:11.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:11 smithi178 ceph-mon[36650]: pgmap v348: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-07T07:41:11.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:11 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/755117927' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-07T07:41:11.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:11 smithi178 ceph-mon[36650]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-07T07:41:11.849 DEBUG:teuthology.orchestra.run.smithi160:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 8b896a9e145796119f0451201dc4d53ddac97db0 2024-03-07T07:41:11.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:11 smithi177 ceph-mon[36578]: pgmap v348: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-07T07:41:11.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:11 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/755117927' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-07T07:41:11.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:11 smithi177 ceph-mon[36578]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-07T07:41:11.904 INFO:tasks.workunit.client.0.smithi160.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-07T07:41:12.177 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:41:11 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:41:11.692+0000 7fc506a65700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-07T07:41:12.178 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:41:11 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:41:11.692+0000 7fc506a65700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-07T07:41:12.178 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:41:11 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:41:11.692+0000 7fc506a65700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-07T07:41:12.178 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:41:11 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:41:11.692+0000 7fc506a65700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-07T07:41:12.178 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:41:11 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x[32196]: 2024-03-07T07:41:11.692+0000 7fc506a65700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-07T07:41:12.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:12 smithi160 ceph-mon[31963]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-07T07:41:12.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:12 smithi160 ceph-mon[31963]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-03-07T07:41:12.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:12 smithi178 ceph-mon[36650]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-07T07:41:12.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:12 smithi178 ceph-mon[36650]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-03-07T07:41:12.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:12 smithi177 ceph-mon[36578]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-07T07:41:12.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:12 smithi177 ceph-mon[36578]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-03-07T07:41:13.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:13 smithi160 ceph-mon[31963]: from='client.15267 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:41:13.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:13 smithi160 ceph-mon[31963]: pgmap v349: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-07T07:41:13.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:13 smithi178 ceph-mon[36650]: from='client.15267 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:41:13.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:13 smithi178 ceph-mon[36650]: pgmap v349: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-07T07:41:13.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:13 smithi177 ceph-mon[36578]: from='client.15267 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-07T07:41:13.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:13 smithi177 ceph-mon[36578]: pgmap v349: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-07T07:41:15.680 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:15 smithi160 ceph-mon[31963]: pgmap v350: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-03-07T07:41:15.680 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:15 smithi160 ceph-mon[31963]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-03-07T07:41:15.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:15 smithi178 ceph-mon[36650]: pgmap v350: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-03-07T07:41:15.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:15 smithi178 ceph-mon[36650]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-03-07T07:41:15.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:15 smithi177 ceph-mon[36578]: pgmap v350: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-03-07T07:41:15.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:15 smithi177 ceph-mon[36578]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-03-07T07:41:17.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:17 smithi160 ceph-mon[31963]: pgmap v351: 97 pgs: 97 active+clean; 652 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-03-07T07:41:17.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:17 smithi178 ceph-mon[36650]: pgmap v351: 97 pgs: 97 active+clean; 652 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-03-07T07:41:17.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:17 smithi177 ceph-mon[36578]: pgmap v351: 97 pgs: 97 active+clean; 652 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-03-07T07:41:19.659 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:19 smithi177 ceph-mon[36578]: pgmap v352: 97 pgs: 97 active+clean; 652 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-03-07T07:41:19.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:19 smithi160 ceph-mon[31963]: pgmap v352: 97 pgs: 97 active+clean; 652 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-03-07T07:41:19.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:19 smithi178 ceph-mon[36650]: pgmap v352: 97 pgs: 97 active+clean; 652 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-03-07T07:41:20.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:20 smithi178 ceph-mon[36650]: pgmap v353: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-03-07T07:41:20.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:20 smithi177 ceph-mon[36578]: pgmap v353: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-03-07T07:41:20.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:20 smithi160 ceph-mon[31963]: pgmap v353: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-03-07T07:41:23.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:23 smithi177 ceph-mon[36578]: pgmap v354: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2024-03-07T07:41:23.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:23 smithi160 ceph-mon[31963]: pgmap v354: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2024-03-07T07:41:23.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:23 smithi178 ceph-mon[36650]: pgmap v354: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2024-03-07T07:41:24.898 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.883+0000 7faddcdeb640 1 -- 172.21.15.160:0/3471034177 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fadd8103240 msgr2=0x7fadd8109ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.883+0000 7faddcdeb640 1 --2- 172.21.15.160:0/3471034177 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fadd8103240 0x7fadd8109ad0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fadd0009900 tx=0x7fadd002bc30 comp rx=0 tx=0).stop 2024-03-07T07:41:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.884+0000 7faddcdeb640 1 -- 172.21.15.160:0/3471034177 shutdown_connections 2024-03-07T07:41:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.884+0000 7faddcdeb640 1 --2- 172.21.15.160:0/3471034177 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fadd8103240 0x7fadd8109ad0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.884+0000 7faddcdeb640 1 --2- 172.21.15.160:0/3471034177 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadd8102880 0x7fadd8102d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.884+0000 7faddcdeb640 1 --2- 172.21.15.160:0/3471034177 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fadd8101690 0x7fadd8101ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.884+0000 7faddcdeb640 1 -- 172.21.15.160:0/3471034177 >> 172.21.15.160:0/3471034177 conn(0x7fadd80fcc20 msgr2=0x7fadd80ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.884+0000 7faddcdeb640 1 -- 172.21.15.160:0/3471034177 shutdown_connections 2024-03-07T07:41:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.884+0000 7faddcdeb640 1 -- 172.21.15.160:0/3471034177 wait complete. 2024-03-07T07:41:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.884+0000 7faddcdeb640 1 Processor -- start 2024-03-07T07:41:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.885+0000 7faddcdeb640 1 -- start start 2024-03-07T07:41:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.885+0000 7faddcdeb640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fadd8101690 0x7fadd8075410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.885+0000 7faddcdeb640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fadd8102880 0x7fadd8073a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.885+0000 7faddcdeb640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadd8103240 0x7fadd8073f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.885+0000 7faddcdeb640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd810d310 con 0x7fadd8101690 2024-03-07T07:41:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.885+0000 7faddcdeb640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd810d190 con 0x7fadd8102880 2024-03-07T07:41:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.885+0000 7faddcdeb640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd810d490 con 0x7fadd8103240 2024-03-07T07:41:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.885+0000 7fadd659c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fadd8101690 0x7fadd8075410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.885+0000 7fadd659c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fadd8101690 0x7fadd8075410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59298/0 (socket says 172.21.15.160:59298) 2024-03-07T07:41:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.885+0000 7fadd659c640 1 -- 172.21.15.160:0/2326975660 learned_addr learned my addr 172.21.15.160:0/2326975660 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:41:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.885+0000 7fadd6d9d640 1 --2- 172.21.15.160:0/2326975660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadd8103240 0x7fadd8073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.885+0000 7fadd5d9b640 1 --2- 172.21.15.160:0/2326975660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fadd8102880 0x7fadd8073a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.885+0000 7fadd659c640 1 -- 172.21.15.160:0/2326975660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadd8103240 msgr2=0x7fadd8073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.885+0000 7fadd659c640 1 --2- 172.21.15.160:0/2326975660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadd8103240 0x7fadd8073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.885+0000 7fadd659c640 1 -- 172.21.15.160:0/2326975660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fadd8102880 msgr2=0x7fadd8073a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.885+0000 7fadd659c640 1 --2- 172.21.15.160:0/2326975660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fadd8102880 0x7fadd8073a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.885+0000 7fadd659c640 1 -- 172.21.15.160:0/2326975660 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd8074820 con 0x7fadd8101690 2024-03-07T07:41:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.885+0000 7fadd659c640 1 --2- 172.21.15.160:0/2326975660 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fadd8101690 0x7fadd8075410 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fadc800ebc0 tx=0x7fadc800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.885+0000 7fadc77fe640 1 -- 172.21.15.160:0/2326975660 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc8003b70 con 0x7fadd8101690 2024-03-07T07:41:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.886+0000 7faddcdeb640 1 -- 172.21.15.160:0/2326975660 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadd81a19b0 con 0x7fadd8101690 2024-03-07T07:41:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.886+0000 7fadc77fe640 1 -- 172.21.15.160:0/2326975660 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fadc8003d10 con 0x7fadd8101690 2024-03-07T07:41:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.886+0000 7fadc77fe640 1 -- 172.21.15.160:0/2326975660 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc8019980 con 0x7fadd8101690 2024-03-07T07:41:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.886+0000 7faddcdeb640 1 -- 172.21.15.160:0/2326975660 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadd81a1f50 con 0x7fadd8101690 2024-03-07T07:41:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.888+0000 7fadc77fe640 1 -- 172.21.15.160:0/2326975660 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 136805+0+0 (secure 0 0 0) 0x7fadc8019b20 con 0x7fadd8101690 2024-03-07T07:41:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.888+0000 7faddfbcf740 1 Processor -- start 2024-03-07T07:41:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.888+0000 7fadc77fe640 1 --2- 172.21.15.160:0/2326975660 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fadc00a1ec0 0x7fadc00a4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:24.904 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.888+0000 7fadc77fe640 1 -- 172.21.15.160:0/2326975660 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fadc80ce7b0 con 0x7fadd8101690 2024-03-07T07:41:24.904 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.889+0000 7fadd6d9d640 1 --2- 172.21.15.160:0/2326975660 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fadc00a1ec0 0x7fadc00a4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:24.904 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.889+0000 7faddfbcf740 1 -- start start 2024-03-07T07:41:24.904 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.889+0000 7faddfbcf740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5597ef9f81a0 0x5597ef9f85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:24.904 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.889+0000 7faddfbcf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597ef9f8b90 0x5597ef9fd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:24.904 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.889+0000 7faddfbcf740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5597ef9fdd00 0x5597efa00190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:24.904 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.889+0000 7faddfbcf740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597ef95d5c0 con 0x5597ef9fdd00 2024-03-07T07:41:24.904 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.889+0000 7faddfbcf740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597ef906900 con 0x5597ef9f81a0 2024-03-07T07:41:24.905 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.889+0000 7faddfbcf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597ef959660 con 0x5597ef9f8b90 2024-03-07T07:41:24.905 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.889+0000 7fadd659c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5597ef9f81a0 0x5597ef9f85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:24.905 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.889+0000 7fadd5d9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597ef9f8b90 0x5597ef9fd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:24.905 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.889+0000 7fadd659c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5597ef9f81a0 0x5597ef9f85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.160:32786/0 (socket says 172.21.15.160:32786) 2024-03-07T07:41:24.905 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.889+0000 7fadd5d9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597ef9f8b90 0x5597ef9fd690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.160:41850/0 (socket says 172.21.15.160:41850) 2024-03-07T07:41:24.905 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.889+0000 7fadd659c640 1 -- 172.21.15.160:0/3196519030 learned_addr learned my addr 172.21.15.160:0/3196519030 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:41:24.905 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.889+0000 7fadd6d9d640 1 --2- 172.21.15.160:0/3196519030 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5597ef9fdd00 0x5597efa00190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:24.905 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.889+0000 7fadd659c640 1 -- 172.21.15.160:0/3196519030 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597ef9f8b90 msgr2=0x5597ef9fd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:41:24.905 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.889+0000 7fadd659c640 1 --2- 172.21.15.160:0/3196519030 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597ef9f8b90 0x5597ef9fd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:24.906 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.889+0000 7fadd659c640 1 -- 172.21.15.160:0/3196519030 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5597ef9fdd00 msgr2=0x5597efa00190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:41:24.906 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.889+0000 7fadd659c640 1 --2- 172.21.15.160:0/3196519030 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5597ef9fdd00 0x5597efa00190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:24.906 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.889+0000 7fadd659c640 1 -- 172.21.15.160:0/3196519030 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597ef672210 con 0x5597ef9f81a0 2024-03-07T07:41:24.906 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.889+0000 7fadd6d9d640 1 --2- 172.21.15.160:0/3196519030 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5597ef9fdd00 0x5597efa00190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:41:24.906 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.889+0000 7fadd5d9b640 1 --2- 172.21.15.160:0/3196519030 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597ef9f8b90 0x5597ef9fd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:41:24.906 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.889+0000 7fadd6d9d640 1 --2- 172.21.15.160:0/2326975660 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fadc00a1ec0 0x7fadc00a4380 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fadd0005ab0 tx=0x7fadd0005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:24.906 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.889+0000 7fadd659c640 1 --2- 172.21.15.160:0/3196519030 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5597ef9f81a0 0x5597ef9f85c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fadc8097d50 tx=0x7fadc809af80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eaea9cbe760a8dab server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:24.906 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.890+0000 7fadc4ff9640 1 -- 172.21.15.160:0/3196519030 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc809b380 con 0x5597ef9f81a0 2024-03-07T07:41:24.907 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.890+0000 7fadc4ff9640 1 -- 172.21.15.160:0/3196519030 <== mon.2 v2:172.21.15.177:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fadc809f4d0 con 0x5597ef9f81a0 2024-03-07T07:41:24.907 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.890+0000 7fadc4ff9640 1 -- 172.21.15.160:0/3196519030 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc80a3690 con 0x5597ef9f81a0 2024-03-07T07:41:24.907 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.890+0000 7faddfbcf740 1 -- 172.21.15.160:0/3196519030 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5597ef9f81a0 msgr2=0x5597ef9f85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:24.907 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.890+0000 7faddfbcf740 1 --2- 172.21.15.160:0/3196519030 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5597ef9f81a0 0x5597ef9f85c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fadc8097d50 tx=0x7fadc809af80 comp rx=0 tx=0).stop 2024-03-07T07:41:24.907 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.890+0000 7faddfbcf740 1 -- 172.21.15.160:0/3196519030 shutdown_connections 2024-03-07T07:41:24.907 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.890+0000 7faddfbcf740 1 --2- 172.21.15.160:0/3196519030 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5597ef9fdd00 0x5597efa00190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:24.907 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.890+0000 7faddfbcf740 1 --2- 172.21.15.160:0/3196519030 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597ef9f8b90 0x5597ef9fd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:24.907 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.890+0000 7faddfbcf740 1 --2- 172.21.15.160:0/3196519030 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5597ef9f81a0 0x5597ef9f85c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.890+0000 7faddfbcf740 1 -- 172.21.15.160:0/3196519030 >> 172.21.15.160:0/3196519030 conn(0x5597ef9c0580 msgr2=0x5597ef9f3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.890+0000 7faddfbcf740 1 -- 172.21.15.160:0/3196519030 shutdown_connections 2024-03-07T07:41:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.890+0000 7faddfbcf740 1 -- 172.21.15.160:0/3196519030 wait complete. 2024-03-07T07:41:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.891+0000 7faddfbcf740 1 -- start start 2024-03-07T07:41:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.891+0000 7faddfbcf740 1 Processor -- start 2024-03-07T07:41:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.891+0000 7faddfbcf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597ef9f81a0 0x5597efa02560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.891+0000 7faddfbcf740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5597ef9f8b90 0x5597efa02aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.891+0000 7faddfbcf740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5597ef9fdd00 0x5597efa09ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:24.909 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.891+0000 7faddfbcf740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597efa00cb0 con 0x5597ef9f8b90 2024-03-07T07:41:24.909 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.891+0000 7faddfbcf740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597ef919d60 con 0x5597ef9fdd00 2024-03-07T07:41:24.909 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.891+0000 7faddfbcf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597efa00e30 con 0x5597ef9f81a0 2024-03-07T07:41:24.909 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.891+0000 7faddfbcf740 10 client.0 _ll_register_callbacks cb 0x5597ef671a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-07T07:41:24.909 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.891+0000 7fadd5d9b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5597ef9f8b90 0x5597efa02aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:24.909 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.891+0000 7fadd5d9b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5597ef9f8b90 0x5597efa02aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59304/0 (socket says 172.21.15.160:59304) 2024-03-07T07:41:24.909 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.891+0000 7fadd5d9b640 1 -- 172.21.15.160:0/4043695382 learned_addr learned my addr 172.21.15.160:0/4043695382 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:41:24.909 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.891+0000 7fadd6d9d640 1 --2- 172.21.15.160:0/4043695382 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5597ef9fdd00 0x5597efa09ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.891+0000 7fadd659c640 1 --2- 172.21.15.160:0/4043695382 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597ef9f81a0 0x5597efa02560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.891+0000 7fadd6d9d640 1 -- 172.21.15.160:0/4043695382 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597ef9f81a0 msgr2=0x5597efa02560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:41:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.891+0000 7fadd6d9d640 1 --2- 172.21.15.160:0/4043695382 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597ef9f81a0 0x5597efa02560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.891+0000 7fadd6d9d640 1 -- 172.21.15.160:0/4043695382 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5597ef9f8b90 msgr2=0x5597efa02aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:41:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.891+0000 7fadd6d9d640 1 --2- 172.21.15.160:0/4043695382 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5597ef9f8b90 0x5597efa02aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.892+0000 7fadd6d9d640 1 -- 172.21.15.160:0/4043695382 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5597ef73e620 con 0x5597ef9fdd00 2024-03-07T07:41:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.892+0000 7fadd5d9b640 1 --2- 172.21.15.160:0/4043695382 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5597ef9f8b90 0x5597efa02aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:41:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.892+0000 7fadd6d9d640 1 --2- 172.21.15.160:0/4043695382 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5597ef9fdd00 0x5597efa09ae0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fadd002c170 tx=0x7fadd002bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca4a13b56182f2f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:24.911 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.892+0000 7fadbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-03-07T07:41:24.911 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.892+0000 7fadbdffb640 1 -- 172.21.15.160:0/4043695382 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd0050070 con 0x5597ef9fdd00 2024-03-07T07:41:24.911 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.892+0000 7fadbdffb640 1 -- 172.21.15.160:0/4043695382 <== mon.2 v2:172.21.15.177:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fadd00048c0 con 0x5597ef9fdd00 2024-03-07T07:41:24.911 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.892+0000 7faddfbcf740 1 -- 172.21.15.160:0/4043695382 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fadc809f4d0 con 0x5597ef9fdd00 2024-03-07T07:41:24.911 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.892+0000 7fadbdffb640 1 -- 172.21.15.160:0/4043695382 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fadd005c050 con 0x5597ef9fdd00 2024-03-07T07:41:24.911 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.892+0000 7faddfbcf740 10 client.25124 fetch_fsmap learned FSMap version 22 2024-03-07T07:41:24.911 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.892+0000 7faddfbcf740 1 -- 172.21.15.160:0/4043695382 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5597efa0cec0 con 0x5597ef9fdd00 2024-03-07T07:41:24.911 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.892+0000 7fadbdffb640 1 -- 172.21.15.160:0/4043695382 <== mon.2 v2:172.21.15.177:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fadc809f4d0 con 0x5597ef9fdd00 2024-03-07T07:41:24.912 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.892+0000 7fadbdffb640 1 -- 172.21.15.160:0/4043695382 <== mon.2 v2:172.21.15.177:3300/0 5 ==== fsmap(e 22) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fadd00035f0 con 0x5597ef9fdd00 2024-03-07T07:41:24.912 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.892+0000 7faddfbcf740 10 client.25124 fetch_fsmap finished waiting for FSMap version 22 2024-03-07T07:41:24.912 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.892+0000 7faddfbcf740 10 client.25124 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-07T07:41:24.912 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.892+0000 7faddfbcf740 20 client.25124 populate_metadata read hostname 'smithi160' 2024-03-07T07:41:24.912 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.892+0000 7faddfbcf740 1 --2- 172.21.15.160:0/4043695382 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x5597efa0e550 0x5597efa10980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:24.912 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.892+0000 7faddfbcf740 4 client.25124 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-07T07:41:24.912 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.892+0000 7faddfbcf740 1 -- 172.21.15.160:0/4043695382 --> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5597ef96ab00 con 0x5597efa0e550 2024-03-07T07:41:24.912 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.893+0000 7fadd659c640 1 --2- 172.21.15.160:0/4043695382 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x5597efa0e550 0x5597efa10980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.893+0000 7fadd659c640 1 --2- 172.21.15.160:0/4043695382 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x5597efa0e550 0x5597efa10980 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7827592dde0414ca server_cookie=b562b4f271232cd8 in_seq=0 out_seq=0 2024-03-07T07:41:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.893+0000 7fadbdffb640 10 client.25124 ms_handle_connect on v2:172.21.15.178:6838/2947354024 2024-03-07T07:41:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.894+0000 7fadbdffb640 1 -- 172.21.15.160:0/4043695382 <== mds.0 v2:172.21.15.178:6838/2947354024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5597ef96ab00 con 0x5597efa0e550 2024-03-07T07:41:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.894+0000 7fadbdffb640 10 client.25124 handle_command_reply: tid=0 2024-03-07T07:41:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.894+0000 7faddfbcf740 1 client.25124 shutdown 2024-03-07T07:41:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.894+0000 7faddfbcf740 1 -- 172.21.15.160:0/4043695382 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5597ef9fdd00 msgr2=0x5597efa09ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.894+0000 7faddfbcf740 1 --2- 172.21.15.160:0/4043695382 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5597ef9fdd00 0x5597efa09ae0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fadd002c170 tx=0x7fadd002bc30 comp rx=0 tx=0).stop 2024-03-07T07:41:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.894+0000 7faddfbcf740 1 -- 172.21.15.160:0/4043695382 shutdown_connections 2024-03-07T07:41:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.894+0000 7faddfbcf740 1 --2- 172.21.15.160:0/4043695382 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x5597efa0e550 0x5597efa10980 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.894+0000 7faddfbcf740 1 --2- 172.21.15.160:0/4043695382 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5597ef9fdd00 0x5597efa09ae0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.894+0000 7faddfbcf740 1 --2- 172.21.15.160:0/4043695382 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5597ef9f8b90 0x5597efa02aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.894+0000 7faddfbcf740 1 --2- 172.21.15.160:0/4043695382 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597ef9f81a0 0x5597efa02560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.894+0000 7fadbdffb640 0 client.25124 ms_handle_reset on v2:172.21.15.178:6838/2947354024 2024-03-07T07:41:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.894+0000 7faddfbcf740 1 -- 172.21.15.160:0/4043695382 >> 172.21.15.160:0/4043695382 conn(0x5597ef9c0580 msgr2=0x5597ef9f2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.894+0000 7faddfbcf740 1 -- 172.21.15.160:0/4043695382 shutdown_connections 2024-03-07T07:41:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.894+0000 7faddfbcf740 1 -- 172.21.15.160:0/4043695382 wait complete. 2024-03-07T07:41:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.894+0000 7faddfbcf740 20 client.25124 trim_cache size 0 max 16384 2024-03-07T07:41:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.907+0000 7faddfbcf740 1 Processor -- start 2024-03-07T07:41:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.908+0000 7faddfbcf740 1 -- start start 2024-03-07T07:41:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.908+0000 7faddfbcf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597ef9f81a0 0x5597ef9f3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.908+0000 7faddfbcf740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5597ef9f8b90 0x5597efa09aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.908+0000 7faddfbcf740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5597ef9fdd00 0x5597ef9f3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.908+0000 7faddfbcf740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd81ad140 con 0x5597ef9f8b90 2024-03-07T07:41:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.908+0000 7faddfbcf740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd81aac40 con 0x5597ef9fdd00 2024-03-07T07:41:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.908+0000 7faddfbcf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd81a8cc0 con 0x5597ef9f81a0 2024-03-07T07:41:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.908+0000 7fadd5d9b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5597ef9f8b90 0x5597efa09aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.908+0000 7fadd5d9b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5597ef9f8b90 0x5597efa09aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59312/0 (socket says 172.21.15.160:59312) 2024-03-07T07:41:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.908+0000 7fadd5d9b640 1 -- 172.21.15.160:0/4096140586 learned_addr learned my addr 172.21.15.160:0/4096140586 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:41:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.908+0000 7fadd659c640 1 --2- 172.21.15.160:0/4096140586 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597ef9f81a0 0x5597ef9f3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.908+0000 7fadd6d9d640 1 --2- 172.21.15.160:0/4096140586 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5597ef9fdd00 0x5597ef9f3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.908+0000 7fadd5d9b640 1 -- 172.21.15.160:0/4096140586 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597ef9f81a0 msgr2=0x5597ef9f3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:41:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.908+0000 7fadd5d9b640 1 --2- 172.21.15.160:0/4096140586 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597ef9f81a0 0x5597ef9f3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.908+0000 7fadd5d9b640 1 -- 172.21.15.160:0/4096140586 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5597ef9fdd00 msgr2=0x5597ef9f3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:41:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.908+0000 7fadd5d9b640 1 --2- 172.21.15.160:0/4096140586 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5597ef9fdd00 0x5597ef9f3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.908+0000 7fadd5d9b640 1 -- 172.21.15.160:0/4096140586 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597ef9f99f0 con 0x5597ef9f8b90 2024-03-07T07:41:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.908+0000 7fadd5d9b640 1 --2- 172.21.15.160:0/4096140586 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5597ef9f8b90 0x5597efa09aa0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fadcc012610 tx=0x7fadcc005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39328d1fe6d4e938 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.908+0000 7fadbffff640 1 -- 172.21.15.160:0/4096140586 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadcc022070 con 0x5597ef9f8b90 2024-03-07T07:41:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.908+0000 7fadbffff640 1 -- 172.21.15.160:0/4096140586 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fadcc0035f0 con 0x5597ef9f8b90 2024-03-07T07:41:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.908+0000 7fadbffff640 1 -- 172.21.15.160:0/4096140586 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadcc00f7c0 con 0x5597ef9f8b90 2024-03-07T07:41:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.909+0000 7faddfbcf740 1 -- 172.21.15.160:0/4096140586 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5597ef9f8b90 msgr2=0x5597efa09aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.909+0000 7faddfbcf740 1 --2- 172.21.15.160:0/4096140586 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5597ef9f8b90 0x5597efa09aa0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fadcc012610 tx=0x7fadcc005f90 comp rx=0 tx=0).stop 2024-03-07T07:41:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.909+0000 7faddfbcf740 1 -- 172.21.15.160:0/4096140586 shutdown_connections 2024-03-07T07:41:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.909+0000 7faddfbcf740 1 --2- 172.21.15.160:0/4096140586 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5597ef9fdd00 0x5597ef9f3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.909+0000 7faddfbcf740 1 --2- 172.21.15.160:0/4096140586 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5597ef9f8b90 0x5597efa09aa0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.909+0000 7faddfbcf740 1 --2- 172.21.15.160:0/4096140586 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597ef9f81a0 0x5597ef9f3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.909+0000 7faddfbcf740 1 -- 172.21.15.160:0/4096140586 >> 172.21.15.160:0/4096140586 conn(0x5597ef9c0580 msgr2=0x5597ef9f9f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.909+0000 7faddfbcf740 1 -- 172.21.15.160:0/4096140586 shutdown_connections 2024-03-07T07:41:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.909+0000 7faddfbcf740 1 -- 172.21.15.160:0/4096140586 wait complete. 2024-03-07T07:41:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.909+0000 7faddfbcf740 1 -- start start 2024-03-07T07:41:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.909+0000 7faddfbcf740 1 Processor -- start 2024-03-07T07:41:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.910+0000 7faddfbcf740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5597ef9f81a0 0x5597efa07a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.910+0000 7faddfbcf740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5597ef9f8b90 0x5597efa07f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.910+0000 7faddfbcf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597ef9fdd00 0x5597efa08f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.910+0000 7faddfbcf740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd81a92c0 con 0x5597ef9f8b90 2024-03-07T07:41:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.910+0000 7faddfbcf740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd81aa5e0 con 0x5597ef9f81a0 2024-03-07T07:41:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.910+0000 7faddfbcf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd81a9920 con 0x5597ef9fdd00 2024-03-07T07:41:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.910+0000 7faddfbcf740 10 client.0 _ll_register_callbacks cb 0x7fadd0044070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-07T07:41:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.910+0000 7fadd5d9b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5597ef9f8b90 0x5597efa07f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.910+0000 7fadd5d9b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5597ef9f8b90 0x5597efa07f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:59326/0 (socket says 172.21.15.160:59326) 2024-03-07T07:41:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.910+0000 7fadd659c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5597ef9f81a0 0x5597efa07a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.910+0000 7fadd5d9b640 1 -- 172.21.15.160:0/2091361838 learned_addr learned my addr 172.21.15.160:0/2091361838 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:41:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.910+0000 7fadd6d9d640 1 --2- 172.21.15.160:0/2091361838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597ef9fdd00 0x5597efa08f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.910+0000 7fadd5d9b640 1 -- 172.21.15.160:0/2091361838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597ef9fdd00 msgr2=0x5597efa08f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:41:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.910+0000 7fadd5d9b640 1 --2- 172.21.15.160:0/2091361838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597ef9fdd00 0x5597efa08f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.910+0000 7fadd5d9b640 1 -- 172.21.15.160:0/2091361838 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5597ef9f81a0 msgr2=0x5597efa07a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:41:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.910+0000 7fadd5d9b640 1 --2- 172.21.15.160:0/2091361838 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5597ef9f81a0 0x5597efa07a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.910+0000 7fadd5d9b640 1 -- 172.21.15.160:0/2091361838 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5597efa09510 con 0x5597ef9f8b90 2024-03-07T07:41:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.910+0000 7fadd659c640 1 --2- 172.21.15.160:0/2091361838 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5597ef9f81a0 0x5597efa07a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:41:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.910+0000 7fadd5d9b640 1 --2- 172.21.15.160:0/2091361838 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5597ef9f8b90 0x5597efa07f80 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fadcc0039d0 tx=0x7fadcc003a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72f87ac5e2a88d69 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.910+0000 7fadbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-03-07T07:41:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.910+0000 7fadbdffb640 1 -- 172.21.15.160:0/2091361838 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadcc022070 con 0x5597ef9f8b90 2024-03-07T07:41:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.910+0000 7fadbdffb640 1 -- 172.21.15.160:0/2091361838 <== mon.0 v2:172.21.15.160:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fadcc005260 con 0x5597ef9f8b90 2024-03-07T07:41:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.910+0000 7faddfbcf740 1 -- 172.21.15.160:0/2091361838 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fadcc0035f0 con 0x5597ef9f8b90 2024-03-07T07:41:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.911+0000 7fadbdffb640 1 -- 172.21.15.160:0/2091361838 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fadcc0054a0 con 0x5597ef9f8b90 2024-03-07T07:41:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.911+0000 7faddfbcf740 10 client.15291 fetch_fsmap learned FSMap version 22 2024-03-07T07:41:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.911+0000 7faddfbcf740 1 -- 172.21.15.160:0/2091361838 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5597efa6a5d0 con 0x5597ef9f8b90 2024-03-07T07:41:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.911+0000 7fadbdffb640 1 -- 172.21.15.160:0/2091361838 <== mon.0 v2:172.21.15.160:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fadcc0035f0 con 0x5597ef9f8b90 2024-03-07T07:41:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.911+0000 7fadbdffb640 1 -- 172.21.15.160:0/2091361838 <== mon.0 v2:172.21.15.160:3300/0 5 ==== fsmap(e 22) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fadcc00f560 con 0x5597ef9f8b90 2024-03-07T07:41:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.911+0000 7faddfbcf740 10 client.15291 fetch_fsmap finished waiting for FSMap version 22 2024-03-07T07:41:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.911+0000 7faddfbcf740 10 client.15291 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-07T07:41:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.911+0000 7faddfbcf740 20 client.15291 populate_metadata read hostname 'smithi160' 2024-03-07T07:41:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.911+0000 7faddfbcf740 1 --2- 172.21.15.160:0/2091361838 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x5597efa6aea0 0x5597efa6d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.911+0000 7faddfbcf740 4 client.15291 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-07T07:41:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.911+0000 7faddfbcf740 1 -- 172.21.15.160:0/2091361838 --> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5597ef964e50 con 0x5597efa6aea0 2024-03-07T07:41:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.911+0000 7fadd659c640 1 --2- 172.21.15.160:0/2091361838 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x5597efa6aea0 0x5597efa6d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.911+0000 7fadd659c640 1 --2- 172.21.15.160:0/2091361838 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x5597efa6aea0 0x5597efa6d2c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=951106eccd79abf5 server_cookie=a784d2b5e43ac3a2 in_seq=0 out_seq=0 2024-03-07T07:41:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.911+0000 7fadbdffb640 10 client.15291 ms_handle_connect on v2:172.21.15.178:6838/2947354024 2024-03-07T07:41:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.921+0000 7fadbdffb640 1 -- 172.21.15.160:0/2091361838 <== mds.0 v2:172.21.15.178:6838/2947354024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5597ef964e50 con 0x5597efa6aea0 2024-03-07T07:41:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.921+0000 7fadbdffb640 10 client.15291 handle_command_reply: tid=0 2024-03-07T07:41:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.921+0000 7faddfbcf740 1 client.15291 shutdown 2024-03-07T07:41:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.921+0000 7faddfbcf740 1 -- 172.21.15.160:0/2091361838 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5597ef9f8b90 msgr2=0x5597efa07f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.921+0000 7faddfbcf740 1 --2- 172.21.15.160:0/2091361838 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5597ef9f8b90 0x5597efa07f80 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fadcc0039d0 tx=0x7fadcc003a00 comp rx=0 tx=0).stop 2024-03-07T07:41:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.921+0000 7faddfbcf740 1 -- 172.21.15.160:0/2091361838 shutdown_connections 2024-03-07T07:41:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.921+0000 7faddfbcf740 1 --2- 172.21.15.160:0/2091361838 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x5597efa6aea0 0x5597efa6d2c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.921+0000 7faddfbcf740 1 --2- 172.21.15.160:0/2091361838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597ef9fdd00 0x5597efa08f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.921+0000 7fadbdffb640 0 client.15291 ms_handle_reset on v2:172.21.15.178:6838/2947354024 2024-03-07T07:41:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.921+0000 7faddfbcf740 1 --2- 172.21.15.160:0/2091361838 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5597ef9f8b90 0x5597efa07f80 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.921+0000 7faddfbcf740 1 --2- 172.21.15.160:0/2091361838 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5597ef9f81a0 0x5597efa07a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.921+0000 7faddfbcf740 1 -- 172.21.15.160:0/2091361838 >> 172.21.15.160:0/2091361838 conn(0x5597ef9c0580 msgr2=0x5597ef9f35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.922+0000 7faddfbcf740 1 -- 172.21.15.160:0/2091361838 shutdown_connections 2024-03-07T07:41:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.922+0000 7faddfbcf740 1 -- 172.21.15.160:0/2091361838 wait complete. 2024-03-07T07:41:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.922+0000 7faddfbcf740 20 client.15291 trim_cache size 0 max 16384 2024-03-07T07:41:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stdout:{ 2024-03-07T07:41:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stdout: "message": "", 2024-03-07T07:41:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stdout: "return_code": 0 2024-03-07T07:41:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stdout:} 2024-03-07T07:41:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.922+0000 7faddcdeb640 1 -- 172.21.15.160:0/2326975660 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fadc00a1ec0 msgr2=0x7fadc00a4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.922+0000 7faddcdeb640 1 --2- 172.21.15.160:0/2326975660 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fadc00a1ec0 0x7fadc00a4380 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fadd0005ab0 tx=0x7fadd0005d50 comp rx=0 tx=0).stop 2024-03-07T07:41:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.922+0000 7faddcdeb640 1 -- 172.21.15.160:0/2326975660 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fadd8101690 msgr2=0x7fadd8075410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.922+0000 7faddcdeb640 1 --2- 172.21.15.160:0/2326975660 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fadd8101690 0x7fadd8075410 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fadc800ebc0 tx=0x7fadc800c6a0 comp rx=0 tx=0).stop 2024-03-07T07:41:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.922+0000 7faddcdeb640 1 -- 172.21.15.160:0/2326975660 shutdown_connections 2024-03-07T07:41:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.922+0000 7faddcdeb640 1 --2- 172.21.15.160:0/2326975660 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fadc00a1ec0 0x7fadc00a4380 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.922+0000 7faddcdeb640 1 --2- 172.21.15.160:0/2326975660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadd8103240 0x7fadd8073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.922+0000 7faddcdeb640 1 --2- 172.21.15.160:0/2326975660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fadd8102880 0x7fadd8073a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.922+0000 7faddcdeb640 1 --2- 172.21.15.160:0/2326975660 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fadd8101690 0x7fadd8075410 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.922+0000 7faddcdeb640 1 -- 172.21.15.160:0/2326975660 >> 172.21.15.160:0/2326975660 conn(0x7fadd80fcc20 msgr2=0x7fadd8105ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.922+0000 7faddcdeb640 1 -- 172.21.15.160:0/2326975660 shutdown_connections 2024-03-07T07:41:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:24.922+0000 7faddcdeb640 1 -- 172.21.15.160:0/2326975660 wait complete. 2024-03-07T07:41:25.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:25 smithi160 ceph-mon[31963]: pgmap v355: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-03-07T07:41:25.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:25 smithi178 ceph-mon[36650]: pgmap v355: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-03-07T07:41:25.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:25 smithi177 ceph-mon[36578]: pgmap v355: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-03-07T07:41:27.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:27 smithi160 ceph-mon[31963]: pgmap v356: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-03-07T07:41:27.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:27 smithi178 ceph-mon[36650]: pgmap v356: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-03-07T07:41:27.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:27 smithi177 ceph-mon[36578]: pgmap v356: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-03-07T07:41:29.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:29 smithi160 ceph-mon[31963]: pgmap v357: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-03-07T07:41:29.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:29 smithi178 ceph-mon[36650]: pgmap v357: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-03-07T07:41:29.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:29 smithi177 ceph-mon[36578]: pgmap v357: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-03-07T07:41:31.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:31 smithi160 ceph-mon[31963]: pgmap v358: 97 pgs: 97 active+clean; 655 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-07T07:41:31.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:31 smithi178 ceph-mon[36650]: pgmap v358: 97 pgs: 97 active+clean; 655 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-07T07:41:31.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:31 smithi177 ceph-mon[36578]: pgmap v358: 97 pgs: 97 active+clean; 655 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-07T07:41:32.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:32 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:41:32.687 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:32 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:41:32.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:32 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:41:33.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:33 smithi160 ceph-mon[31963]: pgmap v359: 97 pgs: 97 active+clean; 655 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-03-07T07:41:33.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:33 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:41:33.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:33 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:41:33.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:33 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:41:33.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:33 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:41:33.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:33 smithi178 ceph-mon[36650]: pgmap v359: 97 pgs: 97 active+clean; 655 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-03-07T07:41:33.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:33 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:41:33.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:33 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:41:33.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:33 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:41:33.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:33 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:41:33.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:33 smithi177 ceph-mon[36578]: pgmap v359: 97 pgs: 97 active+clean; 655 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-03-07T07:41:33.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:33 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:41:33.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:33 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:41:33.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:33 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:41:33.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:33 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:41:34.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:34 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:41:34.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:34 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:41:34.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:34 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:41:34.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:34 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:41:34.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:34 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:41:34.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:34 smithi178 ceph-mon[36650]: pgmap v360: 97 pgs: 97 active+clean; 655 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-03-07T07:41:34.824 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:34 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:41:34.825 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:34 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:41:34.825 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:34 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:41:34.825 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:34 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:41:34.825 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:34 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:41:34.825 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:34 smithi160 ceph-mon[31963]: pgmap v360: 97 pgs: 97 active+clean; 655 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-03-07T07:41:34.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:34 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:41:34.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:34 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:41:34.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:34 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:41:34.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:34 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:41:34.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:34 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:41:34.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:34 smithi177 ceph-mon[36578]: pgmap v360: 97 pgs: 97 active+clean; 655 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-03-07T07:41:37.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:37 smithi160 ceph-mon[31963]: pgmap v361: 97 pgs: 97 active+clean; 655 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-03-07T07:41:37.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:37 smithi178 ceph-mon[36650]: pgmap v361: 97 pgs: 97 active+clean; 655 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-03-07T07:41:37.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:37 smithi177 ceph-mon[36578]: pgmap v361: 97 pgs: 97 active+clean; 655 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-03-07T07:41:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.019+0000 7f0c07e2b640 1 -- 172.21.15.160:0/1127446131 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c00100580 msgr2=0x7f0c001009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.019+0000 7f0c07e2b640 1 --2- 172.21.15.160:0/1127446131 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c00100580 0x7f0c001009a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4009900 tx=0x7f0bf402bc30 comp rx=0 tx=0).stop 2024-03-07T07:41:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.019+0000 7f0c07e2b640 1 -- 172.21.15.160:0/1127446131 shutdown_connections 2024-03-07T07:41:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.019+0000 7f0c07e2b640 1 --2- 172.21.15.160:0/1127446131 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c00102130 0x7f0c001089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.019+0000 7f0c07e2b640 1 --2- 172.21.15.160:0/1127446131 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c00101770 0x7f0c00101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.019+0000 7f0c07e2b640 1 --2- 172.21.15.160:0/1127446131 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c00100580 0x7f0c001009a0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.019+0000 7f0c07e2b640 1 -- 172.21.15.160:0/1127446131 >> 172.21.15.160:0/1127446131 conn(0x7f0c000fbaf0 msgr2=0x7f0c000fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.019+0000 7f0c07e2b640 1 -- 172.21.15.160:0/1127446131 shutdown_connections 2024-03-07T07:41:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.019+0000 7f0c07e2b640 1 -- 172.21.15.160:0/1127446131 wait complete. 2024-03-07T07:41:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.020+0000 7f0c07e2b640 1 Processor -- start 2024-03-07T07:41:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.020+0000 7f0c07e2b640 1 -- start start 2024-03-07T07:41:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.020+0000 7f0c07e2b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c00100580 0x7f0c00197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.020+0000 7f0c07e2b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c00101770 0x7f0c00197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.020+0000 7f0c07e2b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c00102130 0x7f0c0019eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.020+0000 7f0c07e2b640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0010c210 con 0x7f0c00100580 2024-03-07T07:41:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.020+0000 7f0c07e2b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0010c090 con 0x7f0c00101770 2024-03-07T07:41:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.020+0000 7f0c07e2b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0010c390 con 0x7f0c00102130 2024-03-07T07:41:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.020+0000 7f0c05bc7640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c00100580 0x7f0c00197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.020+0000 7f0c05bc7640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c00100580 0x7f0c00197960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:35620/0 (socket says 172.21.15.160:35620) 2024-03-07T07:41:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.020+0000 7f0c05bc7640 1 -- 172.21.15.160:0/1632388435 learned_addr learned my addr 172.21.15.160:0/1632388435 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:41:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.020+0000 7f0c063c8640 1 --2- 172.21.15.160:0/1632388435 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c00102130 0x7f0c0019eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.020+0000 7f0c053c6640 1 --2- 172.21.15.160:0/1632388435 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c00101770 0x7f0c00197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.020+0000 7f0c05bc7640 1 -- 172.21.15.160:0/1632388435 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c00102130 msgr2=0x7f0c0019eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.020+0000 7f0c05bc7640 1 --2- 172.21.15.160:0/1632388435 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c00102130 0x7f0c0019eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.020+0000 7f0c05bc7640 1 -- 172.21.15.160:0/1632388435 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c00101770 msgr2=0x7f0c00197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.020+0000 7f0c05bc7640 1 --2- 172.21.15.160:0/1632388435 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c00101770 0x7f0c00197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.020+0000 7f0c05bc7640 1 -- 172.21.15.160:0/1632388435 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c0019f530 con 0x7f0c00100580 2024-03-07T07:41:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.021+0000 7f0c05bc7640 1 --2- 172.21.15.160:0/1632388435 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c00100580 0x7f0c00197960 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4000c00 tx=0x7f0bf402c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.021+0000 7f0bf2ffd640 1 -- 172.21.15.160:0/1632388435 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4041070 con 0x7f0c00100580 2024-03-07T07:41:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.021+0000 7f0bf2ffd640 1 -- 172.21.15.160:0/1632388435 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0bf4004450 con 0x7f0c00100580 2024-03-07T07:41:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.021+0000 7f0bf2ffd640 1 -- 172.21.15.160:0/1632388435 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4034440 con 0x7f0c00100580 2024-03-07T07:41:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.021+0000 7f0c07e2b640 1 -- 172.21.15.160:0/1632388435 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c0019f7c0 con 0x7f0c00100580 2024-03-07T07:41:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.021+0000 7f0c07e2b640 1 -- 172.21.15.160:0/1632388435 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c0019fc60 con 0x7f0c00100580 2024-03-07T07:41:38.030 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.023+0000 7f0bf2ffd640 1 -- 172.21.15.160:0/1632388435 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 136805+0+0 (secure 0 0 0) 0x7f0bf4044070 con 0x7f0c00100580 2024-03-07T07:41:38.030 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.024+0000 7f0c0ac14740 1 Processor -- start 2024-03-07T07:41:38.030 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.024+0000 7f0c0ac14740 1 -- start start 2024-03-07T07:41:38.030 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.024+0000 7f0c0ac14740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565cc0e81a0 0x5565cc0e85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:38.030 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.024+0000 7f0c0ac14740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565cc0e8b90 0x5565cc0ed690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.024+0000 7f0bf2ffd640 1 --2- 172.21.15.160:0/1632388435 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f0bec0a1ec0 0x7f0bec0a4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.024+0000 7f0c0ac14740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565cc0edd00 0x5565cc0f0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.024+0000 7f0c0ac14740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565cc04d5c0 con 0x5565cc0e81a0 2024-03-07T07:41:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.024+0000 7f0c0ac14740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565cbff6900 con 0x5565cc0edd00 2024-03-07T07:41:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.024+0000 7f0c0ac14740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565cc049660 con 0x5565cc0e8b90 2024-03-07T07:41:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.024+0000 7f0c063c8640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565cc0e81a0 0x5565cc0e85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.024+0000 7f0c063c8640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565cc0e81a0 0x5565cc0e85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:35626/0 (socket says 172.21.15.160:35626) 2024-03-07T07:41:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.024+0000 7f0c063c8640 1 -- 172.21.15.160:0/3114507924 learned_addr learned my addr 172.21.15.160:0/3114507924 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:41:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.024+0000 7f0c05bc7640 1 --2- 172.21.15.160:0/3114507924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565cc0e8b90 0x5565cc0ed690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.025+0000 7f0c053c6640 1 --2- 172.21.15.160:0/1632388435 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f0bec0a1ec0 0x7f0bec0a4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.025+0000 7f0c063c8640 1 --2- 172.21.15.160:0/3114507924 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565cc0edd00 0x5565cc0f0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.025+0000 7f0bf2ffd640 1 -- 172.21.15.160:0/1632388435 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0bf40ef920 con 0x7f0c00100580 2024-03-07T07:41:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.025+0000 7f0c05bc7640 1 -- 172.21.15.160:0/3114507924 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565cc0edd00 msgr2=0x5565cc0f0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:41:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.025+0000 7f0c05bc7640 1 --2- 172.21.15.160:0/3114507924 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565cc0edd00 0x5565cc0f0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.025+0000 7f0c05bc7640 1 -- 172.21.15.160:0/3114507924 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565cc0e81a0 msgr2=0x5565cc0e85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:41:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.025+0000 7f0c05bc7640 1 --2- 172.21.15.160:0/3114507924 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565cc0e81a0 0x5565cc0e85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.025+0000 7f0c05bc7640 1 -- 172.21.15.160:0/3114507924 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565cbd62210 con 0x5565cc0e8b90 2024-03-07T07:41:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.025+0000 7f0c05bc7640 1 --2- 172.21.15.160:0/3114507924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565cc0e8b90 0x5565cc0ed690 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0bf40ed3a0 tx=0x7f0bf404b770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4dad544da63f5d2d server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.025+0000 7f0c053c6640 1 --2- 172.21.15.160:0/1632388435 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f0bec0a1ec0 0x7f0bec0a4380 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0c00198e20 tx=0x7f0bfc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.025+0000 7f0bebfff640 1 -- 172.21.15.160:0/3114507924 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf404b070 con 0x5565cc0e8b90 2024-03-07T07:41:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.025+0000 7f0bebfff640 1 -- 172.21.15.160:0/3114507924 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0bf4004c80 con 0x5565cc0e8b90 2024-03-07T07:41:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.025+0000 7f0bebfff640 1 -- 172.21.15.160:0/3114507924 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf40c07a0 con 0x5565cc0e8b90 2024-03-07T07:41:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.026+0000 7f0c0ac14740 1 -- 172.21.15.160:0/3114507924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565cc0e8b90 msgr2=0x5565cc0ed690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.026+0000 7f0c0ac14740 1 --2- 172.21.15.160:0/3114507924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565cc0e8b90 0x5565cc0ed690 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0bf40ed3a0 tx=0x7f0bf404b770 comp rx=0 tx=0).stop 2024-03-07T07:41:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.026+0000 7f0c0ac14740 1 -- 172.21.15.160:0/3114507924 shutdown_connections 2024-03-07T07:41:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.026+0000 7f0c0ac14740 1 --2- 172.21.15.160:0/3114507924 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565cc0edd00 0x5565cc0f0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.026+0000 7f0c0ac14740 1 --2- 172.21.15.160:0/3114507924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565cc0e8b90 0x5565cc0ed690 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.026+0000 7f0c0ac14740 1 --2- 172.21.15.160:0/3114507924 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565cc0e81a0 0x5565cc0e85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.026+0000 7f0c0ac14740 1 -- 172.21.15.160:0/3114507924 >> 172.21.15.160:0/3114507924 conn(0x5565cc0b0580 msgr2=0x5565cc0e3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.026+0000 7f0c0ac14740 1 -- 172.21.15.160:0/3114507924 shutdown_connections 2024-03-07T07:41:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.026+0000 7f0c0ac14740 1 -- 172.21.15.160:0/3114507924 wait complete. 2024-03-07T07:41:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.026+0000 7f0c0ac14740 1 -- start start 2024-03-07T07:41:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.026+0000 7f0c0ac14740 1 Processor -- start 2024-03-07T07:41:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.027+0000 7f0c0ac14740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565cc0e81a0 0x5565cc0f21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.027+0000 7f0c0ac14740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565cc0e8b90 0x5565cc0f2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.027+0000 7f0c0ac14740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565cc0edd00 0x5565cc0f9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.027+0000 7f0c0ac14740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565cc0f0a70 con 0x5565cc0edd00 2024-03-07T07:41:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.027+0000 7f0c0ac14740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565cc009d60 con 0x5565cc0e81a0 2024-03-07T07:41:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.027+0000 7f0c0ac14740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565cc0f0bf0 con 0x5565cc0e8b90 2024-03-07T07:41:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.027+0000 7f0c0ac14740 10 client.0 _ll_register_callbacks cb 0x5565cbd61a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-07T07:41:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.027+0000 7f0c063c8640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565cc0edd00 0x5565cc0f9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.027+0000 7f0c063c8640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565cc0edd00 0x5565cc0f9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:35636/0 (socket says 172.21.15.160:35636) 2024-03-07T07:41:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.027+0000 7f0c063c8640 1 -- 172.21.15.160:0/3992971242 learned_addr learned my addr 172.21.15.160:0/3992971242 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:41:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.027+0000 7f0c053c6640 1 --2- 172.21.15.160:0/3992971242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565cc0e8b90 0x5565cc0f2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.027+0000 7f0c05bc7640 1 --2- 172.21.15.160:0/3992971242 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565cc0e81a0 0x5565cc0f21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.027+0000 7f0c063c8640 1 -- 172.21.15.160:0/3992971242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565cc0e8b90 msgr2=0x5565cc0f2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:41:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.027+0000 7f0c063c8640 1 --2- 172.21.15.160:0/3992971242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565cc0e8b90 0x5565cc0f2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.027+0000 7f0c063c8640 1 -- 172.21.15.160:0/3992971242 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565cc0e81a0 msgr2=0x5565cc0f21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:41:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.027+0000 7f0c063c8640 1 --2- 172.21.15.160:0/3992971242 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565cc0e81a0 0x5565cc0f21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.027+0000 7f0c063c8640 1 -- 172.21.15.160:0/3992971242 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5565cbe2e620 con 0x5565cc0edd00 2024-03-07T07:41:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.027+0000 7f0c063c8640 1 --2- 172.21.15.160:0/3992971242 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565cc0edd00 0x5565cc0f9740 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0bf80135d0 tx=0x7f0bf800f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b997bc9fcb6531d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.027+0000 7f0be97fa640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-03-07T07:41:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.028+0000 7f0be97fa640 1 -- 172.21.15.160:0/3992971242 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf8004b20 con 0x5565cc0edd00 2024-03-07T07:41:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.028+0000 7f0be97fa640 1 -- 172.21.15.160:0/3992971242 <== mon.0 v2:172.21.15.160:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0bf8009040 con 0x5565cc0edd00 2024-03-07T07:41:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.028+0000 7f0c0ac14740 1 -- 172.21.15.160:0/3992971242 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0bf4004c80 con 0x5565cc0edd00 2024-03-07T07:41:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.028+0000 7f0be97fa640 1 -- 172.21.15.160:0/3992971242 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bf8004cc0 con 0x5565cc0edd00 2024-03-07T07:41:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.028+0000 7f0c0ac14740 10 client.15303 fetch_fsmap learned FSMap version 22 2024-03-07T07:41:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.028+0000 7f0c0ac14740 1 -- 172.21.15.160:0/3992971242 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5565cc0fcb20 con 0x5565cc0edd00 2024-03-07T07:41:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.028+0000 7f0be97fa640 1 -- 172.21.15.160:0/3992971242 <== mon.0 v2:172.21.15.160:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0bf4004c80 con 0x5565cc0edd00 2024-03-07T07:41:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.028+0000 7f0be97fa640 1 -- 172.21.15.160:0/3992971242 <== mon.0 v2:172.21.15.160:3300/0 5 ==== fsmap(e 22) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f0bf8004520 con 0x5565cc0edd00 2024-03-07T07:41:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.028+0000 7f0c0ac14740 10 client.15303 fetch_fsmap finished waiting for FSMap version 22 2024-03-07T07:41:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.028+0000 7f0c0ac14740 10 client.15303 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-07T07:41:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.028+0000 7f0c0ac14740 20 client.15303 populate_metadata read hostname 'smithi160' 2024-03-07T07:41:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.028+0000 7f0c0ac14740 1 --2- 172.21.15.160:0/3992971242 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x5565cc0fe1a0 0x5565cc100610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.028+0000 7f0c0ac14740 4 client.15303 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-07T07:41:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.028+0000 7f0c0ac14740 1 -- 172.21.15.160:0/3992971242 --> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5565cc05ab00 con 0x5565cc0fe1a0 2024-03-07T07:41:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.028+0000 7f0c05bc7640 1 --2- 172.21.15.160:0/3992971242 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x5565cc0fe1a0 0x5565cc100610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.029+0000 7f0c05bc7640 1 --2- 172.21.15.160:0/3992971242 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x5565cc0fe1a0 0x5565cc100610 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=614a471d21de532 server_cookie=deb597cb4d2e7842 in_seq=0 out_seq=0 2024-03-07T07:41:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.029+0000 7f0be97fa640 10 client.15303 ms_handle_connect on v2:172.21.15.178:6838/2947354024 2024-03-07T07:41:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.030+0000 7f0be97fa640 1 -- 172.21.15.160:0/3992971242 <== mds.0 v2:172.21.15.178:6838/2947354024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5565cc05ab00 con 0x5565cc0fe1a0 2024-03-07T07:41:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.030+0000 7f0be97fa640 10 client.15303 handle_command_reply: tid=0 2024-03-07T07:41:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.030+0000 7f0c0ac14740 1 client.15303 shutdown 2024-03-07T07:41:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.030+0000 7f0c0ac14740 1 -- 172.21.15.160:0/3992971242 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565cc0edd00 msgr2=0x5565cc0f9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.030+0000 7f0c0ac14740 1 --2- 172.21.15.160:0/3992971242 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565cc0edd00 0x5565cc0f9740 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0bf80135d0 tx=0x7f0bf800f850 comp rx=0 tx=0).stop 2024-03-07T07:41:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.030+0000 7f0c0ac14740 1 -- 172.21.15.160:0/3992971242 shutdown_connections 2024-03-07T07:41:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.030+0000 7f0c0ac14740 1 --2- 172.21.15.160:0/3992971242 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x5565cc0fe1a0 0x5565cc100610 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.030+0000 7f0c0ac14740 1 --2- 172.21.15.160:0/3992971242 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565cc0edd00 0x5565cc0f9740 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.030+0000 7f0c0ac14740 1 --2- 172.21.15.160:0/3992971242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565cc0e8b90 0x5565cc0f2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.030+0000 7f0c0ac14740 1 --2- 172.21.15.160:0/3992971242 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565cc0e81a0 0x5565cc0f21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.030+0000 7f0be97fa640 0 client.15303 ms_handle_reset on v2:172.21.15.178:6838/2947354024 2024-03-07T07:41:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.030+0000 7f0c0ac14740 1 -- 172.21.15.160:0/3992971242 >> 172.21.15.160:0/3992971242 conn(0x5565cc0b0580 msgr2=0x5565cc0e2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:38.037 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.030+0000 7f0c0ac14740 1 -- 172.21.15.160:0/3992971242 shutdown_connections 2024-03-07T07:41:38.037 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.030+0000 7f0c0ac14740 1 -- 172.21.15.160:0/3992971242 wait complete. 2024-03-07T07:41:38.037 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.030+0000 7f0c0ac14740 20 client.15303 trim_cache size 0 max 16384 2024-03-07T07:41:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.044+0000 7f0c0ac14740 1 Processor -- start 2024-03-07T07:41:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.044+0000 7f0c0ac14740 1 -- start start 2024-03-07T07:41:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.044+0000 7f0c0ac14740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565cc0e81a0 0x5565cc0f21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.044+0000 7f0c0ac14740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565cc0e8b90 0x5565cc0f9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.044+0000 7f0c0ac14740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565cc0edd00 0x5565cc0f2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.044+0000 7f0c0ac14740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c001aaf70 con 0x5565cc0edd00 2024-03-07T07:41:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.044+0000 7f0c0ac14740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c001a8a70 con 0x5565cc0e81a0 2024-03-07T07:41:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.044+0000 7f0c0ac14740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c001a6af0 con 0x5565cc0e8b90 2024-03-07T07:41:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.044+0000 7f0c063c8640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565cc0edd00 0x5565cc0f2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.044+0000 7f0c063c8640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565cc0edd00 0x5565cc0f2700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:35644/0 (socket says 172.21.15.160:35644) 2024-03-07T07:41:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.044+0000 7f0c063c8640 1 -- 172.21.15.160:0/1154647717 learned_addr learned my addr 172.21.15.160:0/1154647717 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:41:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.045+0000 7f0c053c6640 1 --2- 172.21.15.160:0/1154647717 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565cc0e8b90 0x5565cc0f9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.045+0000 7f0c05bc7640 1 --2- 172.21.15.160:0/1154647717 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565cc0e81a0 0x5565cc0f21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.045+0000 7f0c063c8640 1 -- 172.21.15.160:0/1154647717 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565cc0e8b90 msgr2=0x5565cc0f9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:41:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.045+0000 7f0c063c8640 1 --2- 172.21.15.160:0/1154647717 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565cc0e8b90 0x5565cc0f9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.045+0000 7f0c063c8640 1 -- 172.21.15.160:0/1154647717 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565cc0e81a0 msgr2=0x5565cc0f21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:41:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.045+0000 7f0c063c8640 1 --2- 172.21.15.160:0/1154647717 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565cc0e81a0 0x5565cc0f21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.045+0000 7f0c063c8640 1 -- 172.21.15.160:0/1154647717 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565cc0e4860 con 0x5565cc0edd00 2024-03-07T07:41:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.045+0000 7f0c063c8640 1 --2- 172.21.15.160:0/1154647717 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565cc0edd00 0x5565cc0f2700 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8009270 tx=0x7f0bf8009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39aa29b7be89f5a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.045+0000 7f0beb7fe640 1 -- 172.21.15.160:0/1154647717 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf8004990 con 0x5565cc0edd00 2024-03-07T07:41:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.045+0000 7f0beb7fe640 1 -- 172.21.15.160:0/1154647717 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0bf80059e0 con 0x5565cc0edd00 2024-03-07T07:41:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.045+0000 7f0beb7fe640 1 -- 172.21.15.160:0/1154647717 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf802b4a0 con 0x5565cc0edd00 2024-03-07T07:41:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.046+0000 7f0c0ac14740 1 -- 172.21.15.160:0/1154647717 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565cc0edd00 msgr2=0x5565cc0f2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.046+0000 7f0c0ac14740 1 --2- 172.21.15.160:0/1154647717 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565cc0edd00 0x5565cc0f2700 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8009270 tx=0x7f0bf8009500 comp rx=0 tx=0).stop 2024-03-07T07:41:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.046+0000 7f0c0ac14740 1 -- 172.21.15.160:0/1154647717 shutdown_connections 2024-03-07T07:41:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.046+0000 7f0c0ac14740 1 --2- 172.21.15.160:0/1154647717 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565cc0edd00 0x5565cc0f2700 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.046+0000 7f0c0ac14740 1 --2- 172.21.15.160:0/1154647717 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565cc0e8b90 0x5565cc0f9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.046+0000 7f0c0ac14740 1 --2- 172.21.15.160:0/1154647717 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565cc0e81a0 0x5565cc0f21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.046+0000 7f0c0ac14740 1 -- 172.21.15.160:0/1154647717 >> 172.21.15.160:0/1154647717 conn(0x5565cc0b0580 msgr2=0x5565cc0e3630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.046+0000 7f0c0ac14740 1 -- 172.21.15.160:0/1154647717 shutdown_connections 2024-03-07T07:41:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.046+0000 7f0c0ac14740 1 -- 172.21.15.160:0/1154647717 wait complete. 2024-03-07T07:41:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.046+0000 7f0c0ac14740 1 -- start start 2024-03-07T07:41:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.046+0000 7f0c0ac14740 1 Processor -- start 2024-03-07T07:41:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.046+0000 7f0c0ac14740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565cc0e81a0 0x5565cc0f7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.046+0000 7f0c0ac14740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565cc0e8b90 0x5565cc0f7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.046+0000 7f0c0ac14740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565cc0edd00 0x5565cc0f8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.046+0000 7f0c0ac14740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c001a70f0 con 0x5565cc0e8b90 2024-03-07T07:41:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.046+0000 7f0c0ac14740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c001a8410 con 0x5565cc0e81a0 2024-03-07T07:41:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.046+0000 7f0c0ac14740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c001a7750 con 0x5565cc0edd00 2024-03-07T07:41:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.047+0000 7f0c0ac14740 10 client.0 _ll_register_callbacks cb 0x7f0bf8013ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-07T07:41:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.047+0000 7f0c053c6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565cc0e8b90 0x5565cc0f7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.047+0000 7f0c053c6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565cc0e8b90 0x5565cc0f7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:35652/0 (socket says 172.21.15.160:35652) 2024-03-07T07:41:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.047+0000 7f0c053c6640 1 -- 172.21.15.160:0/1389572325 learned_addr learned my addr 172.21.15.160:0/1389572325 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:41:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.047+0000 7f0c05bc7640 1 --2- 172.21.15.160:0/1389572325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565cc0e81a0 0x5565cc0f7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.047+0000 7f0c063c8640 1 --2- 172.21.15.160:0/1389572325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565cc0edd00 0x5565cc0f8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.047+0000 7f0c053c6640 1 -- 172.21.15.160:0/1389572325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565cc0edd00 msgr2=0x5565cc0f8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:41:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.047+0000 7f0c053c6640 1 --2- 172.21.15.160:0/1389572325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565cc0edd00 0x5565cc0f8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.047+0000 7f0c053c6640 1 -- 172.21.15.160:0/1389572325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565cc0e81a0 msgr2=0x5565cc0f7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:41:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.047+0000 7f0c053c6640 1 --2- 172.21.15.160:0/1389572325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565cc0e81a0 0x5565cc0f7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.047+0000 7f0c053c6640 1 -- 172.21.15.160:0/1389572325 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5565cc0f90e0 con 0x5565cc0e8b90 2024-03-07T07:41:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.047+0000 7f0c05bc7640 1 --2- 172.21.15.160:0/1389572325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565cc0e81a0 0x5565cc0f7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:41:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.047+0000 7f0c053c6640 1 --2- 172.21.15.160:0/1389572325 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565cc0e8b90 0x5565cc0f7b50 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc003aa0 tx=0x7f0bfc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1cdf8f74bf3619e server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.047+0000 7f0be97fa640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-03-07T07:41:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.047+0000 7f0be97fa640 1 -- 172.21.15.160:0/1389572325 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc02c060 con 0x5565cc0e8b90 2024-03-07T07:41:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.047+0000 7f0c0ac14740 1 -- 172.21.15.160:0/1389572325 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0bf80059e0 con 0x5565cc0e8b90 2024-03-07T07:41:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.047+0000 7f0be97fa640 1 -- 172.21.15.160:0/1389572325 <== mon.0 v2:172.21.15.160:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0bfc01d8e0 con 0x5565cc0e8b90 2024-03-07T07:41:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.047+0000 7f0be97fa640 1 -- 172.21.15.160:0/1389572325 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bfc01dae0 con 0x5565cc0e8b90 2024-03-07T07:41:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.047+0000 7f0c0ac14740 10 client.15309 fetch_fsmap learned FSMap version 22 2024-03-07T07:41:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.047+0000 7f0c0ac14740 1 -- 172.21.15.160:0/1389572325 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5565cc15a3e0 con 0x5565cc0e8b90 2024-03-07T07:41:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.048+0000 7f0be97fa640 1 -- 172.21.15.160:0/1389572325 <== mon.0 v2:172.21.15.160:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0bf80059e0 con 0x5565cc0e8b90 2024-03-07T07:41:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.048+0000 7f0be97fa640 1 -- 172.21.15.160:0/1389572325 <== mon.0 v2:172.21.15.160:3300/0 5 ==== fsmap(e 22) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f0bfc031070 con 0x5565cc0e8b90 2024-03-07T07:41:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.048+0000 7f0c0ac14740 10 client.15309 fetch_fsmap finished waiting for FSMap version 22 2024-03-07T07:41:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.048+0000 7f0c0ac14740 10 client.15309 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-07T07:41:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.048+0000 7f0c0ac14740 20 client.15309 populate_metadata read hostname 'smithi160' 2024-03-07T07:41:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.048+0000 7f0c0ac14740 1 --2- 172.21.15.160:0/1389572325 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x5565cc15ada0 0x5565cc15d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.048+0000 7f0c0ac14740 4 client.15309 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-07T07:41:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.048+0000 7f0c0ac14740 1 -- 172.21.15.160:0/1389572325 --> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5565cc054e50 con 0x5565cc15ada0 2024-03-07T07:41:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.048+0000 7f0c05bc7640 1 --2- 172.21.15.160:0/1389572325 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x5565cc15ada0 0x5565cc15d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.048+0000 7f0be97fa640 10 client.15309 ms_handle_connect on v2:172.21.15.178:6838/2947354024 2024-03-07T07:41:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.048+0000 7f0c05bc7640 1 --2- 172.21.15.160:0/1389572325 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x5565cc15ada0 0x5565cc15d1c0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd51765fe9232dba server_cookie=a8410bf16becc216 in_seq=0 out_seq=0 2024-03-07T07:41:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.051+0000 7f0be97fa640 1 -- 172.21.15.160:0/1389572325 <== mds.0 v2:172.21.15.178:6838/2947354024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5565cc054e50 con 0x5565cc15ada0 2024-03-07T07:41:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.051+0000 7f0be97fa640 10 client.15309 handle_command_reply: tid=0 2024-03-07T07:41:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.051+0000 7f0c0ac14740 1 client.15309 shutdown 2024-03-07T07:41:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.051+0000 7f0c0ac14740 1 -- 172.21.15.160:0/1389572325 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565cc0e8b90 msgr2=0x5565cc0f7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.051+0000 7f0c0ac14740 1 --2- 172.21.15.160:0/1389572325 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565cc0e8b90 0x5565cc0f7b50 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc003aa0 tx=0x7f0bfc004730 comp rx=0 tx=0).stop 2024-03-07T07:41:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.051+0000 7f0c0ac14740 1 -- 172.21.15.160:0/1389572325 shutdown_connections 2024-03-07T07:41:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.051+0000 7f0c0ac14740 1 --2- 172.21.15.160:0/1389572325 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x5565cc15ada0 0x5565cc15d1c0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.051+0000 7f0c0ac14740 1 --2- 172.21.15.160:0/1389572325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5565cc0edd00 0x5565cc0f8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.051+0000 7f0c0ac14740 1 --2- 172.21.15.160:0/1389572325 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5565cc0e8b90 0x5565cc0f7b50 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.051+0000 7f0be97fa640 0 client.15309 ms_handle_reset on v2:172.21.15.178:6838/2947354024 2024-03-07T07:41:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.051+0000 7f0c0ac14740 1 --2- 172.21.15.160:0/1389572325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565cc0e81a0 0x5565cc0f7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.052+0000 7f0c0ac14740 1 -- 172.21.15.160:0/1389572325 >> 172.21.15.160:0/1389572325 conn(0x5565cc0b0580 msgr2=0x5565cc0ed070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.052+0000 7f0c0ac14740 1 -- 172.21.15.160:0/1389572325 shutdown_connections 2024-03-07T07:41:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.052+0000 7f0c0ac14740 1 -- 172.21.15.160:0/1389572325 wait complete. 2024-03-07T07:41:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.052+0000 7f0c0ac14740 20 client.15309 trim_cache size 0 max 16384 2024-03-07T07:41:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stdout:{ 2024-03-07T07:41:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stdout: "message": "", 2024-03-07T07:41:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stdout: "return_code": 0 2024-03-07T07:41:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stdout:} 2024-03-07T07:41:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.052+0000 7f0c07e2b640 1 -- 172.21.15.160:0/1632388435 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f0bec0a1ec0 msgr2=0x7f0bec0a4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.052+0000 7f0c07e2b640 1 --2- 172.21.15.160:0/1632388435 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f0bec0a1ec0 0x7f0bec0a4380 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0c00198e20 tx=0x7f0bfc005f90 comp rx=0 tx=0).stop 2024-03-07T07:41:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.052+0000 7f0c07e2b640 1 -- 172.21.15.160:0/1632388435 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c00100580 msgr2=0x7f0c00197960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.052+0000 7f0c07e2b640 1 --2- 172.21.15.160:0/1632388435 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c00100580 0x7f0c00197960 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4000c00 tx=0x7f0bf402c640 comp rx=0 tx=0).stop 2024-03-07T07:41:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.052+0000 7f0c07e2b640 1 -- 172.21.15.160:0/1632388435 shutdown_connections 2024-03-07T07:41:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.052+0000 7f0c07e2b640 1 --2- 172.21.15.160:0/1632388435 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f0bec0a1ec0 0x7f0bec0a4380 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.052+0000 7f0c07e2b640 1 --2- 172.21.15.160:0/1632388435 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c00102130 0x7f0c0019eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.052+0000 7f0c07e2b640 1 --2- 172.21.15.160:0/1632388435 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c00101770 0x7f0c00197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:38.059 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.052+0000 7f0c07e2b640 1 --2- 172.21.15.160:0/1632388435 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c00100580 0x7f0c00197960 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:38.059 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.052+0000 7f0c07e2b640 1 -- 172.21.15.160:0/1632388435 >> 172.21.15.160:0/1632388435 conn(0x7f0c000fbaf0 msgr2=0x7f0c000fd520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:38.059 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.052+0000 7f0c07e2b640 1 -- 172.21.15.160:0/1632388435 shutdown_connections 2024-03-07T07:41:38.059 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:38.052+0000 7f0c07e2b640 1 -- 172.21.15.160:0/1632388435 wait complete. 2024-03-07T07:41:39.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:39 smithi160 ceph-mon[31963]: pgmap v362: 97 pgs: 97 active+clean; 655 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-03-07T07:41:39.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:39 smithi178 ceph-mon[36650]: pgmap v362: 97 pgs: 97 active+clean; 655 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-03-07T07:41:39.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:39 smithi177 ceph-mon[36578]: pgmap v362: 97 pgs: 97 active+clean; 655 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-03-07T07:41:41.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:41 smithi160 ceph-mon[31963]: pgmap v363: 97 pgs: 97 active+clean; 656 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-03-07T07:41:41.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:41 smithi178 ceph-mon[36650]: pgmap v363: 97 pgs: 97 active+clean; 656 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-03-07T07:41:41.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:41 smithi177 ceph-mon[36578]: pgmap v363: 97 pgs: 97 active+clean; 656 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-03-07T07:41:43.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:43 smithi160 ceph-mon[31963]: pgmap v364: 97 pgs: 97 active+clean; 656 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-07T07:41:43.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:43 smithi178 ceph-mon[36650]: pgmap v364: 97 pgs: 97 active+clean; 656 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-07T07:41:43.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:43 smithi177 ceph-mon[36578]: pgmap v364: 97 pgs: 97 active+clean; 656 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-07T07:41:45.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:45 smithi160 ceph-mon[31963]: pgmap v365: 97 pgs: 97 active+clean; 656 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-07T07:41:45.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:45 smithi178 ceph-mon[36650]: pgmap v365: 97 pgs: 97 active+clean; 656 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-07T07:41:45.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:45 smithi177 ceph-mon[36578]: pgmap v365: 97 pgs: 97 active+clean; 656 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-07T07:41:47.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:47 smithi160 ceph-mon[31963]: pgmap v366: 97 pgs: 97 active+clean; 656 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-07T07:41:47.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:47 smithi178 ceph-mon[36650]: pgmap v366: 97 pgs: 97 active+clean; 656 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-07T07:41:47.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:47 smithi177 ceph-mon[36578]: pgmap v366: 97 pgs: 97 active+clean; 656 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-07T07:41:49.658 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:49 smithi177 ceph-mon[36578]: pgmap v367: 97 pgs: 97 active+clean; 656 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-07T07:41:49.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:49 smithi160 ceph-mon[31963]: pgmap v367: 97 pgs: 97 active+clean; 656 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-07T07:41:49.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:49 smithi178 ceph-mon[36650]: pgmap v367: 97 pgs: 97 active+clean; 656 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-07T07:41:51.144 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.141+0000 7f9db3954640 1 -- 172.21.15.160:0/781851889 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9dac103260 msgr2=0x7f9dac109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:51.144 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.141+0000 7f9db3954640 1 --2- 172.21.15.160:0/781851889 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9dac103260 0x7f9dac109af0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9da4009ca0 tx=0x7f9da402c0f0 comp rx=0 tx=0).stop 2024-03-07T07:41:51.144 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.141+0000 7f9db3954640 1 -- 172.21.15.160:0/781851889 shutdown_connections 2024-03-07T07:41:51.144 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.141+0000 7f9db3954640 1 --2- 172.21.15.160:0/781851889 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9dac103260 0x7f9dac109af0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:51.144 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.141+0000 7f9db3954640 1 --2- 172.21.15.160:0/781851889 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9dac1028a0 0x7f9dac102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:51.144 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.141+0000 7f9db3954640 1 --2- 172.21.15.160:0/781851889 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9dac1016b0 0x7f9dac101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:51.144 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.141+0000 7f9db3954640 1 -- 172.21.15.160:0/781851889 >> 172.21.15.160:0/781851889 conn(0x7f9dac0fcc40 msgr2=0x7f9dac0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:51.144 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.141+0000 7f9db3954640 1 -- 172.21.15.160:0/781851889 shutdown_connections 2024-03-07T07:41:51.144 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.141+0000 7f9db3954640 1 -- 172.21.15.160:0/781851889 wait complete. 2024-03-07T07:41:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.142+0000 7f9db3954640 1 Processor -- start 2024-03-07T07:41:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.142+0000 7f9db3954640 1 -- start start 2024-03-07T07:41:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.142+0000 7f9db3954640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9dac1016b0 0x7f9dac197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.142+0000 7f9db3954640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9dac1028a0 0x7f9dac197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.142+0000 7f9db3954640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9dac103260 0x7f9dac19ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.142+0000 7f9db3954640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dac10d330 con 0x7f9dac1016b0 2024-03-07T07:41:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.142+0000 7f9db3954640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dac10d1b0 con 0x7f9dac1028a0 2024-03-07T07:41:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.142+0000 7f9db3954640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dac10d4b0 con 0x7f9dac103260 2024-03-07T07:41:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.142+0000 7f9db16f0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9dac1016b0 0x7f9dac197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.142+0000 7f9db16f0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9dac1016b0 0x7f9dac197880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:55006/0 (socket says 172.21.15.160:55006) 2024-03-07T07:41:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.142+0000 7f9db16f0640 1 -- 172.21.15.160:0/3804079491 learned_addr learned my addr 172.21.15.160:0/3804079491 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:41:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.142+0000 7f9db0eef640 1 --2- 172.21.15.160:0/3804079491 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9dac1028a0 0x7f9dac197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:51.146 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.142+0000 7f9db1ef1640 1 --2- 172.21.15.160:0/3804079491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9dac103260 0x7f9dac19ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:51.146 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.142+0000 7f9db16f0640 1 -- 172.21.15.160:0/3804079491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9dac103260 msgr2=0x7f9dac19ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:51.146 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.142+0000 7f9db16f0640 1 --2- 172.21.15.160:0/3804079491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9dac103260 0x7f9dac19ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:51.146 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.142+0000 7f9db16f0640 1 -- 172.21.15.160:0/3804079491 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9dac1028a0 msgr2=0x7f9dac197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:51.146 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.142+0000 7f9db16f0640 1 --2- 172.21.15.160:0/3804079491 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9dac1028a0 0x7f9dac197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:51.146 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.142+0000 7f9db16f0640 1 -- 172.21.15.160:0/3804079491 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dac19f4a0 con 0x7f9dac1016b0 2024-03-07T07:41:51.146 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.142+0000 7f9db16f0640 1 --2- 172.21.15.160:0/3804079491 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9dac1016b0 0x7f9dac197880 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9da800e9c0 tx=0x7f9da800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:51.146 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.142+0000 7f9da27fc640 1 -- 172.21.15.160:0/3804079491 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da800cd80 con 0x7f9dac1016b0 2024-03-07T07:41:51.146 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.143+0000 7f9da27fc640 1 -- 172.21.15.160:0/3804079491 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9da80035f0 con 0x7f9dac1016b0 2024-03-07T07:41:51.146 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.143+0000 7f9db3954640 1 -- 172.21.15.160:0/3804079491 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dac19f790 con 0x7f9dac1016b0 2024-03-07T07:41:51.146 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.143+0000 7f9da27fc640 1 -- 172.21.15.160:0/3804079491 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da8010690 con 0x7f9dac1016b0 2024-03-07T07:41:51.146 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.143+0000 7f9db3954640 1 -- 172.21.15.160:0/3804079491 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dac19fce0 con 0x7f9dac1016b0 2024-03-07T07:41:51.154 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.145+0000 7f9da27fc640 1 -- 172.21.15.160:0/3804079491 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 136805+0+0 (secure 0 0 0) 0x7f9da80037a0 con 0x7f9dac1016b0 2024-03-07T07:41:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.145+0000 7f9db6818740 1 Processor -- start 2024-03-07T07:41:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.145+0000 7f9db6818740 1 -- start start 2024-03-07T07:41:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.145+0000 7f9db6818740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c4697ec5b0 0x55c4697ec9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.145+0000 7f9da27fc640 1 --2- 172.21.15.160:0/3804079491 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f9d980a1ec0 0x7f9d980a4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.145+0000 7f9db6818740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4697ecfa0 0x55c4697f1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.145+0000 7f9db6818740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c4697f2110 0x55c4697f45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.145+0000 7f9db6818740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c46974d5c0 con 0x55c4697f2110 2024-03-07T07:41:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.145+0000 7f9db6818740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4696f6900 con 0x55c4697ec5b0 2024-03-07T07:41:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.145+0000 7f9db6818740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c469749660 con 0x55c4697ecfa0 2024-03-07T07:41:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.146+0000 7f9db16f0640 1 --2- 172.21.15.160:0/3804079491 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f9d980a1ec0 0x7f9d980a4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.146+0000 7f9db0eef640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4697ecfa0 0x55c4697f1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.146+0000 7f9db0eef640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4697ecfa0 0x55c4697f1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.160:48286/0 (socket says 172.21.15.160:48286) 2024-03-07T07:41:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.146+0000 7f9db0eef640 1 -- 172.21.15.160:0/3990754195 learned_addr learned my addr 172.21.15.160:0/3990754195 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:41:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.146+0000 7f9db1ef1640 1 --2- 172.21.15.160:0/3990754195 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c4697ec5b0 0x55c4697ec9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.146+0000 7f9db1ef1640 1 --2- 172.21.15.160:0/3990754195 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c4697f2110 0x55c4697f45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.146+0000 7f9da27fc640 1 -- 172.21.15.160:0/3804079491 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9da80ce2c0 con 0x7f9dac1016b0 2024-03-07T07:41:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.146+0000 7f9db0eef640 1 -- 172.21.15.160:0/3990754195 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c4697ec5b0 msgr2=0x55c4697ec9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:41:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.146+0000 7f9db0eef640 1 --2- 172.21.15.160:0/3990754195 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c4697ec5b0 0x55c4697ec9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.146+0000 7f9db0eef640 1 -- 172.21.15.160:0/3990754195 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c4697f2110 msgr2=0x55c4697f45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:41:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.146+0000 7f9db0eef640 1 --2- 172.21.15.160:0/3990754195 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c4697f2110 0x55c4697f45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.146+0000 7f9db0eef640 1 -- 172.21.15.160:0/3990754195 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c469462210 con 0x55c4697ecfa0 2024-03-07T07:41:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.146+0000 7f9db1ef1640 1 --2- 172.21.15.160:0/3990754195 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c4697f2110 0x55c4697f45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:41:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.146+0000 7f9db16f0640 1 --2- 172.21.15.160:0/3804079491 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f9d980a1ec0 0x7f9d980a4380 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9da809ae20 tx=0x7f9da80097d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.146+0000 7f9db0eef640 1 --2- 172.21.15.160:0/3990754195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4697ecfa0 0x55c4697f1aa0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c009ca0 tx=0x7f9d9c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1060fc9a5d865b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.146+0000 7f9d977fe640 1 -- 172.21.15.160:0/3990754195 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c01e070 con 0x55c4697ecfa0 2024-03-07T07:41:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.146+0000 7f9d977fe640 1 -- 172.21.15.160:0/3990754195 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9d9c019b10 con 0x55c4697ecfa0 2024-03-07T07:41:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.146+0000 7f9d977fe640 1 -- 172.21.15.160:0/3990754195 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c00eaa0 con 0x55c4697ecfa0 2024-03-07T07:41:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.147+0000 7f9db6818740 1 -- 172.21.15.160:0/3990754195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4697ecfa0 msgr2=0x55c4697f1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.147+0000 7f9db6818740 1 --2- 172.21.15.160:0/3990754195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4697ecfa0 0x55c4697f1aa0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c009ca0 tx=0x7f9d9c011810 comp rx=0 tx=0).stop 2024-03-07T07:41:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.147+0000 7f9db6818740 1 -- 172.21.15.160:0/3990754195 shutdown_connections 2024-03-07T07:41:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.147+0000 7f9db6818740 1 --2- 172.21.15.160:0/3990754195 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c4697f2110 0x55c4697f45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.147+0000 7f9db6818740 1 --2- 172.21.15.160:0/3990754195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4697ecfa0 0x55c4697f1aa0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.147+0000 7f9db6818740 1 --2- 172.21.15.160:0/3990754195 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c4697ec5b0 0x55c4697ec9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.147+0000 7f9db6818740 1 -- 172.21.15.160:0/3990754195 >> 172.21.15.160:0/3990754195 conn(0x55c4697b0580 msgr2=0x55c4697e3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.147+0000 7f9db6818740 1 -- 172.21.15.160:0/3990754195 shutdown_connections 2024-03-07T07:41:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.147+0000 7f9db6818740 1 -- 172.21.15.160:0/3990754195 wait complete. 2024-03-07T07:41:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.147+0000 7f9db6818740 1 -- start start 2024-03-07T07:41:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.147+0000 7f9db6818740 1 Processor -- start 2024-03-07T07:41:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.148+0000 7f9db6818740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c4697ec5b0 0x55c4697f68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.148+0000 7f9db6818740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4697ecfa0 0x55c4697f6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.148+0000 7f9db6818740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c4697f2110 0x55c4697fde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.148+0000 7f9db6818740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4697f50c0 con 0x55c4697f2110 2024-03-07T07:41:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.148+0000 7f9db6818740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c469709d60 con 0x55c4697ec5b0 2024-03-07T07:41:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.148+0000 7f9db6818740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4697f5240 con 0x55c4697ecfa0 2024-03-07T07:41:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.148+0000 7f9db1ef1640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c4697f2110 0x55c4697fde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.148+0000 7f9db6818740 10 client.0 _ll_register_callbacks cb 0x55c469461a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-07T07:41:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.148+0000 7f9db1ef1640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c4697f2110 0x55c4697fde60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:55026/0 (socket says 172.21.15.160:55026) 2024-03-07T07:41:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.148+0000 7f9db0eef640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4697ecfa0 0x55c4697f6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.148+0000 7f9db1ef1640 1 -- 172.21.15.160:0/2104696383 learned_addr learned my addr 172.21.15.160:0/2104696383 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:41:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.148+0000 7f9db1ef1640 1 --2- 172.21.15.160:0/2104696383 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c4697ec5b0 0x55c4697f68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:51.159 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.148+0000 7f9db1ef1640 1 -- 172.21.15.160:0/2104696383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4697ecfa0 msgr2=0x55c4697f6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:41:51.159 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.148+0000 7f9db1ef1640 1 --2- 172.21.15.160:0/2104696383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4697ecfa0 0x55c4697f6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:51.159 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.148+0000 7f9db1ef1640 1 -- 172.21.15.160:0/2104696383 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c4697ec5b0 msgr2=0x55c4697f68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:41:51.159 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.148+0000 7f9db1ef1640 1 --2- 172.21.15.160:0/2104696383 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c4697ec5b0 0x55c4697f68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:51.159 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.148+0000 7f9db1ef1640 1 -- 172.21.15.160:0/2104696383 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c46952e620 con 0x55c4697f2110 2024-03-07T07:41:51.159 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.148+0000 7f9db0eef640 1 --2- 172.21.15.160:0/2104696383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4697ecfa0 0x55c4697f6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:41:51.159 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.148+0000 7f9db1ef1640 1 --2- 172.21.15.160:0/2104696383 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c4697f2110 0x55c4697fde60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9da40324c0 tx=0x7f9da402c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=377acaff5f602890 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:51.159 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.148+0000 7f9d94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-03-07T07:41:51.159 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.149+0000 7f9d94ff9640 1 -- 172.21.15.160:0/2104696383 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da4039040 con 0x55c4697f2110 2024-03-07T07:41:51.159 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.149+0000 7f9d94ff9640 1 -- 172.21.15.160:0/2104696383 <== mon.0 v2:172.21.15.160:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9da4003d80 con 0x55c4697f2110 2024-03-07T07:41:51.159 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.149+0000 7f9db6818740 1 -- 172.21.15.160:0/2104696383 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9d9c019b10 con 0x55c4697f2110 2024-03-07T07:41:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.149+0000 7f9d94ff9640 1 -- 172.21.15.160:0/2104696383 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9da4056050 con 0x55c4697f2110 2024-03-07T07:41:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.149+0000 7f9db6818740 10 client.15321 fetch_fsmap learned FSMap version 22 2024-03-07T07:41:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.149+0000 7f9db6818740 1 -- 172.21.15.160:0/2104696383 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55c469801240 con 0x55c4697f2110 2024-03-07T07:41:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.149+0000 7f9d94ff9640 1 -- 172.21.15.160:0/2104696383 <== mon.0 v2:172.21.15.160:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9d9c019b10 con 0x55c4697f2110 2024-03-07T07:41:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.149+0000 7f9d94ff9640 1 -- 172.21.15.160:0/2104696383 <== mon.0 v2:172.21.15.160:3300/0 5 ==== fsmap(e 22) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f9da404d070 con 0x55c4697f2110 2024-03-07T07:41:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.149+0000 7f9db6818740 10 client.15321 fetch_fsmap finished waiting for FSMap version 22 2024-03-07T07:41:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.149+0000 7f9db6818740 10 client.15321 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-07T07:41:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.149+0000 7f9db6818740 20 client.15321 populate_metadata read hostname 'smithi160' 2024-03-07T07:41:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.149+0000 7f9db6818740 1 --2- 172.21.15.160:0/2104696383 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x55c469802870 0x55c469804c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.149+0000 7f9db6818740 4 client.15321 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-07T07:41:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.149+0000 7f9db6818740 1 -- 172.21.15.160:0/2104696383 --> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c46975ab00 con 0x55c469802870 2024-03-07T07:41:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.149+0000 7f9db0eef640 1 --2- 172.21.15.160:0/2104696383 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x55c469802870 0x55c469804c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.150+0000 7f9d94ff9640 10 client.15321 ms_handle_connect on v2:172.21.15.178:6838/2947354024 2024-03-07T07:41:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.150+0000 7f9db0eef640 1 --2- 172.21.15.160:0/2104696383 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x55c469802870 0x55c469804c80 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c17faf6545acb92a server_cookie=6eccce42d4c1a244 in_seq=0 out_seq=0 2024-03-07T07:41:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.150+0000 7f9d94ff9640 1 -- 172.21.15.160:0/2104696383 <== mds.0 v2:172.21.15.178:6838/2947354024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c46975ab00 con 0x55c469802870 2024-03-07T07:41:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.150+0000 7f9d94ff9640 10 client.15321 handle_command_reply: tid=0 2024-03-07T07:41:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.150+0000 7f9db6818740 1 client.15321 shutdown 2024-03-07T07:41:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.150+0000 7f9db6818740 1 -- 172.21.15.160:0/2104696383 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c4697f2110 msgr2=0x55c4697fde60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.150+0000 7f9db6818740 1 --2- 172.21.15.160:0/2104696383 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c4697f2110 0x55c4697fde60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9da40324c0 tx=0x7f9da402c0f0 comp rx=0 tx=0).stop 2024-03-07T07:41:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.151+0000 7f9db6818740 1 -- 172.21.15.160:0/2104696383 shutdown_connections 2024-03-07T07:41:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.151+0000 7f9db6818740 1 --2- 172.21.15.160:0/2104696383 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x55c469802870 0x55c469804c80 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.151+0000 7f9d94ff9640 0 client.15321 ms_handle_reset on v2:172.21.15.178:6838/2947354024 2024-03-07T07:41:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.151+0000 7f9db6818740 1 --2- 172.21.15.160:0/2104696383 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c4697f2110 0x55c4697fde60 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.151+0000 7f9db6818740 1 --2- 172.21.15.160:0/2104696383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4697ecfa0 0x55c4697f6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.151+0000 7f9db6818740 1 --2- 172.21.15.160:0/2104696383 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c4697ec5b0 0x55c4697f68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.151+0000 7f9db6818740 1 -- 172.21.15.160:0/2104696383 >> 172.21.15.160:0/2104696383 conn(0x55c4697b0580 msgr2=0x55c4697e2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.151+0000 7f9db6818740 1 -- 172.21.15.160:0/2104696383 shutdown_connections 2024-03-07T07:41:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.151+0000 7f9db6818740 1 -- 172.21.15.160:0/2104696383 wait complete. 2024-03-07T07:41:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.151+0000 7f9db6818740 20 client.15321 trim_cache size 0 max 16384 2024-03-07T07:41:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.165+0000 7f9db6818740 1 Processor -- start 2024-03-07T07:41:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.165+0000 7f9db6818740 1 -- start start 2024-03-07T07:41:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.165+0000 7f9db6818740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c4697ec5b0 0x55c4697e2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.165+0000 7f9db6818740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c4697ecfa0 0x55c4697fde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.165+0000 7f9db6818740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4697f2110 0x55c4697e2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.165+0000 7f9db6818740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dac1a8cc0 con 0x55c4697ec5b0 2024-03-07T07:41:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.165+0000 7f9db6818740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dac1a67c0 con 0x55c4697ecfa0 2024-03-07T07:41:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.165+0000 7f9db6818740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dac1a4840 con 0x55c4697f2110 2024-03-07T07:41:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.165+0000 7f9db1ef1640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c4697ec5b0 0x55c4697e2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.165+0000 7f9db1ef1640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c4697ec5b0 0x55c4697e2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:55036/0 (socket says 172.21.15.160:55036) 2024-03-07T07:41:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.165+0000 7f9db1ef1640 1 -- 172.21.15.160:0/189462637 learned_addr learned my addr 172.21.15.160:0/189462637 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:41:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.165+0000 7f9db0eef640 1 --2- 172.21.15.160:0/189462637 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c4697ecfa0 0x55c4697fde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.165+0000 7f9db1ef1640 1 --2- 172.21.15.160:0/189462637 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4697f2110 0x55c4697e2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.165+0000 7f9db1ef1640 1 -- 172.21.15.160:0/189462637 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4697f2110 msgr2=0x55c4697e2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:41:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.165+0000 7f9db1ef1640 1 --2- 172.21.15.160:0/189462637 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4697f2110 0x55c4697e2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.165+0000 7f9db1ef1640 1 -- 172.21.15.160:0/189462637 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c4697ecfa0 msgr2=0x55c4697fde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:41:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.165+0000 7f9db1ef1640 1 --2- 172.21.15.160:0/189462637 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c4697ecfa0 0x55c4697fde20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.165+0000 7f9db1ef1640 1 -- 172.21.15.160:0/189462637 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4697edd70 con 0x55c4697ec5b0 2024-03-07T07:41:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.165+0000 7f9db1ef1640 1 --2- 172.21.15.160:0/189462637 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c4697ec5b0 0x55c4697e2580 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f9da4009bf0 tx=0x7f9da4032590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44f5da5f3dd109cb server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.166+0000 7f9d96ffd640 1 -- 172.21.15.160:0/189462637 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da4039040 con 0x55c4697ec5b0 2024-03-07T07:41:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.166+0000 7f9d96ffd640 1 -- 172.21.15.160:0/189462637 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9da4032dc0 con 0x55c4697ec5b0 2024-03-07T07:41:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.166+0000 7f9d96ffd640 1 -- 172.21.15.160:0/189462637 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da4035bc0 con 0x55c4697ec5b0 2024-03-07T07:41:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.166+0000 7f9db6818740 1 -- 172.21.15.160:0/189462637 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c4697ec5b0 msgr2=0x55c4697e2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.166+0000 7f9db6818740 1 --2- 172.21.15.160:0/189462637 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c4697ec5b0 0x55c4697e2580 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f9da4009bf0 tx=0x7f9da4032590 comp rx=0 tx=0).stop 2024-03-07T07:41:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.166+0000 7f9db6818740 1 -- 172.21.15.160:0/189462637 shutdown_connections 2024-03-07T07:41:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.166+0000 7f9db6818740 1 --2- 172.21.15.160:0/189462637 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4697f2110 0x55c4697e2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.166+0000 7f9db6818740 1 --2- 172.21.15.160:0/189462637 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c4697ecfa0 0x55c4697fde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.166+0000 7f9db6818740 1 --2- 172.21.15.160:0/189462637 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c4697ec5b0 0x55c4697e2580 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.166+0000 7f9db6818740 1 -- 172.21.15.160:0/189462637 >> 172.21.15.160:0/189462637 conn(0x55c4697b0580 msgr2=0x55c4697e6d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.166+0000 7f9db6818740 1 -- 172.21.15.160:0/189462637 shutdown_connections 2024-03-07T07:41:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.166+0000 7f9db6818740 1 -- 172.21.15.160:0/189462637 wait complete. 2024-03-07T07:41:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.167+0000 7f9db6818740 1 -- start start 2024-03-07T07:41:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.167+0000 7f9db6818740 1 Processor -- start 2024-03-07T07:41:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.167+0000 7f9db6818740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c4697ec5b0 0x55c4697fb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.167+0000 7f9db6818740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4697ecfa0 0x55c4697fbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.167+0000 7f9db6818740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c4697f2110 0x55c4697fce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.167+0000 7f9db6818740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dac1a4e40 con 0x55c4697f2110 2024-03-07T07:41:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.167+0000 7f9db6818740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dac1a6160 con 0x55c4697ec5b0 2024-03-07T07:41:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.167+0000 7f9db6818740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dac1a54a0 con 0x55c4697ecfa0 2024-03-07T07:41:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.167+0000 7f9db6818740 10 client.0 _ll_register_callbacks cb 0x7f9d9c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-07T07:41:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.167+0000 7f9db1ef1640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c4697f2110 0x55c4697fce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.167+0000 7f9db1ef1640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c4697f2110 0x55c4697fce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:55040/0 (socket says 172.21.15.160:55040) 2024-03-07T07:41:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.167+0000 7f9db1ef1640 1 -- 172.21.15.160:0/2257942565 learned_addr learned my addr 172.21.15.160:0/2257942565 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:41:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.167+0000 7f9db0eef640 1 --2- 172.21.15.160:0/2257942565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4697ecfa0 0x55c4697fbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.167+0000 7f9db1ef1640 1 --2- 172.21.15.160:0/2257942565 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c4697ec5b0 0x55c4697fb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.167+0000 7f9db0eef640 1 -- 172.21.15.160:0/2257942565 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c4697ec5b0 msgr2=0x55c4697fb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:41:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.167+0000 7f9db0eef640 1 --2- 172.21.15.160:0/2257942565 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c4697ec5b0 0x55c4697fb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.167+0000 7f9db0eef640 1 -- 172.21.15.160:0/2257942565 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c4697f2110 msgr2=0x55c4697fce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:41:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.167+0000 7f9db0eef640 1 --2- 172.21.15.160:0/2257942565 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c4697f2110 0x55c4697fce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.167+0000 7f9db0eef640 1 -- 172.21.15.160:0/2257942565 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c4697fd400 con 0x55c4697ecfa0 2024-03-07T07:41:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.167+0000 7f9db1ef1640 1 --2- 172.21.15.160:0/2257942565 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c4697f2110 0x55c4697fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:41:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.168+0000 7f9db0eef640 1 --2- 172.21.15.160:0/2257942565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4697ecfa0 0x55c4697fbe70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x55c4697e4a60 tx=0x7f9d9c0092a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36aec871bf190258 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:41:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.168+0000 7f9d94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-03-07T07:41:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.168+0000 7f9d94ff9640 1 -- 172.21.15.160:0/2257942565 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c01e040 con 0x55c4697ecfa0 2024-03-07T07:41:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.168+0000 7f9db6818740 1 -- 172.21.15.160:0/2257942565 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9da4032dc0 con 0x55c4697ecfa0 2024-03-07T07:41:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.168+0000 7f9d94ff9640 1 -- 172.21.15.160:0/2257942565 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9d9c005070 con 0x55c4697ecfa0 2024-03-07T07:41:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.168+0000 7f9d94ff9640 1 -- 172.21.15.160:0/2257942565 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d9c005290 con 0x55c4697ecfa0 2024-03-07T07:41:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.168+0000 7f9db6818740 10 client.25201 fetch_fsmap learned FSMap version 22 2024-03-07T07:41:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.168+0000 7f9db6818740 1 -- 172.21.15.160:0/2257942565 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55c46985e4d0 con 0x55c4697ecfa0 2024-03-07T07:41:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.168+0000 7f9d94ff9640 1 -- 172.21.15.160:0/2257942565 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9da4032dc0 con 0x55c4697ecfa0 2024-03-07T07:41:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.168+0000 7f9d94ff9640 1 -- 172.21.15.160:0/2257942565 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 22) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f9d9c00e4d0 con 0x55c4697ecfa0 2024-03-07T07:41:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.168+0000 7f9db6818740 10 client.25201 fetch_fsmap finished waiting for FSMap version 22 2024-03-07T07:41:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.168+0000 7f9db6818740 10 client.25201 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-07T07:41:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.168+0000 7f9db6818740 20 client.25201 populate_metadata read hostname 'smithi160' 2024-03-07T07:41:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.168+0000 7f9db6818740 1 --2- 172.21.15.160:0/2257942565 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x55c469802870 0x55c469860d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:41:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.168+0000 7f9db6818740 4 client.25201 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-07T07:41:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.168+0000 7f9db6818740 1 -- 172.21.15.160:0/2257942565 --> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c469754e50 con 0x55c469802870 2024-03-07T07:41:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.168+0000 7f9db1ef1640 1 --2- 172.21.15.160:0/2257942565 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x55c469802870 0x55c469860d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:41:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.169+0000 7f9d94ff9640 10 client.25201 ms_handle_connect on v2:172.21.15.178:6838/2947354024 2024-03-07T07:41:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.169+0000 7f9db1ef1640 1 --2- 172.21.15.160:0/2257942565 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x55c469802870 0x55c469860d70 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b6b8ba5f58db45b server_cookie=1d98a40b2a11add6 in_seq=0 out_seq=0 2024-03-07T07:41:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.175+0000 7f9d94ff9640 1 -- 172.21.15.160:0/2257942565 <== mds.0 v2:172.21.15.178:6838/2947354024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c469754e50 con 0x55c469802870 2024-03-07T07:41:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.175+0000 7f9d94ff9640 10 client.25201 handle_command_reply: tid=0 2024-03-07T07:41:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.175+0000 7f9db6818740 1 client.25201 shutdown 2024-03-07T07:41:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.175+0000 7f9db6818740 1 -- 172.21.15.160:0/2257942565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4697ecfa0 msgr2=0x55c4697fbe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.175+0000 7f9db6818740 1 --2- 172.21.15.160:0/2257942565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4697ecfa0 0x55c4697fbe70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x55c4697e4a60 tx=0x7f9d9c0092a0 comp rx=0 tx=0).stop 2024-03-07T07:41:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.175+0000 7f9db6818740 1 -- 172.21.15.160:0/2257942565 shutdown_connections 2024-03-07T07:41:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.175+0000 7f9db6818740 1 --2- 172.21.15.160:0/2257942565 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x55c469802870 0x55c469860d70 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.175+0000 7f9db6818740 1 --2- 172.21.15.160:0/2257942565 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c4697f2110 0x55c4697fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.175+0000 7f9db6818740 1 --2- 172.21.15.160:0/2257942565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4697ecfa0 0x55c4697fbe70 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.175+0000 7f9db6818740 1 --2- 172.21.15.160:0/2257942565 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c4697ec5b0 0x55c4697fb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.175+0000 7f9d94ff9640 0 client.25201 ms_handle_reset on v2:172.21.15.178:6838/2947354024 2024-03-07T07:41:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.175+0000 7f9db6818740 1 -- 172.21.15.160:0/2257942565 >> 172.21.15.160:0/2257942565 conn(0x55c4697b0580 msgr2=0x55c4698095d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.175+0000 7f9db6818740 1 -- 172.21.15.160:0/2257942565 shutdown_connections 2024-03-07T07:41:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.175+0000 7f9db6818740 1 -- 172.21.15.160:0/2257942565 wait complete. 2024-03-07T07:41:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.176+0000 7f9db6818740 20 client.25201 trim_cache size 0 max 16384 2024-03-07T07:41:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stdout:{ 2024-03-07T07:41:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stdout: "message": "", 2024-03-07T07:41:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stdout: "return_code": 0 2024-03-07T07:41:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stdout:} 2024-03-07T07:41:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.176+0000 7f9db3954640 1 -- 172.21.15.160:0/3804079491 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f9d980a1ec0 msgr2=0x7f9d980a4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.176+0000 7f9db3954640 1 --2- 172.21.15.160:0/3804079491 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f9d980a1ec0 0x7f9d980a4380 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9da809ae20 tx=0x7f9da80097d0 comp rx=0 tx=0).stop 2024-03-07T07:41:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.176+0000 7f9db3954640 1 -- 172.21.15.160:0/3804079491 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9dac1016b0 msgr2=0x7f9dac197880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:41:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.176+0000 7f9db3954640 1 --2- 172.21.15.160:0/3804079491 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9dac1016b0 0x7f9dac197880 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9da800e9c0 tx=0x7f9da800ee80 comp rx=0 tx=0).stop 2024-03-07T07:41:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.176+0000 7f9db3954640 1 -- 172.21.15.160:0/3804079491 shutdown_connections 2024-03-07T07:41:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.176+0000 7f9db3954640 1 --2- 172.21.15.160:0/3804079491 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f9d980a1ec0 0x7f9d980a4380 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.176+0000 7f9db3954640 1 --2- 172.21.15.160:0/3804079491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9dac103260 0x7f9dac19ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.176+0000 7f9db3954640 1 --2- 172.21.15.160:0/3804079491 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9dac1028a0 0x7f9dac197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.176+0000 7f9db3954640 1 --2- 172.21.15.160:0/3804079491 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9dac1016b0 0x7f9dac197880 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:41:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.176+0000 7f9db3954640 1 -- 172.21.15.160:0/3804079491 >> 172.21.15.160:0/3804079491 conn(0x7f9dac0fcc40 msgr2=0x7f9dac105ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:41:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.176+0000 7f9db3954640 1 -- 172.21.15.160:0/3804079491 shutdown_connections 2024-03-07T07:41:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:41:51.176+0000 7f9db3954640 1 -- 172.21.15.160:0/3804079491 wait complete. 2024-03-07T07:41:51.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:51 smithi160 ceph-mon[31963]: pgmap v368: 97 pgs: 97 active+clean; 665 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-03-07T07:41:51.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:51 smithi178 ceph-mon[36650]: pgmap v368: 97 pgs: 97 active+clean; 665 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-03-07T07:41:51.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:51 smithi177 ceph-mon[36578]: pgmap v368: 97 pgs: 97 active+clean; 665 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-03-07T07:41:53.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:53 smithi160 ceph-mon[31963]: pgmap v369: 97 pgs: 97 active+clean; 665 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1023 B/s wr, 5 op/s 2024-03-07T07:41:53.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:53 smithi178 ceph-mon[36650]: pgmap v369: 97 pgs: 97 active+clean; 665 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1023 B/s wr, 5 op/s 2024-03-07T07:41:53.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:53 smithi177 ceph-mon[36578]: pgmap v369: 97 pgs: 97 active+clean; 665 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1023 B/s wr, 5 op/s 2024-03-07T07:41:55.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:55 smithi160 ceph-mon[31963]: pgmap v370: 97 pgs: 97 active+clean; 665 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-03-07T07:41:55.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:55 smithi178 ceph-mon[36650]: pgmap v370: 97 pgs: 97 active+clean; 665 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-03-07T07:41:55.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:55 smithi177 ceph-mon[36578]: pgmap v370: 97 pgs: 97 active+clean; 665 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-03-07T07:41:57.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:57 smithi160 ceph-mon[31963]: pgmap v371: 97 pgs: 97 active+clean; 666 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-03-07T07:41:57.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:57 smithi178 ceph-mon[36650]: pgmap v371: 97 pgs: 97 active+clean; 666 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-03-07T07:41:57.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:57 smithi177 ceph-mon[36578]: pgmap v371: 97 pgs: 97 active+clean; 666 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-03-07T07:41:59.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:41:59 smithi160 ceph-mon[31963]: pgmap v372: 97 pgs: 97 active+clean; 666 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-03-07T07:41:59.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:41:59 smithi178 ceph-mon[36650]: pgmap v372: 97 pgs: 97 active+clean; 666 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-03-07T07:41:59.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:41:59 smithi177 ceph-mon[36578]: pgmap v372: 97 pgs: 97 active+clean; 666 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-03-07T07:42:01.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:42:01 smithi160 ceph-mon[31963]: pgmap v373: 97 pgs: 97 active+clean; 666 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-07T07:42:01.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:42:01 smithi178 ceph-mon[36650]: pgmap v373: 97 pgs: 97 active+clean; 666 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-07T07:42:01.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:42:01 smithi177 ceph-mon[36578]: pgmap v373: 97 pgs: 97 active+clean; 666 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-07T07:42:02.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:42:02 smithi160 ceph-mon[31963]: pgmap v374: 97 pgs: 97 active+clean; 666 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-03-07T07:42:02.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:42:02 smithi178 ceph-mon[36650]: pgmap v374: 97 pgs: 97 active+clean; 666 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-03-07T07:42:02.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:42:02 smithi177 ceph-mon[36578]: pgmap v374: 97 pgs: 97 active+clean; 666 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-03-07T07:42:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.299+0000 7f8e1de34640 1 -- 172.21.15.160:0/2551838949 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8e18102130 msgr2=0x7f8e181089c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:42:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.299+0000 7f8e1de34640 1 --2- 172.21.15.160:0/2551838949 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8e18102130 0x7f8e181089c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8e10009ca0 tx=0x7f8e1002c0f0 comp rx=0 tx=0).stop 2024-03-07T07:42:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.299+0000 7f8e1de34640 1 -- 172.21.15.160:0/2551838949 shutdown_connections 2024-03-07T07:42:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.299+0000 7f8e1de34640 1 --2- 172.21.15.160:0/2551838949 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8e18102130 0x7f8e181089c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.299+0000 7f8e1de34640 1 --2- 172.21.15.160:0/2551838949 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e18101770 0x7f8e18101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.299+0000 7f8e1de34640 1 --2- 172.21.15.160:0/2551838949 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8e18100580 0x7f8e181009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.299+0000 7f8e1de34640 1 -- 172.21.15.160:0/2551838949 >> 172.21.15.160:0/2551838949 conn(0x7f8e180fbaf0 msgr2=0x7f8e180fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:42:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.299+0000 7f8e1de34640 1 -- 172.21.15.160:0/2551838949 shutdown_connections 2024-03-07T07:42:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.299+0000 7f8e1de34640 1 -- 172.21.15.160:0/2551838949 wait complete. 2024-03-07T07:42:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.300+0000 7f8e1de34640 1 Processor -- start 2024-03-07T07:42:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.300+0000 7f8e1de34640 1 -- start start 2024-03-07T07:42:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.300+0000 7f8e1de34640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8e18100580 0x7f8e18197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.300+0000 7f8e1de34640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e18101770 0x7f8e18197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.300+0000 7f8e1de34640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8e18102130 0x7f8e1819ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.300+0000 7f8e1de34640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1810d320 con 0x7f8e18102130 2024-03-07T07:42:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.300+0000 7f8e1de34640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1810d1a0 con 0x7f8e18100580 2024-03-07T07:42:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.300+0000 7f8e1de34640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1810d4a0 con 0x7f8e18101770 2024-03-07T07:42:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.301+0000 7f8e177fe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8e18100580 0x7f8e18197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.301+0000 7f8e17fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8e18102130 0x7f8e1819ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.301+0000 7f8e177fe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8e18100580 0x7f8e18197890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.160:36112/0 (socket says 172.21.15.160:36112) 2024-03-07T07:42:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.301+0000 7f8e177fe640 1 -- 172.21.15.160:0/3731831197 learned_addr learned my addr 172.21.15.160:0/3731831197 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:42:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.301+0000 7f8e16ffd640 1 --2- 172.21.15.160:0/3731831197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e18101770 0x7f8e18197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.301+0000 7f8e17fff640 1 -- 172.21.15.160:0/3731831197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e18101770 msgr2=0x7f8e18197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:42:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.301+0000 7f8e17fff640 1 --2- 172.21.15.160:0/3731831197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e18101770 0x7f8e18197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.301+0000 7f8e17fff640 1 -- 172.21.15.160:0/3731831197 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8e18100580 msgr2=0x7f8e18197890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:42:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.301+0000 7f8e17fff640 1 --2- 172.21.15.160:0/3731831197 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8e18100580 0x7f8e18197890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.301+0000 7f8e17fff640 1 -- 172.21.15.160:0/3731831197 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e1819f4b0 con 0x7f8e18102130 2024-03-07T07:42:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.301+0000 7f8e177fe640 1 --2- 172.21.15.160:0/3731831197 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8e18100580 0x7f8e18197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:42:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.301+0000 7f8e17fff640 1 --2- 172.21.15.160:0/3731831197 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8e18102130 0x7f8e1819ee40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8e1002c630 tx=0x7f8e10032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:42:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.301+0000 7f8e14ff9640 1 -- 172.21.15.160:0/3731831197 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e10041020 con 0x7f8e18102130 2024-03-07T07:42:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.301+0000 7f8e1de34640 1 -- 172.21.15.160:0/3731831197 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e18109380 con 0x7f8e18102130 2024-03-07T07:42:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.301+0000 7f8e14ff9640 1 -- 172.21.15.160:0/3731831197 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e10034cd0 con 0x7f8e18102130 2024-03-07T07:42:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.301+0000 7f8e14ff9640 1 -- 172.21.15.160:0/3731831197 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e10039040 con 0x7f8e18102130 2024-03-07T07:42:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.302+0000 7f8e1de34640 1 -- 172.21.15.160:0/3731831197 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e18109840 con 0x7f8e18102130 2024-03-07T07:42:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.304+0000 7f8e14ff9640 1 -- 172.21.15.160:0/3731831197 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 136805+0+0 (secure 0 0 0) 0x7f8e10044020 con 0x7f8e18102130 2024-03-07T07:42:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.305+0000 7f8e14ff9640 1 --2- 172.21.15.160:0/3731831197 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f8e000a1ec0 0x7f8e000a4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.305+0000 7f8e20c36740 1 Processor -- start 2024-03-07T07:42:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.305+0000 7f8e20c36740 1 -- start start 2024-03-07T07:42:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.305+0000 7f8e177fe640 1 --2- 172.21.15.160:0/3731831197 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f8e000a1ec0 0x7f8e000a4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.305+0000 7f8e14ff9640 1 -- 172.21.15.160:0/3731831197 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8e100efbb0 con 0x7f8e18102130 2024-03-07T07:42:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.306+0000 7f8e20c36740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a5717d51a0 0x55a5717d55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.306+0000 7f8e20c36740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a5717d5b90 0x55a5717da690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.306+0000 7f8e20c36740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5717dad00 0x55a5717dd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.306+0000 7f8e20c36740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a57173a5c0 con 0x55a5717d5b90 2024-03-07T07:42:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.306+0000 7f8e20c36740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5716e3900 con 0x55a5717d51a0 2024-03-07T07:42:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.306+0000 7f8e20c36740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a571736660 con 0x55a5717dad00 2024-03-07T07:42:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.306+0000 7f8e177fe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a5717d51a0 0x55a5717d55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.306+0000 7f8e17fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5717dad00 0x55a5717dd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.306+0000 7f8e16ffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a5717d5b90 0x55a5717da690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.306+0000 7f8e17fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5717dad00 0x55a5717dd190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.160:48218/0 (socket says 172.21.15.160:48218) 2024-03-07T07:42:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.306+0000 7f8e17fff640 1 -- 172.21.15.160:0/114615356 learned_addr learned my addr 172.21.15.160:0/114615356 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:42:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.306+0000 7f8e177fe640 1 --2- 172.21.15.160:0/3731831197 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f8e000a1ec0 0x7f8e000a4380 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8e080095f0 tx=0x7f8e08012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:42:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.306+0000 7f8e17fff640 1 -- 172.21.15.160:0/114615356 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a5717d51a0 msgr2=0x55a5717d55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:42:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.306+0000 7f8e17fff640 1 --2- 172.21.15.160:0/114615356 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a5717d51a0 0x55a5717d55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.306+0000 7f8e17fff640 1 -- 172.21.15.160:0/114615356 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a5717d5b90 msgr2=0x55a5717da690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:42:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.306+0000 7f8e17fff640 1 --2- 172.21.15.160:0/114615356 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a5717d5b90 0x55a5717da690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.306+0000 7f8e17fff640 1 -- 172.21.15.160:0/114615356 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a57144f210 con 0x55a5717dad00 2024-03-07T07:42:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.306+0000 7f8e17fff640 1 --2- 172.21.15.160:0/114615356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5717dad00 0x55a5717dd190 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8e100bc8b0 tx=0x7f8e100b8c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc4577d7f86afabf server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:42:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.306+0000 7f8dfdffb640 1 -- 172.21.15.160:0/114615356 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e100312a0 con 0x55a5717dad00 2024-03-07T07:42:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.307+0000 7f8dfdffb640 1 -- 172.21.15.160:0/114615356 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e100098d0 con 0x55a5717dad00 2024-03-07T07:42:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.307+0000 7f8dfdffb640 1 -- 172.21.15.160:0/114615356 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e100b8f90 con 0x55a5717dad00 2024-03-07T07:42:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.307+0000 7f8e20c36740 1 -- 172.21.15.160:0/114615356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5717dad00 msgr2=0x55a5717dd190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:42:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.307+0000 7f8e20c36740 1 --2- 172.21.15.160:0/114615356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5717dad00 0x55a5717dd190 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8e100bc8b0 tx=0x7f8e100b8c80 comp rx=0 tx=0).stop 2024-03-07T07:42:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.307+0000 7f8e20c36740 1 -- 172.21.15.160:0/114615356 shutdown_connections 2024-03-07T07:42:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.307+0000 7f8e20c36740 1 --2- 172.21.15.160:0/114615356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5717dad00 0x55a5717dd190 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.307+0000 7f8e20c36740 1 --2- 172.21.15.160:0/114615356 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a5717d5b90 0x55a5717da690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.307+0000 7f8e20c36740 1 --2- 172.21.15.160:0/114615356 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a5717d51a0 0x55a5717d55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.307+0000 7f8e20c36740 1 -- 172.21.15.160:0/114615356 >> 172.21.15.160:0/114615356 conn(0x55a57179d580 msgr2=0x55a5717d0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:42:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.308+0000 7f8e20c36740 1 -- 172.21.15.160:0/114615356 shutdown_connections 2024-03-07T07:42:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.308+0000 7f8e20c36740 1 -- 172.21.15.160:0/114615356 wait complete. 2024-03-07T07:42:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.308+0000 7f8e20c36740 1 -- start start 2024-03-07T07:42:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.308+0000 7f8e20c36740 1 Processor -- start 2024-03-07T07:42:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.309+0000 7f8e20c36740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a5717d51a0 0x55a5717df170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.309+0000 7f8e20c36740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a5717d5b90 0x55a5717df6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.309+0000 7f8e20c36740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5717dad00 0x55a5717e66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.309+0000 7f8e20c36740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5717dda70 con 0x55a5717d51a0 2024-03-07T07:42:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.309+0000 7f8e20c36740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5716f6d60 con 0x55a5717d5b90 2024-03-07T07:42:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.309+0000 7f8e177fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a5717d51a0 0x55a5717df170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.309+0000 7f8e20c36740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5717ddbf0 con 0x55a5717dad00 2024-03-07T07:42:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.309+0000 7f8e177fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a5717d51a0 0x55a5717df170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:37900/0 (socket says 172.21.15.160:37900) 2024-03-07T07:42:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.309+0000 7f8e177fe640 1 -- 172.21.15.160:0/377976991 learned_addr learned my addr 172.21.15.160:0/377976991 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:42:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.309+0000 7f8e17fff640 1 --2- 172.21.15.160:0/377976991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5717dad00 0x55a5717e66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.309+0000 7f8e16ffd640 1 --2- 172.21.15.160:0/377976991 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a5717d5b90 0x55a5717df6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.309+0000 7f8e20c36740 10 client.0 _ll_register_callbacks cb 0x55a57144ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-07T07:42:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.309+0000 7f8e17fff640 1 -- 172.21.15.160:0/377976991 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a5717d5b90 msgr2=0x55a5717df6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:42:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.309+0000 7f8e17fff640 1 --2- 172.21.15.160:0/377976991 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a5717d5b90 0x55a5717df6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.309+0000 7f8e17fff640 1 -- 172.21.15.160:0/377976991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a5717d51a0 msgr2=0x55a5717df170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:42:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.309+0000 7f8e17fff640 1 --2- 172.21.15.160:0/377976991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a5717d51a0 0x55a5717df170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.309+0000 7f8e17fff640 1 -- 172.21.15.160:0/377976991 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a57151b620 con 0x55a5717dad00 2024-03-07T07:42:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.309+0000 7f8e177fe640 1 --2- 172.21.15.160:0/377976991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a5717d51a0 0x55a5717df170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:42:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.310+0000 7f8e17fff640 1 --2- 172.21.15.160:0/377976991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5717dad00 0x55a5717e66f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8e100bc8b0 tx=0x7f8e100b91c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b843263ba02387 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:42:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.310+0000 7f8df6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-03-07T07:42:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.310+0000 7f8df6ffd640 1 -- 172.21.15.160:0/377976991 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e100312a0 con 0x55a5717dad00 2024-03-07T07:42:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.310+0000 7f8df6ffd640 1 -- 172.21.15.160:0/377976991 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8e100bcd90 con 0x55a5717dad00 2024-03-07T07:42:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.310+0000 7f8e20c36740 1 -- 172.21.15.160:0/377976991 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a571747b00 con 0x55a5717dad00 2024-03-07T07:42:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.310+0000 7f8df6ffd640 1 -- 172.21.15.160:0/377976991 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e100bc920 con 0x55a5717dad00 2024-03-07T07:42:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.310+0000 7f8e20c36740 10 client.25213 fetch_fsmap learned FSMap version 22 2024-03-07T07:42:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.310+0000 7f8e20c36740 1 -- 172.21.15.160:0/377976991 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55a5717dda70 con 0x55a5717dad00 2024-03-07T07:42:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.310+0000 7f8df6ffd640 1 -- 172.21.15.160:0/377976991 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e100c0a70 con 0x55a5717dad00 2024-03-07T07:42:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.310+0000 7f8df6ffd640 1 -- 172.21.15.160:0/377976991 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 22) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f8e1003c030 con 0x55a5717dad00 2024-03-07T07:42:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.310+0000 7f8e20c36740 10 client.25213 fetch_fsmap finished waiting for FSMap version 22 2024-03-07T07:42:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.310+0000 7f8e20c36740 10 client.25213 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-07T07:42:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.310+0000 7f8e20c36740 20 client.25213 populate_metadata read hostname 'smithi160' 2024-03-07T07:42:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.310+0000 7f8e20c36740 1 --2- 172.21.15.160:0/377976991 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x55a5717eadd0 0x55a5717ed1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.310+0000 7f8e20c36740 4 client.25213 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-07T07:42:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.310+0000 7f8e20c36740 1 -- 172.21.15.160:0/377976991 --> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a571747b00 con 0x55a5717eadd0 2024-03-07T07:42:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.311+0000 7f8e177fe640 1 --2- 172.21.15.160:0/377976991 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x55a5717eadd0 0x55a5717ed1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.311+0000 7f8df6ffd640 10 client.25213 ms_handle_connect on v2:172.21.15.178:6838/2947354024 2024-03-07T07:42:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.311+0000 7f8e177fe640 1 --2- 172.21.15.160:0/377976991 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x55a5717eadd0 0x55a5717ed1c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=647a7217b3cbc47 server_cookie=7f8c11630b64bc57 in_seq=0 out_seq=0 2024-03-07T07:42:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.311+0000 7f8df6ffd640 1 -- 172.21.15.160:0/377976991 <== mds.0 v2:172.21.15.178:6838/2947354024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a571747b00 con 0x55a5717eadd0 2024-03-07T07:42:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.312+0000 7f8df6ffd640 10 client.25213 handle_command_reply: tid=0 2024-03-07T07:42:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.312+0000 7f8e20c36740 1 client.25213 shutdown 2024-03-07T07:42:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.312+0000 7f8e20c36740 1 -- 172.21.15.160:0/377976991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5717dad00 msgr2=0x55a5717e66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:42:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.312+0000 7f8e20c36740 1 --2- 172.21.15.160:0/377976991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5717dad00 0x55a5717e66f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8e100bc8b0 tx=0x7f8e100b91c0 comp rx=0 tx=0).stop 2024-03-07T07:42:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.312+0000 7f8e20c36740 1 -- 172.21.15.160:0/377976991 shutdown_connections 2024-03-07T07:42:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.312+0000 7f8e20c36740 1 --2- 172.21.15.160:0/377976991 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x55a5717eadd0 0x55a5717ed1c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.312+0000 7f8e20c36740 1 --2- 172.21.15.160:0/377976991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5717dad00 0x55a5717e66f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.312+0000 7f8df6ffd640 0 client.25213 ms_handle_reset on v2:172.21.15.178:6838/2947354024 2024-03-07T07:42:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.312+0000 7f8e20c36740 1 --2- 172.21.15.160:0/377976991 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a5717d5b90 0x55a5717df6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.312+0000 7f8e20c36740 1 --2- 172.21.15.160:0/377976991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a5717d51a0 0x55a5717df170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.312+0000 7f8e20c36740 1 -- 172.21.15.160:0/377976991 >> 172.21.15.160:0/377976991 conn(0x55a57179d580 msgr2=0x55a5717cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:42:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.312+0000 7f8e20c36740 1 -- 172.21.15.160:0/377976991 shutdown_connections 2024-03-07T07:42:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.312+0000 7f8e20c36740 1 -- 172.21.15.160:0/377976991 wait complete. 2024-03-07T07:42:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.312+0000 7f8e20c36740 20 client.25213 trim_cache size 0 max 16384 2024-03-07T07:42:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.331+0000 7f8e20c36740 1 Processor -- start 2024-03-07T07:42:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.331+0000 7f8e20c36740 1 -- start start 2024-03-07T07:42:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.332+0000 7f8e20c36740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5717d51a0 0x55a5717d3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.332+0000 7f8e20c36740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a5717d5b90 0x55a5717e66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.332+0000 7f8e20c36740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a5717dad00 0x55a5717d4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.332+0000 7f8e20c36740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e181a8cc0 con 0x55a5717dad00 2024-03-07T07:42:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.332+0000 7f8e20c36740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e181a67c0 con 0x55a5717d5b90 2024-03-07T07:42:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.332+0000 7f8e20c36740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e181a4840 con 0x55a5717d51a0 2024-03-07T07:42:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.332+0000 7f8e16ffd640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a5717d5b90 0x55a5717e66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.332+0000 7f8e177fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5717d51a0 0x55a5717d3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.332+0000 7f8e16ffd640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a5717d5b90 0x55a5717e66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.160:36136/0 (socket says 172.21.15.160:36136) 2024-03-07T07:42:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.332+0000 7f8e17fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a5717dad00 0x55a5717d4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.332+0000 7f8e16ffd640 1 -- 172.21.15.160:0/523975255 learned_addr learned my addr 172.21.15.160:0/523975255 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:42:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.332+0000 7f8e16ffd640 1 -- 172.21.15.160:0/523975255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5717d51a0 msgr2=0x55a5717d3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:42:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.332+0000 7f8e16ffd640 1 --2- 172.21.15.160:0/523975255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5717d51a0 0x55a5717d3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.332+0000 7f8e16ffd640 1 -- 172.21.15.160:0/523975255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a5717dad00 msgr2=0x55a5717d4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:42:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.332+0000 7f8e16ffd640 1 --2- 172.21.15.160:0/523975255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a5717dad00 0x55a5717d4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.332+0000 7f8e16ffd640 1 -- 172.21.15.160:0/523975255 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5717de250 con 0x55a5717d5b90 2024-03-07T07:42:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.332+0000 7f8e16ffd640 1 --2- 172.21.15.160:0/523975255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a5717d5b90 0x55a5717e66b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8e18198da0 tx=0x7f8e0c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be2f29949bec1458 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:42:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.332+0000 7f8dfd7fa640 1 -- 172.21.15.160:0/523975255 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c022070 con 0x55a5717d5b90 2024-03-07T07:42:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.333+0000 7f8dfd7fa640 1 -- 172.21.15.160:0/523975255 <== mon.2 v2:172.21.15.177:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e0c0035f0 con 0x55a5717d5b90 2024-03-07T07:42:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.333+0000 7f8dfd7fa640 1 -- 172.21.15.160:0/523975255 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c01d6a0 con 0x55a5717d5b90 2024-03-07T07:42:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.333+0000 7f8e20c36740 1 -- 172.21.15.160:0/523975255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a5717d5b90 msgr2=0x55a5717e66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:42:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.333+0000 7f8e20c36740 1 --2- 172.21.15.160:0/523975255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a5717d5b90 0x55a5717e66b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8e18198da0 tx=0x7f8e0c005f90 comp rx=0 tx=0).stop 2024-03-07T07:42:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.333+0000 7f8e20c36740 1 -- 172.21.15.160:0/523975255 shutdown_connections 2024-03-07T07:42:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.333+0000 7f8e20c36740 1 --2- 172.21.15.160:0/523975255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a5717dad00 0x55a5717d4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.333+0000 7f8e20c36740 1 --2- 172.21.15.160:0/523975255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a5717d5b90 0x55a5717e66b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.333+0000 7f8e20c36740 1 --2- 172.21.15.160:0/523975255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5717d51a0 0x55a5717d3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.333+0000 7f8e20c36740 1 -- 172.21.15.160:0/523975255 >> 172.21.15.160:0/523975255 conn(0x55a57179d580 msgr2=0x55a5717df150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:42:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.333+0000 7f8e20c36740 1 -- 172.21.15.160:0/523975255 shutdown_connections 2024-03-07T07:42:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.333+0000 7f8e20c36740 1 -- 172.21.15.160:0/523975255 wait complete. 2024-03-07T07:42:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.334+0000 7f8e20c36740 1 -- start start 2024-03-07T07:42:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.334+0000 7f8e20c36740 1 Processor -- start 2024-03-07T07:42:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.334+0000 7f8e20c36740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a5717d51a0 0x55a5717e4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.334+0000 7f8e20c36740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5717d5b90 0x55a5717e4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.334+0000 7f8e20c36740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a5717dad00 0x55a5717e5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.334+0000 7f8e20c36740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e181a4e40 con 0x55a5717d51a0 2024-03-07T07:42:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.334+0000 7f8e20c36740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e181a6160 con 0x55a5717dad00 2024-03-07T07:42:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.334+0000 7f8e20c36740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e181a54a0 con 0x55a5717d5b90 2024-03-07T07:42:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.334+0000 7f8e20c36740 10 client.0 _ll_register_callbacks cb 0x7f8e100b84c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-07T07:42:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.334+0000 7f8e177fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a5717d51a0 0x55a5717e4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.334+0000 7f8e177fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a5717d51a0 0x55a5717e4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:37922/0 (socket says 172.21.15.160:37922) 2024-03-07T07:42:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.334+0000 7f8e177fe640 1 -- 172.21.15.160:0/3336396117 learned_addr learned my addr 172.21.15.160:0/3336396117 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:42:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.334+0000 7f8e16ffd640 1 --2- 172.21.15.160:0/3336396117 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5717d5b90 0x55a5717e4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.334+0000 7f8e17fff640 1 --2- 172.21.15.160:0/3336396117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a5717dad00 0x55a5717e5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.335+0000 7f8e177fe640 1 -- 172.21.15.160:0/3336396117 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5717d5b90 msgr2=0x55a5717e4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:42:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.335+0000 7f8e177fe640 1 --2- 172.21.15.160:0/3336396117 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5717d5b90 0x55a5717e4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.335+0000 7f8e177fe640 1 -- 172.21.15.160:0/3336396117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a5717dad00 msgr2=0x55a5717e5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:42:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.335+0000 7f8e177fe640 1 --2- 172.21.15.160:0/3336396117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a5717dad00 0x55a5717e5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.335+0000 7f8e177fe640 1 -- 172.21.15.160:0/3336396117 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a5717e5e20 con 0x55a5717d51a0 2024-03-07T07:42:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.335+0000 7f8e177fe640 1 --2- 172.21.15.160:0/3336396117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a5717d51a0 0x55a5717e4330 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f8e080058c0 tx=0x7f8e08004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1a566affd5e8249 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:42:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.335+0000 7f8df6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-03-07T07:42:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.335+0000 7f8df6ffd640 1 -- 172.21.15.160:0/3336396117 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0802f520 con 0x55a5717d51a0 2024-03-07T07:42:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.335+0000 7f8df6ffd640 1 -- 172.21.15.160:0/3336396117 <== mon.0 v2:172.21.15.160:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8e0802f6c0 con 0x55a5717d51a0 2024-03-07T07:42:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.335+0000 7f8e20c36740 1 -- 172.21.15.160:0/3336396117 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8e0c0035f0 con 0x55a5717d51a0 2024-03-07T07:42:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.335+0000 7f8df6ffd640 1 -- 172.21.15.160:0/3336396117 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e0802f8c0 con 0x55a5717d51a0 2024-03-07T07:42:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.335+0000 7f8e20c36740 10 client.15345 fetch_fsmap learned FSMap version 22 2024-03-07T07:42:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.335+0000 7f8e20c36740 1 -- 172.21.15.160:0/3336396117 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55a571846c80 con 0x55a5717d51a0 2024-03-07T07:42:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.336+0000 7f8df6ffd640 1 -- 172.21.15.160:0/3336396117 <== mon.0 v2:172.21.15.160:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e0c0035f0 con 0x55a5717d51a0 2024-03-07T07:42:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.336+0000 7f8df6ffd640 1 -- 172.21.15.160:0/3336396117 <== mon.0 v2:172.21.15.160:3300/0 5 ==== fsmap(e 22) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f8e08025040 con 0x55a5717d51a0 2024-03-07T07:42:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.336+0000 7f8e20c36740 10 client.15345 fetch_fsmap finished waiting for FSMap version 22 2024-03-07T07:42:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.336+0000 7f8e20c36740 10 client.15345 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-07T07:42:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.336+0000 7f8e20c36740 20 client.15345 populate_metadata read hostname 'smithi160' 2024-03-07T07:42:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.336+0000 7f8e20c36740 1 --2- 172.21.15.160:0/3336396117 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x55a571847610 0x55a571849a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.336+0000 7f8e20c36740 4 client.15345 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-07T07:42:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.336+0000 7f8e20c36740 1 -- 172.21.15.160:0/3336396117 --> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a571741e50 con 0x55a571847610 2024-03-07T07:42:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.336+0000 7f8e16ffd640 1 --2- 172.21.15.160:0/3336396117 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x55a571847610 0x55a571849a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.337+0000 7f8e16ffd640 1 --2- 172.21.15.160:0/3336396117 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x55a571847610 0x55a571849a30 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d200561d602d64ed server_cookie=9b55b001bfd999c in_seq=0 out_seq=0 2024-03-07T07:42:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.337+0000 7f8df6ffd640 10 client.15345 ms_handle_connect on v2:172.21.15.178:6838/2947354024 2024-03-07T07:42:04.342 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.340+0000 7f8df6ffd640 1 -- 172.21.15.160:0/3336396117 <== mds.0 v2:172.21.15.178:6838/2947354024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a571741e50 con 0x55a571847610 2024-03-07T07:42:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.340+0000 7f8df6ffd640 10 client.15345 handle_command_reply: tid=0 2024-03-07T07:42:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.340+0000 7f8e20c36740 1 client.15345 shutdown 2024-03-07T07:42:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.340+0000 7f8e20c36740 1 -- 172.21.15.160:0/3336396117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a5717d51a0 msgr2=0x55a5717e4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:42:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.340+0000 7f8e20c36740 1 --2- 172.21.15.160:0/3336396117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a5717d51a0 0x55a5717e4330 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f8e080058c0 tx=0x7f8e08004730 comp rx=0 tx=0).stop 2024-03-07T07:42:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.340+0000 7f8e20c36740 1 -- 172.21.15.160:0/3336396117 shutdown_connections 2024-03-07T07:42:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.340+0000 7f8e20c36740 1 --2- 172.21.15.160:0/3336396117 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x55a571847610 0x55a571849a30 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.340+0000 7f8e20c36740 1 --2- 172.21.15.160:0/3336396117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a5717dad00 0x55a5717e5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.340+0000 7f8df6ffd640 0 client.15345 ms_handle_reset on v2:172.21.15.178:6838/2947354024 2024-03-07T07:42:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.340+0000 7f8e20c36740 1 --2- 172.21.15.160:0/3336396117 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5717d5b90 0x55a5717e4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.340+0000 7f8e20c36740 1 --2- 172.21.15.160:0/3336396117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a5717d51a0 0x55a5717e4330 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.340+0000 7f8e20c36740 1 -- 172.21.15.160:0/3336396117 >> 172.21.15.160:0/3336396117 conn(0x55a57179d580 msgr2=0x55a5717d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:42:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.340+0000 7f8e20c36740 1 -- 172.21.15.160:0/3336396117 shutdown_connections 2024-03-07T07:42:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.340+0000 7f8e20c36740 1 -- 172.21.15.160:0/3336396117 wait complete. 2024-03-07T07:42:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.340+0000 7f8e20c36740 20 client.15345 trim_cache size 0 max 16384 2024-03-07T07:42:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stdout:{ 2024-03-07T07:42:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stdout: "message": "", 2024-03-07T07:42:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stdout: "return_code": 0 2024-03-07T07:42:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stdout:} 2024-03-07T07:42:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.341+0000 7f8e1de34640 1 -- 172.21.15.160:0/3731831197 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f8e000a1ec0 msgr2=0x7f8e000a4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:42:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.341+0000 7f8e1de34640 1 --2- 172.21.15.160:0/3731831197 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f8e000a1ec0 0x7f8e000a4380 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8e080095f0 tx=0x7f8e08012040 comp rx=0 tx=0).stop 2024-03-07T07:42:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.341+0000 7f8e1de34640 1 -- 172.21.15.160:0/3731831197 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8e18102130 msgr2=0x7f8e1819ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:42:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.341+0000 7f8e1de34640 1 --2- 172.21.15.160:0/3731831197 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8e18102130 0x7f8e1819ee40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8e1002c630 tx=0x7f8e10032f20 comp rx=0 tx=0).stop 2024-03-07T07:42:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.341+0000 7f8e1de34640 1 -- 172.21.15.160:0/3731831197 shutdown_connections 2024-03-07T07:42:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.341+0000 7f8e1de34640 1 --2- 172.21.15.160:0/3731831197 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f8e000a1ec0 0x7f8e000a4380 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.341+0000 7f8e1de34640 1 --2- 172.21.15.160:0/3731831197 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8e18102130 0x7f8e1819ee40 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.341+0000 7f8e1de34640 1 --2- 172.21.15.160:0/3731831197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e18101770 0x7f8e18197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.341+0000 7f8e1de34640 1 --2- 172.21.15.160:0/3731831197 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8e18100580 0x7f8e18197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.341+0000 7f8e1de34640 1 -- 172.21.15.160:0/3731831197 >> 172.21.15.160:0/3731831197 conn(0x7f8e180fbaf0 msgr2=0x7f8e18104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:42:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.341+0000 7f8e1de34640 1 -- 172.21.15.160:0/3731831197 shutdown_connections 2024-03-07T07:42:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:04.341+0000 7f8e1de34640 1 -- 172.21.15.160:0/3731831197 wait complete. 2024-03-07T07:42:05.679 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:42:05 smithi160 ceph-mon[31963]: pgmap v375: 97 pgs: 97 active+clean; 666 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 7 op/s 2024-03-07T07:42:05.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:42:05 smithi178 ceph-mon[36650]: pgmap v375: 97 pgs: 97 active+clean; 666 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 7 op/s 2024-03-07T07:42:05.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:42:05 smithi177 ceph-mon[36578]: pgmap v375: 97 pgs: 97 active+clean; 666 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 7 op/s 2024-03-07T07:42:07.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:42:07 smithi160 ceph-mon[31963]: pgmap v376: 97 pgs: 97 active+clean; 667 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 853 B/s wr, 8 op/s 2024-03-07T07:42:07.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:42:07 smithi178 ceph-mon[36650]: pgmap v376: 97 pgs: 97 active+clean; 667 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 853 B/s wr, 8 op/s 2024-03-07T07:42:07.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:42:07 smithi177 ceph-mon[36578]: pgmap v376: 97 pgs: 97 active+clean; 667 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 853 B/s wr, 8 op/s 2024-03-07T07:42:09.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:42:09 smithi160 ceph-mon[31963]: pgmap v377: 97 pgs: 97 active+clean; 667 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-07T07:42:09.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:42:09 smithi178 ceph-mon[36650]: pgmap v377: 97 pgs: 97 active+clean; 667 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-07T07:42:09.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:42:09 smithi177 ceph-mon[36578]: pgmap v377: 97 pgs: 97 active+clean; 667 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-07T07:42:11.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:42:11 smithi160 ceph-mon[31963]: pgmap v378: 97 pgs: 97 active+clean; 667 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-07T07:42:11.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:42:11 smithi178 ceph-mon[36650]: pgmap v378: 97 pgs: 97 active+clean; 667 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-07T07:42:11.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:42:11 smithi177 ceph-mon[36578]: pgmap v378: 97 pgs: 97 active+clean; 667 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-07T07:42:13.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:42:13 smithi160 ceph-mon[31963]: pgmap v379: 97 pgs: 97 active+clean; 667 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-07T07:42:13.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:42:13 smithi178 ceph-mon[36650]: pgmap v379: 97 pgs: 97 active+clean; 667 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-07T07:42:13.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:42:13 smithi177 ceph-mon[36578]: pgmap v379: 97 pgs: 97 active+clean; 667 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-07T07:42:15.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:42:15 smithi160 ceph-mon[31963]: pgmap v380: 97 pgs: 97 active+clean; 667 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-07T07:42:15.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:42:15 smithi178 ceph-mon[36650]: pgmap v380: 97 pgs: 97 active+clean; 667 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-07T07:42:15.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:42:15 smithi177 ceph-mon[36578]: pgmap v380: 97 pgs: 97 active+clean; 667 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-07T07:42:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.473+0000 7efd531b1640 1 -- 172.21.15.160:0/297279665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd4c1021c0 msgr2=0x7efd4c1045b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:42:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.473+0000 7efd531b1640 1 --2- 172.21.15.160:0/297279665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd4c1021c0 0x7efd4c1045b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efd40009860 tx=0x7efd4002bb70 comp rx=0 tx=0).stop 2024-03-07T07:42:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.473+0000 7efd531b1640 1 -- 172.21.15.160:0/297279665 shutdown_connections 2024-03-07T07:42:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.473+0000 7efd531b1640 1 --2- 172.21.15.160:0/297279665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd4c107420 0x7efd4c109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.473+0000 7efd531b1640 1 --2- 172.21.15.160:0/297279665 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd4c104af0 0x7efd4c106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.473+0000 7efd531b1640 1 --2- 172.21.15.160:0/297279665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd4c1021c0 0x7efd4c1045b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.473+0000 7efd531b1640 1 -- 172.21.15.160:0/297279665 >> 172.21.15.160:0/297279665 conn(0x7efd4c0fbaf0 msgr2=0x7efd4c0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:42:17.476 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.473+0000 7efd531b1640 1 -- 172.21.15.160:0/297279665 shutdown_connections 2024-03-07T07:42:17.476 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.473+0000 7efd531b1640 1 -- 172.21.15.160:0/297279665 wait complete. 2024-03-07T07:42:17.476 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.473+0000 7efd531b1640 1 Processor -- start 2024-03-07T07:42:17.476 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.474+0000 7efd531b1640 1 -- start start 2024-03-07T07:42:17.476 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.474+0000 7efd531b1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd4c1021c0 0x7efd4c197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:17.476 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.474+0000 7efd531b1640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd4c104af0 0x7efd4c197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:17.476 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.474+0000 7efd531b1640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd4c107420 0x7efd4c19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:17.476 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.474+0000 7efd5174e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd4c107420 0x7efd4c19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:17.476 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.474+0000 7efd50f4d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd4c1021c0 0x7efd4c197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:17.476 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.474+0000 7efd5174e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd4c107420 0x7efd4c19eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:53220/0 (socket says 172.21.15.160:53220) 2024-03-07T07:42:17.476 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.474+0000 7efd50f4d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd4c1021c0 0x7efd4c197940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.160:40708/0 (socket says 172.21.15.160:40708) 2024-03-07T07:42:17.476 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.474+0000 7efd5174e640 1 -- 172.21.15.160:0/3838714171 learned_addr learned my addr 172.21.15.160:0/3838714171 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:42:17.476 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.474+0000 7efd531b1640 1 -- 172.21.15.160:0/3838714171 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4c10d330 con 0x7efd4c107420 2024-03-07T07:42:17.477 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.474+0000 7efd531b1640 1 -- 172.21.15.160:0/3838714171 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4c10d1b0 con 0x7efd4c104af0 2024-03-07T07:42:17.477 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.474+0000 7efd531b1640 1 -- 172.21.15.160:0/3838714171 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4c10d4b0 con 0x7efd4c1021c0 2024-03-07T07:42:17.477 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.474+0000 7efd4bfff640 1 --2- 172.21.15.160:0/3838714171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd4c104af0 0x7efd4c197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:17.477 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.474+0000 7efd5174e640 1 -- 172.21.15.160:0/3838714171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd4c1021c0 msgr2=0x7efd4c197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:42:17.477 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.474+0000 7efd5174e640 1 --2- 172.21.15.160:0/3838714171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd4c1021c0 0x7efd4c197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:17.477 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.474+0000 7efd5174e640 1 -- 172.21.15.160:0/3838714171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd4c104af0 msgr2=0x7efd4c197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:42:17.477 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.474+0000 7efd5174e640 1 --2- 172.21.15.160:0/3838714171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd4c104af0 0x7efd4c197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:17.477 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.474+0000 7efd5174e640 1 -- 172.21.15.160:0/3838714171 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd4c19f560 con 0x7efd4c107420 2024-03-07T07:42:17.477 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.474+0000 7efd50f4d640 1 --2- 172.21.15.160:0/3838714171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd4c1021c0 0x7efd4c197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:42:17.477 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.474+0000 7efd4bfff640 1 --2- 172.21.15.160:0/3838714171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd4c104af0 0x7efd4c197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:42:17.477 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.475+0000 7efd5174e640 1 --2- 172.21.15.160:0/3838714171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd4c107420 0x7efd4c19eef0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7efd4400eb10 tx=0x7efd4400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:42:17.477 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.475+0000 7efd49ffb640 1 -- 172.21.15.160:0/3838714171 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd44004b20 con 0x7efd4c107420 2024-03-07T07:42:17.477 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.475+0000 7efd531b1640 1 -- 172.21.15.160:0/3838714171 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd4c19f850 con 0x7efd4c107420 2024-03-07T07:42:17.477 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.475+0000 7efd49ffb640 1 -- 172.21.15.160:0/3838714171 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efd44004cc0 con 0x7efd4c107420 2024-03-07T07:42:17.478 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.475+0000 7efd531b1640 1 -- 172.21.15.160:0/3838714171 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd4c19fd40 con 0x7efd4c107420 2024-03-07T07:42:17.478 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.476+0000 7efd49ffb640 1 -- 172.21.15.160:0/3838714171 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd44010700 con 0x7efd4c107420 2024-03-07T07:42:17.479 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.478+0000 7efd49ffb640 1 -- 172.21.15.160:0/3838714171 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) v1 ==== 136805+0+0 (secure 0 0 0) 0x7efd440108a0 con 0x7efd4c107420 2024-03-07T07:42:17.479 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.479+0000 7efd55f98740 1 Processor -- start 2024-03-07T07:42:17.479 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.479+0000 7efd55f98740 1 -- start start 2024-03-07T07:42:17.479 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.479+0000 7efd55f98740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5570348e35b0 0x5570348e39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:17.479 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.479+0000 7efd55f98740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570348e3fa0 0x5570348e8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:17.479 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.479+0000 7efd55f98740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570348e9110 0x5570348eb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:17.479 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.479+0000 7efd55f98740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570348445c0 con 0x5570348e3fa0 2024-03-07T07:42:17.479 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.479+0000 7efd55f98740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570347ed900 con 0x5570348e35b0 2024-03-07T07:42:17.479 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.479+0000 7efd55f98740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x557034840660 con 0x5570348e9110 2024-03-07T07:42:17.480 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.479+0000 7efd50f4d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570348e3fa0 0x5570348e8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:17.480 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.479+0000 7efd4bfff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5570348e35b0 0x5570348e39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:17.480 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.479+0000 7efd50f4d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570348e3fa0 0x5570348e8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:53234/0 (socket says 172.21.15.160:53234) 2024-03-07T07:42:17.481 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.479+0000 7efd50f4d640 1 -- 172.21.15.160:0/832274475 learned_addr learned my addr 172.21.15.160:0/832274475 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:42:17.481 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.479+0000 7efd4bfff640 1 --2- 172.21.15.160:0/832274475 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570348e9110 0x5570348eb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:17.481 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.479+0000 7efd4bfff640 1 -- 172.21.15.160:0/832274475 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570348e9110 msgr2=0x5570348eb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:42:17.481 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.479+0000 7efd4bfff640 1 --2- 172.21.15.160:0/832274475 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570348e9110 0x5570348eb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:17.481 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.479+0000 7efd4bfff640 1 -- 172.21.15.160:0/832274475 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570348e3fa0 msgr2=0x5570348e8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:42:17.481 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.479+0000 7efd4bfff640 1 --2- 172.21.15.160:0/832274475 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570348e3fa0 0x5570348e8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:17.481 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.479+0000 7efd49ffb640 1 --2- 172.21.15.160:0/3838714171 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7efd3c0a1ec0 0x7efd3c0a4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:17.481 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.479+0000 7efd4bfff640 1 -- 172.21.15.160:0/832274475 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557034559210 con 0x5570348e35b0 2024-03-07T07:42:17.481 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.480+0000 7efd4bfff640 1 --2- 172.21.15.160:0/832274475 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5570348e35b0 0x5570348e39d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efd4c198e50 tx=0x7efd38009610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9b3f3ad63b9980a server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:42:17.481 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.480+0000 7efd49ffb640 1 -- 172.21.15.160:0/3838714171 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7efd440ce570 con 0x7efd4c107420 2024-03-07T07:42:17.481 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.480+0000 7efd36ffd640 1 -- 172.21.15.160:0/832274475 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd38010e30 con 0x5570348e35b0 2024-03-07T07:42:17.481 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.480+0000 7efd36ffd640 1 -- 172.21.15.160:0/832274475 <== mon.2 v2:172.21.15.177:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efd38005020 con 0x5570348e35b0 2024-03-07T07:42:17.481 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.480+0000 7efd5174e640 1 --2- 172.21.15.160:0/3838714171 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7efd3c0a1ec0 0x7efd3c0a4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:17.481 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.480+0000 7efd36ffd640 1 -- 172.21.15.160:0/832274475 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd38004a00 con 0x5570348e35b0 2024-03-07T07:42:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.481+0000 7efd5174e640 1 --2- 172.21.15.160:0/3838714171 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7efd3c0a1ec0 0x7efd3c0a4380 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7efd44009ae0 tx=0x7efd44009a70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:42:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.481+0000 7efd55f98740 1 -- 172.21.15.160:0/832274475 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5570348e35b0 msgr2=0x5570348e39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:42:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.481+0000 7efd55f98740 1 --2- 172.21.15.160:0/832274475 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5570348e35b0 0x5570348e39d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efd4c198e50 tx=0x7efd38009610 comp rx=0 tx=0).stop 2024-03-07T07:42:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.481+0000 7efd55f98740 1 -- 172.21.15.160:0/832274475 shutdown_connections 2024-03-07T07:42:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.481+0000 7efd55f98740 1 --2- 172.21.15.160:0/832274475 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570348e9110 0x5570348eb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.481+0000 7efd55f98740 1 --2- 172.21.15.160:0/832274475 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570348e3fa0 0x5570348e8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.481+0000 7efd55f98740 1 --2- 172.21.15.160:0/832274475 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5570348e35b0 0x5570348e39d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.481+0000 7efd55f98740 1 -- 172.21.15.160:0/832274475 >> 172.21.15.160:0/832274475 conn(0x5570348a7580 msgr2=0x5570348def20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:42:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.481+0000 7efd55f98740 1 -- 172.21.15.160:0/832274475 shutdown_connections 2024-03-07T07:42:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.481+0000 7efd55f98740 1 -- 172.21.15.160:0/832274475 wait complete. 2024-03-07T07:42:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.482+0000 7efd55f98740 1 -- start start 2024-03-07T07:42:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.482+0000 7efd55f98740 1 Processor -- start 2024-03-07T07:42:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.482+0000 7efd55f98740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5570348e35b0 0x5570348e66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.482+0000 7efd55f98740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570348e3fa0 0x5570348e6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.482+0000 7efd55f98740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570348e9110 0x5570348e7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.483+0000 7efd4bfff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5570348e35b0 0x5570348e66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.483+0000 7efd4bfff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5570348e35b0 0x5570348e66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.160:50474/0 (socket says 172.21.15.160:50474) 2024-03-07T07:42:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.483+0000 7efd4bfff640 1 -- 172.21.15.160:0/3372110166 learned_addr learned my addr 172.21.15.160:0/3372110166 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:42:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.483+0000 7efd50f4d640 1 --2- 172.21.15.160:0/3372110166 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570348e3fa0 0x5570348e6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.483+0000 7efd55f98740 1 -- 172.21.15.160:0/3372110166 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570348da820 con 0x5570348e3fa0 2024-03-07T07:42:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.483+0000 7efd55f98740 1 -- 172.21.15.160:0/3372110166 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x557034800d60 con 0x5570348e35b0 2024-03-07T07:42:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.483+0000 7efd55f98740 1 -- 172.21.15.160:0/3372110166 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570348da9a0 con 0x5570348e9110 2024-03-07T07:42:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.483+0000 7efd4bfff640 1 --2- 172.21.15.160:0/3372110166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570348e9110 0x5570348e7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.483+0000 7efd4bfff640 1 -- 172.21.15.160:0/3372110166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570348e9110 msgr2=0x5570348e7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:42:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.483+0000 7efd4bfff640 1 --2- 172.21.15.160:0/3372110166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570348e9110 0x5570348e7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.483+0000 7efd4bfff640 1 -- 172.21.15.160:0/3372110166 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570348e3fa0 msgr2=0x5570348e6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:42:17.489 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.483+0000 7efd4bfff640 1 --2- 172.21.15.160:0/3372110166 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570348e3fa0 0x5570348e6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:17.489 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.483+0000 7efd4bfff640 1 -- 172.21.15.160:0/3372110166 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557034625620 con 0x5570348e35b0 2024-03-07T07:42:17.489 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.483+0000 7efd50f4d640 1 --2- 172.21.15.160:0/3372110166 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570348e3fa0 0x5570348e6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:42:17.489 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.483+0000 7efd55f98740 10 client.0 _ll_register_callbacks cb 0x557034558a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-07T07:42:17.489 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.483+0000 7efd4bfff640 1 --2- 172.21.15.160:0/3372110166 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5570348e35b0 0x5570348e66d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efd38009480 tx=0x7efd38009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a758ef30e226a81 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:42:17.489 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.483+0000 7efd2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-03-07T07:42:17.489 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.483+0000 7efd55f98740 1 -- 172.21.15.160:0/3372110166 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efd38005020 con 0x5570348e35b0 2024-03-07T07:42:17.489 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.483+0000 7efd2bfff640 1 -- 172.21.15.160:0/3372110166 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd380048c0 con 0x5570348e35b0 2024-03-07T07:42:17.489 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.484+0000 7efd2bfff640 1 -- 172.21.15.160:0/3372110166 <== mon.2 v2:172.21.15.177:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7efd38004a60 con 0x5570348e35b0 2024-03-07T07:42:17.489 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.484+0000 7efd2bfff640 1 -- 172.21.15.160:0/3372110166 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd38004c60 con 0x5570348e35b0 2024-03-07T07:42:17.489 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.484+0000 7efd55f98740 10 client.25196 fetch_fsmap learned FSMap version 22 2024-03-07T07:42:17.489 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.484+0000 7efd55f98740 1 -- 172.21.15.160:0/3372110166 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({fsmap=22}) v3 -- 0x5570348f3a20 con 0x5570348e35b0 2024-03-07T07:42:17.489 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.484+0000 7efd2bfff640 1 -- 172.21.15.160:0/3372110166 <== mon.2 v2:172.21.15.177:3300/0 4 ==== fsmap(e 22) v1 ==== 7918+0+0 (secure 0 0 0) 0x7efd380035f0 con 0x5570348e35b0 2024-03-07T07:42:17.489 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.484+0000 7efd55f98740 10 client.25196 fetch_fsmap finished waiting for FSMap version 22 2024-03-07T07:42:17.490 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.484+0000 7efd55f98740 10 client.25196 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-07T07:42:17.490 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.484+0000 7efd55f98740 20 client.25196 populate_metadata read hostname 'smithi160' 2024-03-07T07:42:17.490 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.484+0000 7efd55f98740 1 --2- 172.21.15.160:0/3372110166 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x5570348f4de0 0x5570348f7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:17.490 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.484+0000 7efd55f98740 4 client.25196 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-07T07:42:17.490 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.484+0000 7efd55f98740 1 -- 172.21.15.160:0/3372110166 --> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7efd38005020 con 0x5570348f4de0 2024-03-07T07:42:17.490 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.484+0000 7efd50f4d640 1 --2- 172.21.15.160:0/3372110166 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x5570348f4de0 0x5570348f7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:17.490 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.485+0000 7efd2bfff640 10 client.25196 ms_handle_connect on v2:172.21.15.178:6838/2947354024 2024-03-07T07:42:17.490 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.485+0000 7efd50f4d640 1 --2- 172.21.15.160:0/3372110166 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x5570348f4de0 0x5570348f7270 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53853185ccd93f01 server_cookie=588fec0e3f670922 in_seq=0 out_seq=0 2024-03-07T07:42:17.490 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.485+0000 7efd2bfff640 1 -- 172.21.15.160:0/3372110166 <== mds.0 v2:172.21.15.178:6838/2947354024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7efd38005020 con 0x5570348f4de0 2024-03-07T07:42:17.490 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.485+0000 7efd2bfff640 10 client.25196 handle_command_reply: tid=0 2024-03-07T07:42:17.490 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.485+0000 7efd55f98740 1 client.25196 shutdown 2024-03-07T07:42:17.490 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.486+0000 7efd55f98740 1 -- 172.21.15.160:0/3372110166 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5570348e35b0 msgr2=0x5570348e66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:42:17.490 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.486+0000 7efd55f98740 1 --2- 172.21.15.160:0/3372110166 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5570348e35b0 0x5570348e66d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efd38009480 tx=0x7efd38009260 comp rx=0 tx=0).stop 2024-03-07T07:42:17.490 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.486+0000 7efd55f98740 1 -- 172.21.15.160:0/3372110166 shutdown_connections 2024-03-07T07:42:17.490 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.486+0000 7efd55f98740 1 --2- 172.21.15.160:0/3372110166 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x5570348f4de0 0x5570348f7270 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.486+0000 7efd55f98740 1 --2- 172.21.15.160:0/3372110166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570348e9110 0x5570348e7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.486+0000 7efd2bfff640 0 client.25196 ms_handle_reset on v2:172.21.15.178:6838/2947354024 2024-03-07T07:42:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.486+0000 7efd55f98740 1 --2- 172.21.15.160:0/3372110166 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570348e3fa0 0x5570348e6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.486+0000 7efd55f98740 1 --2- 172.21.15.160:0/3372110166 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5570348e35b0 0x5570348e66d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.486+0000 7efd55f98740 1 -- 172.21.15.160:0/3372110166 >> 172.21.15.160:0/3372110166 conn(0x5570348a7580 msgr2=0x5570348d9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:42:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.486+0000 7efd55f98740 1 -- 172.21.15.160:0/3372110166 shutdown_connections 2024-03-07T07:42:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.486+0000 7efd55f98740 1 -- 172.21.15.160:0/3372110166 wait complete. 2024-03-07T07:42:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.486+0000 7efd55f98740 20 client.25196 trim_cache size 0 max 16384 2024-03-07T07:42:17.515 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.513+0000 7efd55f98740 1 Processor -- start 2024-03-07T07:42:17.515 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.513+0000 7efd55f98740 1 -- start start 2024-03-07T07:42:17.515 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.513+0000 7efd55f98740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570348e35b0 0x5570348e29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:17.515 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.513+0000 7efd55f98740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5570348e3fa0 0x5570348e69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:17.515 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.513+0000 7efd55f98740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570348e9110 0x5570348d9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:17.515 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.513+0000 7efd55f98740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd300108a0 con 0x5570348e35b0 2024-03-07T07:42:17.515 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.513+0000 7efd55f98740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3000e330 con 0x5570348e3fa0 2024-03-07T07:42:17.515 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.513+0000 7efd55f98740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3000c3b0 con 0x5570348e9110 2024-03-07T07:42:17.515 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.513+0000 7efd4bfff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570348e35b0 0x5570348e29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:17.516 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.513+0000 7efd4bfff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570348e35b0 0x5570348e29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:53254/0 (socket says 172.21.15.160:53254) 2024-03-07T07:42:17.516 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.513+0000 7efd4bfff640 1 -- 172.21.15.160:0/409656164 learned_addr learned my addr 172.21.15.160:0/409656164 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:42:17.516 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.513+0000 7efd50f4d640 1 --2- 172.21.15.160:0/409656164 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5570348e3fa0 0x5570348e69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:17.516 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.513+0000 7efd4bfff640 1 --2- 172.21.15.160:0/409656164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570348e9110 0x5570348d9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:17.516 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.513+0000 7efd4bfff640 1 -- 172.21.15.160:0/409656164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570348e9110 msgr2=0x5570348d9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:42:17.516 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.513+0000 7efd4bfff640 1 --2- 172.21.15.160:0/409656164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570348e9110 0x5570348d9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:17.516 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.513+0000 7efd4bfff640 1 -- 172.21.15.160:0/409656164 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5570348e3fa0 msgr2=0x5570348e69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:42:17.516 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.513+0000 7efd4bfff640 1 --2- 172.21.15.160:0/409656164 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5570348e3fa0 0x5570348e69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:17.516 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.513+0000 7efd4bfff640 1 -- 172.21.15.160:0/409656164 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5570348e31f0 con 0x5570348e35b0 2024-03-07T07:42:17.516 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.514+0000 7efd4bfff640 1 --2- 172.21.15.160:0/409656164 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570348e35b0 0x5570348e29f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7efd38001eb0 tx=0x7efd3802c050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e184f52a39ec784 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:42:17.516 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.514+0000 7efd367fc640 1 -- 172.21.15.160:0/409656164 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd38004470 con 0x5570348e35b0 2024-03-07T07:42:17.516 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.514+0000 7efd367fc640 1 -- 172.21.15.160:0/409656164 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efd380035f0 con 0x5570348e35b0 2024-03-07T07:42:17.516 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.514+0000 7efd367fc640 1 -- 172.21.15.160:0/409656164 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd38029440 con 0x5570348e35b0 2024-03-07T07:42:17.517 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.515+0000 7efd55f98740 1 -- 172.21.15.160:0/409656164 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570348e35b0 msgr2=0x5570348e29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:42:17.517 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.515+0000 7efd55f98740 1 --2- 172.21.15.160:0/409656164 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570348e35b0 0x5570348e29f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7efd38001eb0 tx=0x7efd3802c050 comp rx=0 tx=0).stop 2024-03-07T07:42:17.517 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.515+0000 7efd55f98740 1 -- 172.21.15.160:0/409656164 shutdown_connections 2024-03-07T07:42:17.517 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.515+0000 7efd55f98740 1 --2- 172.21.15.160:0/409656164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570348e9110 0x5570348d9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:17.517 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.515+0000 7efd55f98740 1 --2- 172.21.15.160:0/409656164 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5570348e3fa0 0x5570348e69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:17.517 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.515+0000 7efd55f98740 1 --2- 172.21.15.160:0/409656164 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570348e35b0 0x5570348e29f0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:17.517 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.515+0000 7efd55f98740 1 -- 172.21.15.160:0/409656164 >> 172.21.15.160:0/409656164 conn(0x5570348a7580 msgr2=0x5570348e62e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:42:17.517 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.515+0000 7efd55f98740 1 -- 172.21.15.160:0/409656164 shutdown_connections 2024-03-07T07:42:17.517 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.515+0000 7efd55f98740 1 -- 172.21.15.160:0/409656164 wait complete. 2024-03-07T07:42:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.515+0000 7efd55f98740 1 -- start start 2024-03-07T07:42:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.515+0000 7efd55f98740 1 Processor -- start 2024-03-07T07:42:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.516+0000 7efd55f98740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570348e3fa0 0x5570348ef510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.516+0000 7efd55f98740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570348e9110 0x5570348efa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.516+0000 7efd55f98740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5570348f4de0 0x5570348f09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.516+0000 7efd55f98740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3000c9b0 con 0x5570348e3fa0 2024-03-07T07:42:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.516+0000 7efd55f98740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3000dcd0 con 0x5570348f4de0 2024-03-07T07:42:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.516+0000 7efd55f98740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3000d010 con 0x5570348e9110 2024-03-07T07:42:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.516+0000 7efd55f98740 10 client.0 _ll_register_callbacks cb 0x7efd38005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-07T07:42:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.516+0000 7efd4bfff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570348e3fa0 0x5570348ef510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.516+0000 7efd4bfff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570348e3fa0 0x5570348ef510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:53256/0 (socket says 172.21.15.160:53256) 2024-03-07T07:42:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.516+0000 7efd4bfff640 1 -- 172.21.15.160:0/1071946783 learned_addr learned my addr 172.21.15.160:0/1071946783 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:42:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.516+0000 7efd50f4d640 1 --2- 172.21.15.160:0/1071946783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570348e9110 0x5570348efa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.516+0000 7efd4bfff640 1 --2- 172.21.15.160:0/1071946783 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5570348f4de0 0x5570348f09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.516+0000 7efd50f4d640 1 -- 172.21.15.160:0/1071946783 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5570348f4de0 msgr2=0x5570348f09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:42:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.516+0000 7efd50f4d640 1 --2- 172.21.15.160:0/1071946783 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5570348f4de0 0x5570348f09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.516+0000 7efd50f4d640 1 -- 172.21.15.160:0/1071946783 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570348e3fa0 msgr2=0x5570348ef510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:42:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.516+0000 7efd50f4d640 1 --2- 172.21.15.160:0/1071946783 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570348e3fa0 0x5570348ef510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.516+0000 7efd50f4d640 1 -- 172.21.15.160:0/1071946783 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5570348f0fe0 con 0x5570348e9110 2024-03-07T07:42:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.516+0000 7efd4bfff640 1 --2- 172.21.15.160:0/1071946783 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5570348f4de0 0x5570348f09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T07:42:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.516+0000 7efd50f4d640 1 --2- 172.21.15.160:0/1071946783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570348e9110 0x5570348efa50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x5570348e4a10 tx=0x7efd40034ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cc81990f6ed588f server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:42:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.516+0000 7efd2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-03-07T07:42:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.517+0000 7efd2bfff640 1 -- 172.21.15.160:0/1071946783 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd40039040 con 0x5570348e9110 2024-03-07T07:42:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.517+0000 7efd2bfff640 1 -- 172.21.15.160:0/1071946783 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7efd40004040 con 0x5570348e9110 2024-03-07T07:42:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.517+0000 7efd55f98740 1 -- 172.21.15.160:0/1071946783 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efd380035f0 con 0x5570348e9110 2024-03-07T07:42:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.517+0000 7efd2bfff640 1 -- 172.21.15.160:0/1071946783 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd4005b050 con 0x5570348e9110 2024-03-07T07:42:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.517+0000 7efd55f98740 10 client.25237 fetch_fsmap learned FSMap version 22 2024-03-07T07:42:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.517+0000 7efd55f98740 1 -- 172.21.15.160:0/1071946783 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x557034950bf0 con 0x5570348e9110 2024-03-07T07:42:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.517+0000 7efd2bfff640 1 -- 172.21.15.160:0/1071946783 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efd380035f0 con 0x5570348e9110 2024-03-07T07:42:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.517+0000 7efd2bfff640 1 -- 172.21.15.160:0/1071946783 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 22) v1 ==== 7918+0+0 (secure 0 0 0) 0x7efd40032640 con 0x5570348e9110 2024-03-07T07:42:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.517+0000 7efd55f98740 10 client.25237 fetch_fsmap finished waiting for FSMap version 22 2024-03-07T07:42:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.517+0000 7efd55f98740 10 client.25237 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-07T07:42:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.517+0000 7efd55f98740 20 client.25237 populate_metadata read hostname 'smithi160' 2024-03-07T07:42:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.517+0000 7efd55f98740 1 --2- 172.21.15.160:0/1071946783 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x557034951620 0x557034953a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.517+0000 7efd55f98740 4 client.25237 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-07T07:42:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.517+0000 7efd55f98740 1 -- 172.21.15.160:0/1071946783 --> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557034851b00 con 0x557034951620 2024-03-07T07:42:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.517+0000 7efd4bfff640 1 --2- 172.21.15.160:0/1071946783 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x557034951620 0x557034953a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.518+0000 7efd2bfff640 10 client.25237 ms_handle_connect on v2:172.21.15.178:6838/2947354024 2024-03-07T07:42:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.518+0000 7efd4bfff640 1 --2- 172.21.15.160:0/1071946783 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x557034951620 0x557034953a40 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b170226dac25ac7 server_cookie=b750b65ecaaa1db0 in_seq=0 out_seq=0 2024-03-07T07:42:17.525 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.521+0000 7efd2bfff640 1 -- 172.21.15.160:0/1071946783 <== mds.0 v2:172.21.15.178:6838/2947354024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557034851b00 con 0x557034951620 2024-03-07T07:42:17.525 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.521+0000 7efd2bfff640 10 client.25237 handle_command_reply: tid=0 2024-03-07T07:42:17.525 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.521+0000 7efd55f98740 1 client.25237 shutdown 2024-03-07T07:42:17.525 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.521+0000 7efd55f98740 1 -- 172.21.15.160:0/1071946783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570348e9110 msgr2=0x5570348efa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:42:17.525 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.521+0000 7efd55f98740 1 --2- 172.21.15.160:0/1071946783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570348e9110 0x5570348efa50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x5570348e4a10 tx=0x7efd40034ad0 comp rx=0 tx=0).stop 2024-03-07T07:42:17.525 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.521+0000 7efd55f98740 1 -- 172.21.15.160:0/1071946783 shutdown_connections 2024-03-07T07:42:17.525 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.521+0000 7efd55f98740 1 --2- 172.21.15.160:0/1071946783 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x557034951620 0x557034953a40 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:17.525 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.521+0000 7efd55f98740 1 --2- 172.21.15.160:0/1071946783 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5570348f4de0 0x5570348f09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:17.525 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.521+0000 7efd2bfff640 0 client.25237 ms_handle_reset on v2:172.21.15.178:6838/2947354024 2024-03-07T07:42:17.525 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.521+0000 7efd55f98740 1 --2- 172.21.15.160:0/1071946783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570348e9110 0x5570348efa50 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:17.525 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.521+0000 7efd55f98740 1 --2- 172.21.15.160:0/1071946783 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570348e3fa0 0x5570348ef510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:17.527 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.521+0000 7efd55f98740 1 -- 172.21.15.160:0/1071946783 >> 172.21.15.160:0/1071946783 conn(0x5570348a7580 msgr2=0x5570348e2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:42:17.527 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.521+0000 7efd55f98740 1 -- 172.21.15.160:0/1071946783 shutdown_connections 2024-03-07T07:42:17.527 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.521+0000 7efd55f98740 1 -- 172.21.15.160:0/1071946783 wait complete. 2024-03-07T07:42:17.528 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.521+0000 7efd55f98740 20 client.25237 trim_cache size 0 max 16384 2024-03-07T07:42:17.528 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stdout:{ 2024-03-07T07:42:17.528 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stdout: "message": "", 2024-03-07T07:42:17.528 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stdout: "return_code": 0 2024-03-07T07:42:17.528 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stdout:} 2024-03-07T07:42:17.529 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.522+0000 7efd531b1640 1 -- 172.21.15.160:0/3838714171 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7efd3c0a1ec0 msgr2=0x7efd3c0a4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:42:17.529 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.522+0000 7efd531b1640 1 --2- 172.21.15.160:0/3838714171 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7efd3c0a1ec0 0x7efd3c0a4380 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7efd44009ae0 tx=0x7efd44009a70 comp rx=0 tx=0).stop 2024-03-07T07:42:17.529 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.522+0000 7efd531b1640 1 -- 172.21.15.160:0/3838714171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd4c107420 msgr2=0x7efd4c19eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:42:17.529 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.522+0000 7efd531b1640 1 --2- 172.21.15.160:0/3838714171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd4c107420 0x7efd4c19eef0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7efd4400eb10 tx=0x7efd4400c2d0 comp rx=0 tx=0).stop 2024-03-07T07:42:17.529 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.522+0000 7efd531b1640 1 -- 172.21.15.160:0/3838714171 shutdown_connections 2024-03-07T07:42:17.529 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.522+0000 7efd531b1640 1 --2- 172.21.15.160:0/3838714171 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7efd3c0a1ec0 0x7efd3c0a4380 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:17.529 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.522+0000 7efd531b1640 1 --2- 172.21.15.160:0/3838714171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd4c107420 0x7efd4c19eef0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:17.529 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.522+0000 7efd531b1640 1 --2- 172.21.15.160:0/3838714171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd4c104af0 0x7efd4c197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:17.530 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.522+0000 7efd531b1640 1 --2- 172.21.15.160:0/3838714171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd4c1021c0 0x7efd4c197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:17.530 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.522+0000 7efd531b1640 1 -- 172.21.15.160:0/3838714171 >> 172.21.15.160:0/3838714171 conn(0x7efd4c0fbaf0 msgr2=0x7efd4c102ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:42:17.530 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.522+0000 7efd531b1640 1 -- 172.21.15.160:0/3838714171 shutdown_connections 2024-03-07T07:42:17.530 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:17.522+0000 7efd531b1640 1 -- 172.21.15.160:0/3838714171 wait complete. 2024-03-07T07:42:17.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:42:17 smithi160 ceph-mon[31963]: pgmap v381: 97 pgs: 97 active+clean; 667 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-07T07:42:17.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:42:17 smithi178 ceph-mon[36650]: pgmap v381: 97 pgs: 97 active+clean; 667 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-07T07:42:17.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:42:17 smithi177 ceph-mon[36578]: pgmap v381: 97 pgs: 97 active+clean; 667 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-07T07:42:19.659 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:42:19 smithi177 ceph-mon[36578]: pgmap v382: 97 pgs: 97 active+clean; 667 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-07T07:42:19.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:42:19 smithi160 ceph-mon[31963]: pgmap v382: 97 pgs: 97 active+clean; 667 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-07T07:42:19.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:42:19 smithi178 ceph-mon[36650]: pgmap v382: 97 pgs: 97 active+clean; 667 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-07T07:42:21.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:42:21 smithi160 ceph-mon[31963]: pgmap v383: 97 pgs: 97 active+clean; 668 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-07T07:42:21.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:42:21 smithi178 ceph-mon[36650]: pgmap v383: 97 pgs: 97 active+clean; 668 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-07T07:42:21.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:42:21 smithi177 ceph-mon[36578]: pgmap v383: 97 pgs: 97 active+clean; 668 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-07T07:42:23.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:42:23 smithi160 ceph-mon[31963]: pgmap v384: 97 pgs: 97 active+clean; 668 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-07T07:42:23.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:42:23 smithi178 ceph-mon[36650]: pgmap v384: 97 pgs: 97 active+clean; 668 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-07T07:42:23.975 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:42:23 smithi177 ceph-mon[36578]: pgmap v384: 97 pgs: 97 active+clean; 668 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-07T07:42:25.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:42:25 smithi160 ceph-mon[31963]: pgmap v385: 97 pgs: 97 active+clean; 668 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-07T07:42:25.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:42:25 smithi178 ceph-mon[36650]: pgmap v385: 97 pgs: 97 active+clean; 668 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-07T07:42:25.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:42:25 smithi177 ceph-mon[36578]: pgmap v385: 97 pgs: 97 active+clean; 668 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-07T07:42:26.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:42:26 smithi178 ceph-mon[36650]: pgmap v386: 97 pgs: 97 active+clean; 668 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-07T07:42:26.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:42:26 smithi177 ceph-mon[36578]: pgmap v386: 97 pgs: 97 active+clean; 668 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-07T07:42:26.927 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:42:26 smithi160 ceph-mon[31963]: pgmap v386: 97 pgs: 97 active+clean; 668 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-07T07:42:27.912 INFO:tasks.workunit.client.0.smithi160.stderr:Updating files: 99% (12225/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-03-07T07:42:28.602 INFO:tasks.workunit.client.0.smithi160.stderr:Note: switching to '8b896a9e145796119f0451201dc4d53ddac97db0'. 2024-03-07T07:42:28.602 INFO:tasks.workunit.client.0.smithi160.stderr: 2024-03-07T07:42:28.602 INFO:tasks.workunit.client.0.smithi160.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-07T07:42:28.602 INFO:tasks.workunit.client.0.smithi160.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-07T07:42:28.602 INFO:tasks.workunit.client.0.smithi160.stderr:state without impacting any branches by switching back to a branch. 2024-03-07T07:42:28.602 INFO:tasks.workunit.client.0.smithi160.stderr: 2024-03-07T07:42:28.602 INFO:tasks.workunit.client.0.smithi160.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-07T07:42:28.602 INFO:tasks.workunit.client.0.smithi160.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-07T07:42:28.602 INFO:tasks.workunit.client.0.smithi160.stderr: 2024-03-07T07:42:28.602 INFO:tasks.workunit.client.0.smithi160.stderr: git switch -c 2024-03-07T07:42:28.602 INFO:tasks.workunit.client.0.smithi160.stderr: 2024-03-07T07:42:28.603 INFO:tasks.workunit.client.0.smithi160.stderr:Or undo this operation with: 2024-03-07T07:42:28.603 INFO:tasks.workunit.client.0.smithi160.stderr: 2024-03-07T07:42:28.603 INFO:tasks.workunit.client.0.smithi160.stderr: git switch - 2024-03-07T07:42:28.603 INFO:tasks.workunit.client.0.smithi160.stderr: 2024-03-07T07:42:28.603 INFO:tasks.workunit.client.0.smithi160.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-07T07:42:28.603 INFO:tasks.workunit.client.0.smithi160.stderr: 2024-03-07T07:42:28.603 INFO:tasks.workunit.client.0.smithi160.stderr:HEAD is now at 8b896a9e145 mds/quiesce-db: keep the db thread alive until shutdown 2024-03-07T07:42:28.606 DEBUG:teuthology.orchestra.run.smithi160:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-07T07:42:28.673 INFO:tasks.workunit.client.0.smithi160.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-07T07:42:28.675 INFO:tasks.workunit.client.0.smithi160.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-07T07:42:28.675 INFO:tasks.workunit.client.0.smithi160.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-07T07:42:29.424 INFO:tasks.workunit.client.0.smithi160.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-07T07:42:29.495 INFO:tasks.workunit.client.0.smithi160.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-07T07:42:29.536 INFO:tasks.workunit.client.0.smithi160.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-07T07:42:29.537 INFO:tasks.workunit.client.0.smithi160.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-07T07:42:29.537 INFO:tasks.workunit.client.0.smithi160.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-07T07:42:29.576 INFO:tasks.workunit.client.0.smithi160.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-07T07:42:29.579 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-03-07T07:42:29.579 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-07T07:42:29.635 INFO:tasks.workunit:Running workunits matching suites/fsync-tester.sh on client.0... 2024-03-07T07:42:29.636 INFO:tasks.workunit:Running workunit suites/fsync-tester.sh... 2024-03-07T07:42:29.636 DEBUG:teuthology.orchestra.run.smithi160:workunit test suites/fsync-tester.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8b896a9e145796119f0451201dc4d53ddac97db0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsync-tester.sh 2024-03-07T07:42:29.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:42:29 smithi160 ceph-mon[31963]: pgmap v387: 97 pgs: 97 active+clean; 668 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-07T07:42:29.695 INFO:tasks.workunit.client.0.smithi160.stderr:+ wget http://download.ceph.com/qa/fsync-tester.c 2024-03-07T07:42:29.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:42:29 smithi178 ceph-mon[36650]: pgmap v387: 97 pgs: 97 active+clean; 668 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-07T07:42:29.807 INFO:tasks.workunit.client.0.smithi160.stderr:--2024-03-07 07:42:29-- http://download.ceph.com/qa/fsync-tester.c 2024-03-07T07:42:29.807 INFO:tasks.workunit.client.0.smithi160.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-03-07T07:42:29.828 INFO:tasks.workunit.client.0.smithi160.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-03-07T07:42:29.849 INFO:tasks.workunit.client.0.smithi160.stderr:HTTP request sent, awaiting response... 200 OK 2024-03-07T07:42:29.849 INFO:tasks.workunit.client.0.smithi160.stderr:Length: 3024 (3.0K) [application/octet-stream] 2024-03-07T07:42:29.850 INFO:tasks.workunit.client.0.smithi160.stderr:Saving to: ‘fsync-tester.c’ 2024-03-07T07:42:29.853 INFO:tasks.workunit.client.0.smithi160.stderr: 2024-03-07T07:42:29.853 INFO:tasks.workunit.client.0.smithi160.stderr: 0K .. 100% 519M=0s 2024-03-07T07:42:29.853 INFO:tasks.workunit.client.0.smithi160.stderr: 2024-03-07T07:42:29.853 INFO:tasks.workunit.client.0.smithi160.stderr:2024-03-07 07:42:29 (519 MB/s) - ‘fsync-tester.c’ saved [3024/3024] 2024-03-07T07:42:29.853 INFO:tasks.workunit.client.0.smithi160.stderr: 2024-03-07T07:42:29.853 INFO:tasks.workunit.client.0.smithi160.stderr:+ gcc -D_GNU_SOURCE fsync-tester.c -o fsync-tester 2024-03-07T07:42:29.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:42:29 smithi177 ceph-mon[36578]: pgmap v387: 97 pgs: 97 active+clean; 668 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-07T07:42:29.914 INFO:tasks.workunit.client.0.smithi160.stderr:+ ./fsync-tester 2024-03-07T07:42:30.629 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.617+0000 7f76d891d640 1 -- 172.21.15.160:0/2181248021 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f76d4101770 msgr2=0x7f76d4101bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:42:30.629 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.617+0000 7f76d891d640 1 --2- 172.21.15.160:0/2181248021 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f76d4101770 0x7f76d4101bf0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f76bc009900 tx=0x7f76bc02bc30 comp rx=0 tx=0).stop 2024-03-07T07:42:30.629 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.617+0000 7f76d891d640 1 -- 172.21.15.160:0/2181248021 shutdown_connections 2024-03-07T07:42:30.629 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.617+0000 7f76d891d640 1 --2- 172.21.15.160:0/2181248021 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76d4102130 0x7f76d41089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:30.629 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.617+0000 7f76d891d640 1 --2- 172.21.15.160:0/2181248021 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f76d4101770 0x7f76d4101bf0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:30.629 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.617+0000 7f76d891d640 1 --2- 172.21.15.160:0/2181248021 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76d4100580 0x7f76d41009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.617+0000 7f76d891d640 1 -- 172.21.15.160:0/2181248021 >> 172.21.15.160:0/2181248021 conn(0x7f76d40fbaf0 msgr2=0x7f76d40fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:42:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.618+0000 7f76d891d640 1 -- 172.21.15.160:0/2181248021 shutdown_connections 2024-03-07T07:42:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.618+0000 7f76d891d640 1 -- 172.21.15.160:0/2181248021 wait complete. 2024-03-07T07:42:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.618+0000 7f76d891d640 1 Processor -- start 2024-03-07T07:42:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.618+0000 7f76d891d640 1 -- start start 2024-03-07T07:42:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.618+0000 7f76d891d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f76d4100580 0x7f76d4069820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.618+0000 7f76d891d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76d4101770 0x7f76d419c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.618+0000 7f76d891d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76d4102130 0x7f76d419ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.618+0000 7f76d891d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d410c210 con 0x7f76d4100580 2024-03-07T07:42:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.619+0000 7f76d891d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d410c090 con 0x7f76d4101770 2024-03-07T07:42:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.619+0000 7f76d891d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d410c390 con 0x7f76d4102130 2024-03-07T07:42:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.619+0000 7f76d259c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f76d4100580 0x7f76d4069820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.619+0000 7f76cbfff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76d4101770 0x7f76d419c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.619+0000 7f76d259c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f76d4100580 0x7f76d4069820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:48332/0 (socket says 172.21.15.160:48332) 2024-03-07T07:42:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.619+0000 7f76d259c640 1 -- 172.21.15.160:0/2922518606 learned_addr learned my addr 172.21.15.160:0/2922518606 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:42:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.619+0000 7f76d2d9d640 1 --2- 172.21.15.160:0/2922518606 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76d4102130 0x7f76d419ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.619+0000 7f76cbfff640 1 -- 172.21.15.160:0/2922518606 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76d4102130 msgr2=0x7f76d419ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:42:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.619+0000 7f76cbfff640 1 --2- 172.21.15.160:0/2922518606 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76d4102130 0x7f76d419ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.619+0000 7f76cbfff640 1 -- 172.21.15.160:0/2922518606 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f76d4100580 msgr2=0x7f76d4069820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:42:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.619+0000 7f76cbfff640 1 --2- 172.21.15.160:0/2922518606 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f76d4100580 0x7f76d4069820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.619+0000 7f76cbfff640 1 -- 172.21.15.160:0/2922518606 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d419f4f0 con 0x7f76d4101770 2024-03-07T07:42:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.619+0000 7f76d259c640 1 --2- 172.21.15.160:0/2922518606 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f76d4100580 0x7f76d4069820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:42:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.619+0000 7f76cbfff640 1 --2- 172.21.15.160:0/2922518606 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76d4101770 0x7f76d419c640 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f76bc009230 tx=0x7f76bc02c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:42:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.619+0000 7f76cb7fe640 1 -- 172.21.15.160:0/2922518606 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76bc041070 con 0x7f76d4101770 2024-03-07T07:42:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.619+0000 7f76cb7fe640 1 -- 172.21.15.160:0/2922518606 <== mon.2 v2:172.21.15.177:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f76bc004450 con 0x7f76d4101770 2024-03-07T07:42:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.619+0000 7f76d891d640 1 -- 172.21.15.160:0/2922518606 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76d419f780 con 0x7f76d4101770 2024-03-07T07:42:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.619+0000 7f76cb7fe640 1 -- 172.21.15.160:0/2922518606 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76bc003df0 con 0x7f76d4101770 2024-03-07T07:42:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.619+0000 7f76d891d640 1 -- 172.21.15.160:0/2922518606 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76d419fc70 con 0x7f76d4101770 2024-03-07T07:42:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.621+0000 7f76cb7fe640 1 -- 172.21.15.160:0/2922518606 <== mon.2 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 19) v1 ==== 136805+0+0 (secure 0 0 0) 0x7f76bc044070 con 0x7f76d4101770 2024-03-07T07:42:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.622+0000 7f76cb7fe640 1 --2- 172.21.15.160:0/2922518606 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f76c00a1ec0 0x7f76c00a4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.622+0000 7f76db7e6740 1 Processor -- start 2024-03-07T07:42:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.622+0000 7f76cb7fe640 1 -- 172.21.15.160:0/2922518606 <== mon.2 v2:172.21.15.177:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f76bc0ef880 con 0x7f76d4101770 2024-03-07T07:42:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.622+0000 7f76d259c640 1 --2- 172.21.15.160:0/2922518606 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f76c00a1ec0 0x7f76c00a4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.622+0000 7f76db7e6740 1 -- start start 2024-03-07T07:42:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.622+0000 7f76db7e6740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a2ca2915b0 0x55a2ca2919d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.622+0000 7f76db7e6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2ca291fa0 0x55a2ca296aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.622+0000 7f76db7e6740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a2ca297110 0x55a2ca2995a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.622+0000 7f76db7e6740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2ca1f25c0 con 0x55a2ca2915b0 2024-03-07T07:42:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.622+0000 7f76db7e6740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2ca19b900 con 0x55a2ca297110 2024-03-07T07:42:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.622+0000 7f76db7e6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2ca1ee660 con 0x55a2ca291fa0 2024-03-07T07:42:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.623+0000 7f76d2d9d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a2ca297110 0x55a2ca2995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.623+0000 7f76cbfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2ca291fa0 0x55a2ca296aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.623+0000 7f76cbfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2ca291fa0 0x55a2ca296aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.160:58920/0 (socket says 172.21.15.160:58920) 2024-03-07T07:42:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.623+0000 7f76d259c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a2ca2915b0 0x55a2ca2919d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.623+0000 7f76d2d9d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a2ca297110 0x55a2ca2995a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.160:37814/0 (socket says 172.21.15.160:37814) 2024-03-07T07:42:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.623+0000 7f76cbfff640 1 -- 172.21.15.160:0/2461231203 learned_addr learned my addr 172.21.15.160:0/2461231203 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:42:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.623+0000 7f76d259c640 1 --2- 172.21.15.160:0/2922518606 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f76c00a1ec0 0x7f76c00a4380 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f76c40058c0 tx=0x7f76c4011940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:42:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.623+0000 7f76cbfff640 1 -- 172.21.15.160:0/2461231203 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a2ca297110 msgr2=0x55a2ca2995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:42:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.623+0000 7f76cbfff640 1 --2- 172.21.15.160:0/2461231203 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a2ca297110 0x55a2ca2995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.623+0000 7f76cbfff640 1 -- 172.21.15.160:0/2461231203 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a2ca2915b0 msgr2=0x55a2ca2919d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:42:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.623+0000 7f76cbfff640 1 --2- 172.21.15.160:0/2461231203 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a2ca2915b0 0x55a2ca2919d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.623+0000 7f76cbfff640 1 -- 172.21.15.160:0/2461231203 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2c9f07210 con 0x55a2ca291fa0 2024-03-07T07:42:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.623+0000 7f76cbfff640 1 --2- 172.21.15.160:0/2461231203 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2ca291fa0 0x55a2ca296aa0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f76bc0c47d0 tx=0x7f76bc02bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c98e6b38240c056c server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:42:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.623+0000 7f76c8ff9640 1 -- 172.21.15.160:0/2461231203 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76bc04b070 con 0x55a2ca291fa0 2024-03-07T07:42:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.623+0000 7f76c8ff9640 1 -- 172.21.15.160:0/2461231203 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f76bc0b8040 con 0x55a2ca291fa0 2024-03-07T07:42:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.623+0000 7f76c8ff9640 1 -- 172.21.15.160:0/2461231203 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76bc0035f0 con 0x55a2ca291fa0 2024-03-07T07:42:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.624+0000 7f76db7e6740 1 -- 172.21.15.160:0/2461231203 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2ca291fa0 msgr2=0x55a2ca296aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:42:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.624+0000 7f76db7e6740 1 --2- 172.21.15.160:0/2461231203 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2ca291fa0 0x55a2ca296aa0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f76bc0c47d0 tx=0x7f76bc02bc30 comp rx=0 tx=0).stop 2024-03-07T07:42:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.624+0000 7f76db7e6740 1 -- 172.21.15.160:0/2461231203 shutdown_connections 2024-03-07T07:42:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.624+0000 7f76db7e6740 1 --2- 172.21.15.160:0/2461231203 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a2ca297110 0x55a2ca2995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.624+0000 7f76db7e6740 1 --2- 172.21.15.160:0/2461231203 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2ca291fa0 0x55a2ca296aa0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.624+0000 7f76db7e6740 1 --2- 172.21.15.160:0/2461231203 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a2ca2915b0 0x55a2ca2919d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.624+0000 7f76db7e6740 1 -- 172.21.15.160:0/2461231203 >> 172.21.15.160:0/2461231203 conn(0x55a2ca255580 msgr2=0x55a2ca288b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:42:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.624+0000 7f76db7e6740 1 -- 172.21.15.160:0/2461231203 shutdown_connections 2024-03-07T07:42:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.624+0000 7f76db7e6740 1 -- 172.21.15.160:0/2461231203 wait complete. 2024-03-07T07:42:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.625+0000 7f76db7e6740 1 -- start start 2024-03-07T07:42:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.625+0000 7f76db7e6740 1 Processor -- start 2024-03-07T07:42:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.625+0000 7f76db7e6740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a2ca2915b0 0x55a2ca29b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.625+0000 7f76db7e6740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a2ca291fa0 0x55a2ca29bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.625+0000 7f76db7e6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2ca297110 0x55a2ca2a2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.625+0000 7f76db7e6740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2ca1ee660 con 0x55a2ca291fa0 2024-03-07T07:42:30.636 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.625+0000 7f76db7e6740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2ca1aed60 con 0x55a2ca2915b0 2024-03-07T07:42:30.636 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.625+0000 7f76db7e6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2ca299e80 con 0x55a2ca297110 2024-03-07T07:42:30.636 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.625+0000 7f76db7e6740 10 client.0 _ll_register_callbacks cb 0x55a2c9f06a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-07T07:42:30.636 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.625+0000 7f76cbfff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a2ca291fa0 0x55a2ca29bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:30.636 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.625+0000 7f76d259c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a2ca2915b0 0x55a2ca29b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:30.636 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.625+0000 7f76cbfff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a2ca291fa0 0x55a2ca29bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:48350/0 (socket says 172.21.15.160:48350) 2024-03-07T07:42:30.636 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.625+0000 7f76d259c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a2ca2915b0 0x55a2ca29b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.160:37822/0 (socket says 172.21.15.160:37822) 2024-03-07T07:42:30.636 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.625+0000 7f76cbfff640 1 -- 172.21.15.160:0/3972945641 learned_addr learned my addr 172.21.15.160:0/3972945641 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:42:30.636 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.625+0000 7f76d2d9d640 1 --2- 172.21.15.160:0/3972945641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2ca297110 0x55a2ca2a2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:30.636 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.626+0000 7f76cbfff640 1 -- 172.21.15.160:0/3972945641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2ca297110 msgr2=0x55a2ca2a2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:42:30.637 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.626+0000 7f76cbfff640 1 --2- 172.21.15.160:0/3972945641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2ca297110 0x55a2ca2a2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:30.637 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.626+0000 7f76cbfff640 1 -- 172.21.15.160:0/3972945641 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a2ca2915b0 msgr2=0x55a2ca29b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:42:30.637 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.626+0000 7f76cbfff640 1 --2- 172.21.15.160:0/3972945641 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a2ca2915b0 0x55a2ca29b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:30.637 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.626+0000 7f76cbfff640 1 -- 172.21.15.160:0/3972945641 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a2c9fd3620 con 0x55a2ca291fa0 2024-03-07T07:42:30.637 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.626+0000 7f76d259c640 1 --2- 172.21.15.160:0/3972945641 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a2ca2915b0 0x55a2ca29b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:42:30.637 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.626+0000 7f76cbfff640 1 --2- 172.21.15.160:0/3972945641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a2ca291fa0 0x55a2ca29bad0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f76bc005820 tx=0x7f76bc02c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a25ac49560eccd0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:42:30.637 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.626+0000 7f76b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-03-07T07:42:30.637 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.626+0000 7f76b9ffb640 1 -- 172.21.15.160:0/3972945641 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76bc04b050 con 0x55a2ca291fa0 2024-03-07T07:42:30.637 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.626+0000 7f76b9ffb640 1 -- 172.21.15.160:0/3972945641 <== mon.0 v2:172.21.15.160:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f76bc0c4870 con 0x55a2ca291fa0 2024-03-07T07:42:30.637 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.626+0000 7f76db7e6740 1 -- 172.21.15.160:0/3972945641 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f76bc0b8040 con 0x55a2ca291fa0 2024-03-07T07:42:30.637 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.626+0000 7f76b9ffb640 1 -- 172.21.15.160:0/3972945641 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76bc0bc610 con 0x55a2ca291fa0 2024-03-07T07:42:30.637 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.626+0000 7f76db7e6740 10 client.15375 fetch_fsmap learned FSMap version 22 2024-03-07T07:42:30.637 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.626+0000 7f76db7e6740 1 -- 172.21.15.160:0/3972945641 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55a2ca2a5e80 con 0x55a2ca291fa0 2024-03-07T07:42:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.626+0000 7f76b9ffb640 1 -- 172.21.15.160:0/3972945641 <== mon.0 v2:172.21.15.160:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f76bc0b8040 con 0x55a2ca291fa0 2024-03-07T07:42:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.626+0000 7f76b9ffb640 1 -- 172.21.15.160:0/3972945641 <== mon.0 v2:172.21.15.160:3300/0 5 ==== fsmap(e 22) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f76bc0bba60 con 0x55a2ca291fa0 2024-03-07T07:42:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.627+0000 7f76db7e6740 10 client.15375 fetch_fsmap finished waiting for FSMap version 22 2024-03-07T07:42:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.627+0000 7f76db7e6740 10 client.15375 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-07T07:42:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.627+0000 7f76db7e6740 20 client.15375 populate_metadata read hostname 'smithi160' 2024-03-07T07:42:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.627+0000 7f76db7e6740 1 --2- 172.21.15.160:0/3972945641 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x55a2ca2a7370 0x55a2ca2a9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.627+0000 7f76db7e6740 4 client.15375 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-07T07:42:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.627+0000 7f76db7e6740 1 -- 172.21.15.160:0/3972945641 --> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a2ca1ffb00 con 0x55a2ca2a7370 2024-03-07T07:42:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.627+0000 7f76d259c640 1 --2- 172.21.15.160:0/3972945641 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x55a2ca2a7370 0x55a2ca2a9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.627+0000 7f76b9ffb640 10 client.15375 ms_handle_connect on v2:172.21.15.178:6838/2947354024 2024-03-07T07:42:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.627+0000 7f76d259c640 1 --2- 172.21.15.160:0/3972945641 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x55a2ca2a7370 0x55a2ca2a9760 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=235bd32535979f43 server_cookie=a93db3a0fec40a54 in_seq=0 out_seq=0 2024-03-07T07:42:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.628+0000 7f76b9ffb640 1 -- 172.21.15.160:0/3972945641 <== mds.0 v2:172.21.15.178:6838/2947354024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a2ca1ffb00 con 0x55a2ca2a7370 2024-03-07T07:42:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.628+0000 7f76b9ffb640 10 client.15375 handle_command_reply: tid=0 2024-03-07T07:42:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.628+0000 7f76db7e6740 1 client.15375 shutdown 2024-03-07T07:42:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.628+0000 7f76db7e6740 1 -- 172.21.15.160:0/3972945641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a2ca291fa0 msgr2=0x55a2ca29bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:42:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.628+0000 7f76db7e6740 1 --2- 172.21.15.160:0/3972945641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a2ca291fa0 0x55a2ca29bad0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f76bc005820 tx=0x7f76bc02c450 comp rx=0 tx=0).stop 2024-03-07T07:42:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.628+0000 7f76db7e6740 1 -- 172.21.15.160:0/3972945641 shutdown_connections 2024-03-07T07:42:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.628+0000 7f76db7e6740 1 --2- 172.21.15.160:0/3972945641 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x55a2ca2a7370 0x55a2ca2a9760 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.628+0000 7f76db7e6740 1 --2- 172.21.15.160:0/3972945641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2ca297110 0x55a2ca2a2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.628+0000 7f76b9ffb640 0 client.15375 ms_handle_reset on v2:172.21.15.178:6838/2947354024 2024-03-07T07:42:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.628+0000 7f76db7e6740 1 --2- 172.21.15.160:0/3972945641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a2ca291fa0 0x55a2ca29bad0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.628+0000 7f76db7e6740 1 --2- 172.21.15.160:0/3972945641 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a2ca2915b0 0x55a2ca29b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.628+0000 7f76db7e6740 1 -- 172.21.15.160:0/3972945641 >> 172.21.15.160:0/3972945641 conn(0x55a2ca255580 msgr2=0x55a2ca287e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:42:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.628+0000 7f76db7e6740 1 -- 172.21.15.160:0/3972945641 shutdown_connections 2024-03-07T07:42:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.628+0000 7f76db7e6740 1 -- 172.21.15.160:0/3972945641 wait complete. 2024-03-07T07:42:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.629+0000 7f76db7e6740 20 client.15375 trim_cache size 0 max 16384 2024-03-07T07:42:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.642+0000 7f76db7e6740 1 Processor -- start 2024-03-07T07:42:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.642+0000 7f76db7e6740 1 -- start start 2024-03-07T07:42:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.643+0000 7f76db7e6740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a2ca2915b0 0x55a2ca288590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.643+0000 7f76db7e6740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a2ca291fa0 0x55a2ca2a2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.643+0000 7f76db7e6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2ca297110 0x55a2ca2ae720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.643+0000 7f76db7e6740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d41a8c90 con 0x55a2ca2915b0 2024-03-07T07:42:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.643+0000 7f76db7e6740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d41a6790 con 0x55a2ca291fa0 2024-03-07T07:42:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.643+0000 7f76db7e6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d41a4810 con 0x55a2ca297110 2024-03-07T07:42:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.643+0000 7f76d2d9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2ca297110 0x55a2ca2ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.643+0000 7f76cbfff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a2ca291fa0 0x55a2ca2a2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.643+0000 7f76d259c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a2ca2915b0 0x55a2ca288590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.643+0000 7f76d2d9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2ca297110 0x55a2ca2ae720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.160:58938/0 (socket says 172.21.15.160:58938) 2024-03-07T07:42:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.643+0000 7f76cbfff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a2ca291fa0 0x55a2ca2a2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.160:37826/0 (socket says 172.21.15.160:37826) 2024-03-07T07:42:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.643+0000 7f76d2d9d640 1 -- 172.21.15.160:0/2190463449 learned_addr learned my addr 172.21.15.160:0/2190463449 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:42:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.643+0000 7f76d2d9d640 1 -- 172.21.15.160:0/2190463449 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a2ca291fa0 msgr2=0x55a2ca2a2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:42:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.643+0000 7f76d2d9d640 1 --2- 172.21.15.160:0/2190463449 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a2ca291fa0 0x55a2ca2a2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.643+0000 7f76d2d9d640 1 -- 172.21.15.160:0/2190463449 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a2ca2915b0 msgr2=0x55a2ca288590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:42:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.643+0000 7f76d2d9d640 1 --2- 172.21.15.160:0/2190463449 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a2ca2915b0 0x55a2ca288590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.643+0000 7f76d2d9d640 1 -- 172.21.15.160:0/2190463449 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2ca2a9d90 con 0x55a2ca297110 2024-03-07T07:42:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.643+0000 7f76cbfff640 1 --2- 172.21.15.160:0/2190463449 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a2ca291fa0 0x55a2ca2a2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:42:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.643+0000 7f76d2d9d640 1 --2- 172.21.15.160:0/2190463449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2ca297110 0x55a2ca2ae720 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f76cc003c60 tx=0x7f76cc004410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19a906f5e96c4115 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:42:30.650 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.643+0000 7f76bbfff640 1 -- 172.21.15.160:0/2190463449 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76cc010430 con 0x55a2ca297110 2024-03-07T07:42:30.650 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.644+0000 7f76bbfff640 1 -- 172.21.15.160:0/2190463449 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f76cc010ab0 con 0x55a2ca297110 2024-03-07T07:42:30.650 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.644+0000 7f76bbfff640 1 -- 172.21.15.160:0/2190463449 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76cc01bb00 con 0x55a2ca297110 2024-03-07T07:42:30.650 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.644+0000 7f76db7e6740 1 -- 172.21.15.160:0/2190463449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2ca297110 msgr2=0x55a2ca2ae720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:42:30.650 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.644+0000 7f76db7e6740 1 --2- 172.21.15.160:0/2190463449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2ca297110 0x55a2ca2ae720 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f76cc003c60 tx=0x7f76cc004410 comp rx=0 tx=0).stop 2024-03-07T07:42:30.650 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.644+0000 7f76db7e6740 1 -- 172.21.15.160:0/2190463449 shutdown_connections 2024-03-07T07:42:30.650 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.644+0000 7f76db7e6740 1 --2- 172.21.15.160:0/2190463449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2ca297110 0x55a2ca2ae720 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:30.650 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.644+0000 7f76db7e6740 1 --2- 172.21.15.160:0/2190463449 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a2ca291fa0 0x55a2ca2a2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:30.650 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.644+0000 7f76db7e6740 1 --2- 172.21.15.160:0/2190463449 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a2ca2915b0 0x55a2ca288590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:30.650 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.644+0000 7f76db7e6740 1 -- 172.21.15.160:0/2190463449 >> 172.21.15.160:0/2190463449 conn(0x55a2ca255580 msgr2=0x55a2ca2ae010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:42:30.650 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.644+0000 7f76db7e6740 1 -- 172.21.15.160:0/2190463449 shutdown_connections 2024-03-07T07:42:30.650 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.644+0000 7f76db7e6740 1 -- 172.21.15.160:0/2190463449 wait complete. 2024-03-07T07:42:30.650 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.645+0000 7f76db7e6740 1 -- start start 2024-03-07T07:42:30.651 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.645+0000 7f76db7e6740 1 Processor -- start 2024-03-07T07:42:30.651 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.645+0000 7f76db7e6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2ca2915b0 0x55a2ca2a25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:30.651 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.645+0000 7f76db7e6740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a2ca291fa0 0x55a2ca2a2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:30.651 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.645+0000 7f76db7e6740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a2ca297110 0x55a2ca2a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:30.651 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.645+0000 7f76db7e6740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d41a4e10 con 0x55a2ca291fa0 2024-03-07T07:42:30.651 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.645+0000 7f76db7e6740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d41a6130 con 0x55a2ca297110 2024-03-07T07:42:30.651 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.645+0000 7f76db7e6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d41a5470 con 0x55a2ca2915b0 2024-03-07T07:42:30.651 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.645+0000 7f76db7e6740 10 client.0 _ll_register_callbacks cb 0x7f76c4009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-07T07:42:30.651 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.645+0000 7f76cbfff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a2ca291fa0 0x55a2ca2a2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:30.652 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.645+0000 7f76d259c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2ca2915b0 0x55a2ca2a25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:30.652 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.645+0000 7f76d2d9d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a2ca297110 0x55a2ca2a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:30.652 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.645+0000 7f76cbfff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a2ca291fa0 0x55a2ca2a2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:48358/0 (socket says 172.21.15.160:48358) 2024-03-07T07:42:30.652 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.645+0000 7f76cbfff640 1 -- 172.21.15.160:0/1822894670 learned_addr learned my addr 172.21.15.160:0/1822894670 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:42:30.652 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.645+0000 7f76cbfff640 1 -- 172.21.15.160:0/1822894670 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2ca2915b0 msgr2=0x55a2ca2a25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:42:30.652 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.645+0000 7f76cbfff640 1 --2- 172.21.15.160:0/1822894670 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2ca2915b0 0x55a2ca2a25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:30.652 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.645+0000 7f76cbfff640 1 -- 172.21.15.160:0/1822894670 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a2ca297110 msgr2=0x55a2ca2a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:42:30.652 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.645+0000 7f76cbfff640 1 --2- 172.21.15.160:0/1822894670 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a2ca297110 0x55a2ca2a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:30.652 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.645+0000 7f76cbfff640 1 -- 172.21.15.160:0/1822894670 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a2ca29be40 con 0x55a2ca291fa0 2024-03-07T07:42:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.645+0000 7f76d2d9d640 1 --2- 172.21.15.160:0/1822894670 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a2ca297110 0x55a2ca2a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:42:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.645+0000 7f76d259c640 1 --2- 172.21.15.160:0/1822894670 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2ca2915b0 0x55a2ca2a25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:42:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.646+0000 7f76cbfff640 1 --2- 172.21.15.160:0/1822894670 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a2ca291fa0 0x55a2ca2a2b20 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f76bc0dc300 tx=0x7f76bc0ed690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a76ba15a7cae2 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:42:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.646+0000 7f76b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-03-07T07:42:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.646+0000 7f76b9ffb640 1 -- 172.21.15.160:0/1822894670 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76bc04b070 con 0x55a2ca291fa0 2024-03-07T07:42:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.646+0000 7f76b9ffb640 1 -- 172.21.15.160:0/1822894670 <== mon.0 v2:172.21.15.160:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f76bc0b8c80 con 0x55a2ca291fa0 2024-03-07T07:42:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.646+0000 7f76db7e6740 1 -- 172.21.15.160:0/1822894670 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f76cc010ab0 con 0x55a2ca291fa0 2024-03-07T07:42:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.646+0000 7f76b9ffb640 1 -- 172.21.15.160:0/1822894670 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76bc0c06b0 con 0x55a2ca291fa0 2024-03-07T07:42:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.646+0000 7f76db7e6740 10 client.15381 fetch_fsmap learned FSMap version 22 2024-03-07T07:42:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.646+0000 7f76db7e6740 1 -- 172.21.15.160:0/1822894670 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55a2ca29be40 con 0x55a2ca291fa0 2024-03-07T07:42:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.646+0000 7f76b9ffb640 1 -- 172.21.15.160:0/1822894670 <== mon.0 v2:172.21.15.160:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f76cc010ab0 con 0x55a2ca291fa0 2024-03-07T07:42:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.646+0000 7f76b9ffb640 1 -- 172.21.15.160:0/1822894670 <== mon.0 v2:172.21.15.160:3300/0 5 ==== fsmap(e 22) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f76bc0a5ba0 con 0x55a2ca291fa0 2024-03-07T07:42:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.646+0000 7f76db7e6740 10 client.15381 fetch_fsmap finished waiting for FSMap version 22 2024-03-07T07:42:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.646+0000 7f76db7e6740 10 client.15381 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-07T07:42:30.654 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.646+0000 7f76db7e6740 20 client.15381 populate_metadata read hostname 'smithi160' 2024-03-07T07:42:30.654 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.646+0000 7f76db7e6740 1 --2- 172.21.15.160:0/1822894670 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x55a2ca303c20 0x55a2ca306040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:30.654 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.646+0000 7f76db7e6740 4 client.15381 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-07T07:42:30.654 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.646+0000 7f76db7e6740 1 -- 172.21.15.160:0/1822894670 --> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a2ca1f9e50 con 0x55a2ca303c20 2024-03-07T07:42:30.654 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.647+0000 7f76d259c640 1 --2- 172.21.15.160:0/1822894670 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x55a2ca303c20 0x55a2ca306040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:30.654 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.647+0000 7f76b9ffb640 10 client.15381 ms_handle_connect on v2:172.21.15.178:6838/2947354024 2024-03-07T07:42:30.654 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.647+0000 7f76d259c640 1 --2- 172.21.15.160:0/1822894670 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x55a2ca303c20 0x55a2ca306040 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcb7534e5ca20ef5 server_cookie=b4d44454e456063f in_seq=0 out_seq=0 2024-03-07T07:42:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.656+0000 7f76b9ffb640 1 -- 172.21.15.160:0/1822894670 <== mds.0 v2:172.21.15.178:6838/2947354024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a2ca1f9e50 con 0x55a2ca303c20 2024-03-07T07:42:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.656+0000 7f76b9ffb640 10 client.15381 handle_command_reply: tid=0 2024-03-07T07:42:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.656+0000 7f76db7e6740 1 client.15381 shutdown 2024-03-07T07:42:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.656+0000 7f76db7e6740 1 -- 172.21.15.160:0/1822894670 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a2ca291fa0 msgr2=0x55a2ca2a2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:42:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.656+0000 7f76db7e6740 1 --2- 172.21.15.160:0/1822894670 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a2ca291fa0 0x55a2ca2a2b20 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f76bc0dc300 tx=0x7f76bc0ed690 comp rx=0 tx=0).stop 2024-03-07T07:42:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.656+0000 7f76db7e6740 1 -- 172.21.15.160:0/1822894670 shutdown_connections 2024-03-07T07:42:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.656+0000 7f76db7e6740 1 --2- 172.21.15.160:0/1822894670 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x55a2ca303c20 0x55a2ca306040 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.656+0000 7f76db7e6740 1 --2- 172.21.15.160:0/1822894670 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a2ca297110 0x55a2ca2a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.656+0000 7f76db7e6740 1 --2- 172.21.15.160:0/1822894670 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a2ca291fa0 0x55a2ca2a2b20 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.656+0000 7f76b9ffb640 0 client.15381 ms_handle_reset on v2:172.21.15.178:6838/2947354024 2024-03-07T07:42:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.656+0000 7f76db7e6740 1 --2- 172.21.15.160:0/1822894670 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2ca2915b0 0x55a2ca2a25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.656+0000 7f76db7e6740 1 -- 172.21.15.160:0/1822894670 >> 172.21.15.160:0/1822894670 conn(0x55a2ca255580 msgr2=0x55a2ca2ae8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:42:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.656+0000 7f76db7e6740 1 -- 172.21.15.160:0/1822894670 shutdown_connections 2024-03-07T07:42:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.656+0000 7f76db7e6740 1 -- 172.21.15.160:0/1822894670 wait complete. 2024-03-07T07:42:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.656+0000 7f76db7e6740 20 client.15381 trim_cache size 0 max 16384 2024-03-07T07:42:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stdout:{ 2024-03-07T07:42:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stdout: "message": "", 2024-03-07T07:42:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stdout: "return_code": 0 2024-03-07T07:42:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stdout:} 2024-03-07T07:42:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.657+0000 7f76d891d640 1 -- 172.21.15.160:0/2922518606 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f76c00a1ec0 msgr2=0x7f76c00a4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:42:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.657+0000 7f76d891d640 1 --2- 172.21.15.160:0/2922518606 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f76c00a1ec0 0x7f76c00a4380 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f76c40058c0 tx=0x7f76c4011940 comp rx=0 tx=0).stop 2024-03-07T07:42:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.657+0000 7f76d891d640 1 -- 172.21.15.160:0/2922518606 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76d4101770 msgr2=0x7f76d419c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:42:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.657+0000 7f76d891d640 1 --2- 172.21.15.160:0/2922518606 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76d4101770 0x7f76d419c640 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f76bc009230 tx=0x7f76bc02c650 comp rx=0 tx=0).stop 2024-03-07T07:42:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.657+0000 7f76d891d640 1 -- 172.21.15.160:0/2922518606 shutdown_connections 2024-03-07T07:42:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.657+0000 7f76d891d640 1 --2- 172.21.15.160:0/2922518606 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f76c00a1ec0 0x7f76c00a4380 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.657+0000 7f76d891d640 1 --2- 172.21.15.160:0/2922518606 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76d4102130 0x7f76d419ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.657+0000 7f76d891d640 1 --2- 172.21.15.160:0/2922518606 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76d4101770 0x7f76d419c640 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.657+0000 7f76d891d640 1 --2- 172.21.15.160:0/2922518606 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f76d4100580 0x7f76d4069820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.657+0000 7f76d891d640 1 -- 172.21.15.160:0/2922518606 >> 172.21.15.160:0/2922518606 conn(0x7f76d40fbaf0 msgr2=0x7f76d4104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:42:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.657+0000 7f76d891d640 1 -- 172.21.15.160:0/2922518606 shutdown_connections 2024-03-07T07:42:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:30.657+0000 7f76d891d640 1 -- 172.21.15.160:0/2922518606 wait complete. 2024-03-07T07:42:31.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:42:31 smithi160 ceph-mon[31963]: pgmap v388: 97 pgs: 97 active+clean; 668 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-07T07:42:31.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:42:31 smithi178 ceph-mon[36650]: pgmap v388: 97 pgs: 97 active+clean; 668 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-07T07:42:31.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:42:31 smithi177 ceph-mon[36578]: pgmap v388: 97 pgs: 97 active+clean; 668 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-07T07:42:33.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:42:33 smithi160 ceph-mon[31963]: pgmap v389: 97 pgs: 97 active+clean; 668 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-07T07:42:33.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:42:33 smithi178 ceph-mon[36650]: pgmap v389: 97 pgs: 97 active+clean; 668 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-07T07:42:33.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:42:33 smithi177 ceph-mon[36578]: pgmap v389: 97 pgs: 97 active+clean; 668 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-07T07:42:34.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:42:34 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:42:34.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:42:34 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:42:34.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:42:34 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:42:35.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:42:35 smithi160 ceph-mon[31963]: pgmap v390: 97 pgs: 97 active+clean; 1.7 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 342 KiB/s wr, 8 op/s 2024-03-07T07:42:35.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:42:35 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:42:35.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:42:35 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:42:35.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:42:35 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:42:35.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:42:35 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:42:35.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:42:35 smithi178 ceph-mon[36650]: pgmap v390: 97 pgs: 97 active+clean; 1.7 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 342 KiB/s wr, 8 op/s 2024-03-07T07:42:35.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:42:35 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:42:35.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:42:35 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:42:35.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:42:35 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:42:35.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:42:35 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:42:35.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:42:35 smithi177 ceph-mon[36578]: pgmap v390: 97 pgs: 97 active+clean; 1.7 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 342 KiB/s wr, 8 op/s 2024-03-07T07:42:35.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:42:35 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:42:35.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:42:35 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:42:35.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:42:35 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:42:35.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:42:35 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:42:36.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:42:36 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:42:36.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:42:36 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:42:36.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:42:36 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:42:36.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:42:36 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:42:36.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:42:36 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:42:36.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:42:36 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:42:36.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:42:36 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:42:36.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:42:36 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:42:36.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:42:36 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:42:37.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:42:37 smithi160 ceph-mon[31963]: pgmap v391: 97 pgs: 97 active+clean; 1.7 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 349 KiB/s wr, 12 op/s 2024-03-07T07:42:37.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:42:37 smithi178 ceph-mon[36650]: pgmap v391: 97 pgs: 97 active+clean; 1.7 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 349 KiB/s wr, 12 op/s 2024-03-07T07:42:37.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:42:37 smithi177 ceph-mon[36578]: pgmap v391: 97 pgs: 97 active+clean; 1.7 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 349 KiB/s wr, 12 op/s 2024-03-07T07:42:39.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:42:39 smithi160 ceph-mon[31963]: pgmap v392: 97 pgs: 97 active+clean; 1.7 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 349 KiB/s wr, 10 op/s 2024-03-07T07:42:39.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:42:39 smithi178 ceph-mon[36650]: pgmap v392: 97 pgs: 97 active+clean; 1.7 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 349 KiB/s wr, 10 op/s 2024-03-07T07:42:39.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:42:39 smithi177 ceph-mon[36578]: pgmap v392: 97 pgs: 97 active+clean; 1.7 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 349 KiB/s wr, 10 op/s 2024-03-07T07:42:40.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:42:40 smithi178 ceph-mon[36650]: pgmap v393: 97 pgs: 97 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 779 KiB/s wr, 14 op/s 2024-03-07T07:42:40.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:42:40 smithi178 ceph-mon[36650]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-03-07T07:42:40.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:42:40 smithi177 ceph-mon[36578]: pgmap v393: 97 pgs: 97 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 779 KiB/s wr, 14 op/s 2024-03-07T07:42:40.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:42:40 smithi177 ceph-mon[36578]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-03-07T07:42:40.927 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:42:40 smithi160 ceph-mon[31963]: pgmap v393: 97 pgs: 97 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 779 KiB/s wr, 14 op/s 2024-03-07T07:42:40.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:42:40 smithi160 ceph-mon[31963]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-03-07T07:42:43.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:42:43 smithi160 ceph-mon[31963]: pgmap v394: 97 pgs: 97 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 779 KiB/s wr, 12 op/s 2024-03-07T07:42:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.750+0000 7f1f6cbd2640 1 -- 172.21.15.160:0/913033323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f68102c80 msgr2=0x7f1f681030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:42:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.750+0000 7f1f6cbd2640 1 --2- 172.21.15.160:0/913033323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f68102c80 0x7f1f681030a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1f58009900 tx=0x7f1f5802bc30 comp rx=0 tx=0).stop 2024-03-07T07:42:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.751+0000 7f1f6cbd2640 1 -- 172.21.15.160:0/913033323 shutdown_connections 2024-03-07T07:42:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.751+0000 7f1f6cbd2640 1 --2- 172.21.15.160:0/913033323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f681000b0 0x7f1f68109c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.751+0000 7f1f6cbd2640 1 --2- 172.21.15.160:0/913033323 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f680ff6f0 0x7f1f680ffb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.751+0000 7f1f6cbd2640 1 --2- 172.21.15.160:0/913033323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f68102c80 0x7f1f681030a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.751+0000 7f1f6cbd2640 1 -- 172.21.15.160:0/913033323 >> 172.21.15.160:0/913033323 conn(0x7f1f680fb2b0 msgr2=0x7f1f680fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:42:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.751+0000 7f1f6cbd2640 1 -- 172.21.15.160:0/913033323 shutdown_connections 2024-03-07T07:42:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.751+0000 7f1f6cbd2640 1 -- 172.21.15.160:0/913033323 wait complete. 2024-03-07T07:42:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.751+0000 7f1f6cbd2640 1 Processor -- start 2024-03-07T07:42:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.751+0000 7f1f6cbd2640 1 -- start start 2024-03-07T07:42:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.752+0000 7f1f6cbd2640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f680ff6f0 0x7f1f681978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.752+0000 7f1f6cbd2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f681000b0 0x7f1f68197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.752+0000 7f1f6cbd2640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f68102c80 0x7f1f6819eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.752+0000 7f1f6cbd2640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6810d300 con 0x7f1f681000b0 2024-03-07T07:42:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.752+0000 7f1f6cbd2640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6810d180 con 0x7f1f680ff6f0 2024-03-07T07:42:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.752+0000 7f1f6cbd2640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6810d480 con 0x7f1f68102c80 2024-03-07T07:42:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.752+0000 7f1f65d9b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f681000b0 0x7f1f68197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.752+0000 7f1f6659c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f680ff6f0 0x7f1f681978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.752+0000 7f1f66d9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f68102c80 0x7f1f6819eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.752+0000 7f1f6659c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f680ff6f0 0x7f1f681978f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.160:40918/0 (socket says 172.21.15.160:40918) 2024-03-07T07:42:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.752+0000 7f1f6659c640 1 -- 172.21.15.160:0/3578963852 learned_addr learned my addr 172.21.15.160:0/3578963852 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:42:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.752+0000 7f1f65d9b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f681000b0 0x7f1f68197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41058/0 (socket says 172.21.15.160:41058) 2024-03-07T07:42:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.752+0000 7f1f6659c640 1 -- 172.21.15.160:0/3578963852 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f68102c80 msgr2=0x7f1f6819eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:42:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.752+0000 7f1f6659c640 1 --2- 172.21.15.160:0/3578963852 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f68102c80 0x7f1f6819eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.752+0000 7f1f6659c640 1 -- 172.21.15.160:0/3578963852 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f681000b0 msgr2=0x7f1f68197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:42:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.752+0000 7f1f6659c640 1 --2- 172.21.15.160:0/3578963852 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f681000b0 0x7f1f68197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.752+0000 7f1f6659c640 1 -- 172.21.15.160:0/3578963852 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f6819f510 con 0x7f1f680ff6f0 2024-03-07T07:42:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.752+0000 7f1f66d9d640 1 --2- 172.21.15.160:0/3578963852 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f68102c80 0x7f1f6819eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:42:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.752+0000 7f1f6659c640 1 --2- 172.21.15.160:0/3578963852 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f680ff6f0 0x7f1f681978f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1f58000c00 tx=0x7f1f5802c640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:42:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.752+0000 7f1f577fe640 1 -- 172.21.15.160:0/3578963852 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f58041070 con 0x7f1f680ff6f0 2024-03-07T07:42:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.752+0000 7f1f6cbd2640 1 -- 172.21.15.160:0/3578963852 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f6819f7a0 con 0x7f1f680ff6f0 2024-03-07T07:42:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.753+0000 7f1f577fe640 1 -- 172.21.15.160:0/3578963852 <== mon.2 v2:172.21.15.177:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1f58004450 con 0x7f1f680ff6f0 2024-03-07T07:42:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.753+0000 7f1f577fe640 1 -- 172.21.15.160:0/3578963852 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f58034440 con 0x7f1f680ff6f0 2024-03-07T07:42:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.753+0000 7f1f6cbd2640 1 -- 172.21.15.160:0/3578963852 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f6819fce0 con 0x7f1f680ff6f0 2024-03-07T07:42:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.754+0000 7f1f577fe640 1 -- 172.21.15.160:0/3578963852 <== mon.2 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 20) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f1f58044070 con 0x7f1f680ff6f0 2024-03-07T07:42:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.755+0000 7f1f577fe640 1 --2- 172.21.15.160:0/3578963852 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f1f500a1e20 0x7f1f500a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.755+0000 7f1f577fe640 1 -- 172.21.15.160:0/3578963852 <== mon.2 v2:172.21.15.177:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1f580ef820 con 0x7f1f680ff6f0 2024-03-07T07:42:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.755+0000 7f1f65d9b640 1 --2- 172.21.15.160:0/3578963852 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f1f500a1e20 0x7f1f500a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.755+0000 7f1f6f9b7740 1 Processor -- start 2024-03-07T07:42:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.756+0000 7f1f65d9b640 1 --2- 172.21.15.160:0/3578963852 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f1f500a1e20 0x7f1f500a42e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1f68198e00 tx=0x7f1f5c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:42:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.756+0000 7f1f6f9b7740 1 -- start start 2024-03-07T07:42:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.756+0000 7f1f6f9b7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4e33a01a0 0x55d4e33a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.756+0000 7f1f6f9b7740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d4e33a0b90 0x55d4e33a5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.756+0000 7f1f6f9b7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4e33a5d00 0x55d4e33a8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.756+0000 7f1f6f9b7740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4e33055c0 con 0x55d4e33a0b90 2024-03-07T07:42:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.756+0000 7f1f6f9b7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4e32ae900 con 0x55d4e33a5d00 2024-03-07T07:42:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.756+0000 7f1f6f9b7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4e3301660 con 0x55d4e33a01a0 2024-03-07T07:42:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.756+0000 7f1f65d9b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d4e33a0b90 0x55d4e33a5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.756+0000 7f1f66d9d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4e33a5d00 0x55d4e33a8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.756+0000 7f1f65d9b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d4e33a0b90 0x55d4e33a5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41068/0 (socket says 172.21.15.160:41068) 2024-03-07T07:42:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.756+0000 7f1f6659c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4e33a01a0 0x55d4e33a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.756+0000 7f1f65d9b640 1 -- 172.21.15.160:0/16190327 learned_addr learned my addr 172.21.15.160:0/16190327 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:42:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.756+0000 7f1f66d9d640 1 -- 172.21.15.160:0/16190327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4e33a01a0 msgr2=0x55d4e33a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:42:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.756+0000 7f1f66d9d640 1 --2- 172.21.15.160:0/16190327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4e33a01a0 0x55d4e33a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.756+0000 7f1f66d9d640 1 -- 172.21.15.160:0/16190327 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d4e33a0b90 msgr2=0x55d4e33a5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:42:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.756+0000 7f1f66d9d640 1 --2- 172.21.15.160:0/16190327 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d4e33a0b90 0x55d4e33a5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.756+0000 7f1f66d9d640 1 -- 172.21.15.160:0/16190327 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4e301a210 con 0x55d4e33a5d00 2024-03-07T07:42:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.756+0000 7f1f66d9d640 1 --2- 172.21.15.160:0/16190327 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4e33a5d00 0x55d4e33a8190 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1f60005fd0 tx=0x7f1f60011bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0465804d36376fc server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:42:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.757+0000 7f1f54ff9640 1 -- 172.21.15.160:0/16190327 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f60004d20 con 0x55d4e33a5d00 2024-03-07T07:42:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.757+0000 7f1f54ff9640 1 -- 172.21.15.160:0/16190327 <== mon.2 v2:172.21.15.177:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1f6000f9a0 con 0x55d4e33a5d00 2024-03-07T07:42:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.757+0000 7f1f54ff9640 1 -- 172.21.15.160:0/16190327 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f60004510 con 0x55d4e33a5d00 2024-03-07T07:42:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.757+0000 7f1f6f9b7740 1 -- 172.21.15.160:0/16190327 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4e33a5d00 msgr2=0x55d4e33a8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:42:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.757+0000 7f1f6f9b7740 1 --2- 172.21.15.160:0/16190327 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4e33a5d00 0x55d4e33a8190 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1f60005fd0 tx=0x7f1f60011bf0 comp rx=0 tx=0).stop 2024-03-07T07:42:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.757+0000 7f1f6f9b7740 1 -- 172.21.15.160:0/16190327 shutdown_connections 2024-03-07T07:42:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.757+0000 7f1f6f9b7740 1 --2- 172.21.15.160:0/16190327 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4e33a5d00 0x55d4e33a8190 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.757+0000 7f1f6f9b7740 1 --2- 172.21.15.160:0/16190327 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d4e33a0b90 0x55d4e33a5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.757+0000 7f1f6f9b7740 1 --2- 172.21.15.160:0/16190327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4e33a01a0 0x55d4e33a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.757+0000 7f1f6f9b7740 1 -- 172.21.15.160:0/16190327 >> 172.21.15.160:0/16190327 conn(0x55d4e3368580 msgr2=0x55d4e339bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:42:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.758+0000 7f1f6f9b7740 1 -- 172.21.15.160:0/16190327 shutdown_connections 2024-03-07T07:42:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.758+0000 7f1f6f9b7740 1 -- 172.21.15.160:0/16190327 wait complete. 2024-03-07T07:42:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.758+0000 7f1f6f9b7740 1 -- start start 2024-03-07T07:42:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.758+0000 7f1f6f9b7740 1 Processor -- start 2024-03-07T07:42:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.758+0000 7f1f6f9b7740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d4e33a01a0 0x55d4e33aa500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.758+0000 7f1f6f9b7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4e33a0b90 0x55d4e33aaa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.758+0000 7f1f6f9b7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4e33a5d00 0x55d4e33b1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.758+0000 7f1f6f9b7740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4e33a8cb0 con 0x55d4e33a01a0 2024-03-07T07:42:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.758+0000 7f1f6f9b7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4e32c1d60 con 0x55d4e33a5d00 2024-03-07T07:42:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.758+0000 7f1f6f9b7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4e33a8e30 con 0x55d4e33a0b90 2024-03-07T07:42:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.758+0000 7f1f6f9b7740 10 client.0 _ll_register_callbacks cb 0x55d4e3019a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-07T07:42:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.758+0000 7f1f66d9d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4e33a5d00 0x55d4e33b1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.758+0000 7f1f6659c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d4e33a01a0 0x55d4e33aa500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.758+0000 7f1f66d9d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4e33a5d00 0x55d4e33b1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.160:40932/0 (socket says 172.21.15.160:40932) 2024-03-07T07:42:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.758+0000 7f1f6659c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d4e33a01a0 0x55d4e33aa500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41080/0 (socket says 172.21.15.160:41080) 2024-03-07T07:42:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.758+0000 7f1f66d9d640 1 -- 172.21.15.160:0/1852452827 learned_addr learned my addr 172.21.15.160:0/1852452827 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:42:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.758+0000 7f1f65d9b640 1 --2- 172.21.15.160:0/1852452827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4e33a0b90 0x55d4e33aaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.759+0000 7f1f66d9d640 1 -- 172.21.15.160:0/1852452827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4e33a0b90 msgr2=0x55d4e33aaa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:42:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.759+0000 7f1f66d9d640 1 --2- 172.21.15.160:0/1852452827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4e33a0b90 0x55d4e33aaa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.759+0000 7f1f66d9d640 1 -- 172.21.15.160:0/1852452827 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d4e33a01a0 msgr2=0x55d4e33aa500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:42:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.759+0000 7f1f66d9d640 1 --2- 172.21.15.160:0/1852452827 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d4e33a01a0 0x55d4e33aa500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.759+0000 7f1f66d9d640 1 -- 172.21.15.160:0/1852452827 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d4e30e6620 con 0x55d4e33a5d00 2024-03-07T07:42:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.759+0000 7f1f6659c640 1 --2- 172.21.15.160:0/1852452827 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d4e33a01a0 0x55d4e33aa500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:42:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.759+0000 7f1f66d9d640 1 --2- 172.21.15.160:0/1852452827 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4e33a5d00 0x55d4e33b1a80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1f6000a010 tx=0x7f1f60003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b744b1afa0caf468 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:42:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.759+0000 7f1f4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-03-07T07:42:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.759+0000 7f1f4dffb640 1 -- 172.21.15.160:0/1852452827 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f60011020 con 0x55d4e33a5d00 2024-03-07T07:42:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.759+0000 7f1f4dffb640 1 -- 172.21.15.160:0/1852452827 <== mon.2 v2:172.21.15.177:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1f600111c0 con 0x55d4e33a5d00 2024-03-07T07:42:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.759+0000 7f1f6f9b7740 1 -- 172.21.15.160:0/1852452827 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1f6000f9a0 con 0x55d4e33a5d00 2024-03-07T07:42:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.759+0000 7f1f4dffb640 1 -- 172.21.15.160:0/1852452827 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f60005210 con 0x55d4e33a5d00 2024-03-07T07:42:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.759+0000 7f1f6f9b7740 10 client.25229 fetch_fsmap learned FSMap version 22 2024-03-07T07:42:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.759+0000 7f1f6f9b7740 1 -- 172.21.15.160:0/1852452827 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55d4e33b4df0 con 0x55d4e33a5d00 2024-03-07T07:42:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.759+0000 7f1f4dffb640 1 -- 172.21.15.160:0/1852452827 <== mon.2 v2:172.21.15.177:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1f6000f9a0 con 0x55d4e33a5d00 2024-03-07T07:42:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.759+0000 7f1f4dffb640 1 -- 172.21.15.160:0/1852452827 <== mon.2 v2:172.21.15.177:3300/0 5 ==== fsmap(e 22) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f1f6002d020 con 0x55d4e33a5d00 2024-03-07T07:42:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.760+0000 7f1f6f9b7740 10 client.25229 fetch_fsmap finished waiting for FSMap version 22 2024-03-07T07:42:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.760+0000 7f1f6f9b7740 10 client.25229 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-07T07:42:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.760+0000 7f1f6f9b7740 20 client.25229 populate_metadata read hostname 'smithi160' 2024-03-07T07:42:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.760+0000 7f1f6f9b7740 1 --2- 172.21.15.160:0/1852452827 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x55d4e33b6260 0x55d4e33b86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.760+0000 7f1f6f9b7740 4 client.25229 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-07T07:42:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.760+0000 7f1f6f9b7740 1 -- 172.21.15.160:0/1852452827 --> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d4e3312b00 con 0x55d4e33b6260 2024-03-07T07:42:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.760+0000 7f1f6659c640 1 --2- 172.21.15.160:0/1852452827 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x55d4e33b6260 0x55d4e33b86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.760+0000 7f1f4dffb640 10 client.25229 ms_handle_connect on v2:172.21.15.178:6838/2947354024 2024-03-07T07:42:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.760+0000 7f1f6659c640 1 --2- 172.21.15.160:0/1852452827 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x55d4e33b6260 0x55d4e33b86e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7c757d6c19584ea server_cookie=b872e89c9e425ec6 in_seq=0 out_seq=0 2024-03-07T07:42:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.761+0000 7f1f4dffb640 1 -- 172.21.15.160:0/1852452827 <== mds.0 v2:172.21.15.178:6838/2947354024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d4e3312b00 con 0x55d4e33b6260 2024-03-07T07:42:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.761+0000 7f1f4dffb640 10 client.25229 handle_command_reply: tid=0 2024-03-07T07:42:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.761+0000 7f1f6f9b7740 1 client.25229 shutdown 2024-03-07T07:42:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.761+0000 7f1f6f9b7740 1 -- 172.21.15.160:0/1852452827 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4e33a5d00 msgr2=0x55d4e33b1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:42:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.761+0000 7f1f6f9b7740 1 --2- 172.21.15.160:0/1852452827 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4e33a5d00 0x55d4e33b1a80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1f6000a010 tx=0x7f1f60003cd0 comp rx=0 tx=0).stop 2024-03-07T07:42:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.761+0000 7f1f6f9b7740 1 -- 172.21.15.160:0/1852452827 shutdown_connections 2024-03-07T07:42:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.761+0000 7f1f6f9b7740 1 --2- 172.21.15.160:0/1852452827 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x55d4e33b6260 0x55d4e33b86e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.761+0000 7f1f6f9b7740 1 --2- 172.21.15.160:0/1852452827 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4e33a5d00 0x55d4e33b1a80 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.761+0000 7f1f6f9b7740 1 --2- 172.21.15.160:0/1852452827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4e33a0b90 0x55d4e33aaa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.761+0000 7f1f6f9b7740 1 --2- 172.21.15.160:0/1852452827 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d4e33a01a0 0x55d4e33aa500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.761+0000 7f1f4dffb640 0 client.25229 ms_handle_reset on v2:172.21.15.178:6838/2947354024 2024-03-07T07:42:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.761+0000 7f1f6f9b7740 1 -- 172.21.15.160:0/1852452827 >> 172.21.15.160:0/1852452827 conn(0x55d4e3368580 msgr2=0x55d4e339ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:42:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.761+0000 7f1f6f9b7740 1 -- 172.21.15.160:0/1852452827 shutdown_connections 2024-03-07T07:42:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.761+0000 7f1f6f9b7740 1 -- 172.21.15.160:0/1852452827 wait complete. 2024-03-07T07:42:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.761+0000 7f1f6f9b7740 20 client.25229 trim_cache size 0 max 16384 2024-03-07T07:42:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.775+0000 7f1f6f9b7740 1 Processor -- start 2024-03-07T07:42:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.775+0000 7f1f6f9b7740 1 -- start start 2024-03-07T07:42:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.775+0000 7f1f6f9b7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4e33a01a0 0x55d4e339ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.775+0000 7f1f6f9b7740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d4e33a0b90 0x55d4e339f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.775+0000 7f1f6f9b7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4e33a5d00 0x55d4e33a91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.775+0000 7f1f6f9b7740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f681aaf70 con 0x55d4e33a0b90 2024-03-07T07:42:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.775+0000 7f1f6f9b7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f681a8a70 con 0x55d4e33a01a0 2024-03-07T07:42:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.775+0000 7f1f6f9b7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f681a6af0 con 0x55d4e33a5d00 2024-03-07T07:42:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.775+0000 7f1f65d9b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d4e33a0b90 0x55d4e339f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.775+0000 7f1f6659c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4e33a01a0 0x55d4e339ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.775+0000 7f1f66d9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4e33a5d00 0x55d4e33a91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.775+0000 7f1f6659c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4e33a01a0 0x55d4e339ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.160:40944/0 (socket says 172.21.15.160:40944) 2024-03-07T07:42:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.775+0000 7f1f66d9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4e33a5d00 0x55d4e33a91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.160:43636/0 (socket says 172.21.15.160:43636) 2024-03-07T07:42:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.775+0000 7f1f65d9b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d4e33a0b90 0x55d4e339f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41082/0 (socket says 172.21.15.160:41082) 2024-03-07T07:42:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.775+0000 7f1f6659c640 1 -- 172.21.15.160:0/4143825651 learned_addr learned my addr 172.21.15.160:0/4143825651 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:42:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.776+0000 7f1f6659c640 1 -- 172.21.15.160:0/4143825651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4e33a5d00 msgr2=0x55d4e33a91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:42:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.776+0000 7f1f6659c640 1 --2- 172.21.15.160:0/4143825651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4e33a5d00 0x55d4e33a91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.776+0000 7f1f6659c640 1 -- 172.21.15.160:0/4143825651 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d4e33a0b90 msgr2=0x55d4e339f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:42:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.776+0000 7f1f6659c640 1 --2- 172.21.15.160:0/4143825651 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d4e33a0b90 0x55d4e339f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.776+0000 7f1f6659c640 1 -- 172.21.15.160:0/4143825651 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4e33a8b90 con 0x55d4e33a01a0 2024-03-07T07:42:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.776+0000 7f1f6659c640 1 --2- 172.21.15.160:0/4143825651 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4e33a01a0 0x55d4e339ed50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1f580ed320 tx=0x7f1f580dc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15d8cc8c41575e2d server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:42:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.776+0000 7f1f4ffff640 1 -- 172.21.15.160:0/4143825651 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f580d4070 con 0x55d4e33a01a0 2024-03-07T07:42:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.776+0000 7f1f4ffff640 1 -- 172.21.15.160:0/4143825651 <== mon.2 v2:172.21.15.177:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1f580bc690 con 0x55d4e33a01a0 2024-03-07T07:42:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.776+0000 7f1f4ffff640 1 -- 172.21.15.160:0/4143825651 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5804b040 con 0x55d4e33a01a0 2024-03-07T07:42:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.777+0000 7f1f6f9b7740 1 -- 172.21.15.160:0/4143825651 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4e33a01a0 msgr2=0x55d4e339ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:42:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.777+0000 7f1f6f9b7740 1 --2- 172.21.15.160:0/4143825651 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4e33a01a0 0x55d4e339ed50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1f580ed320 tx=0x7f1f580dc040 comp rx=0 tx=0).stop 2024-03-07T07:42:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.777+0000 7f1f6f9b7740 1 -- 172.21.15.160:0/4143825651 shutdown_connections 2024-03-07T07:42:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.777+0000 7f1f6f9b7740 1 --2- 172.21.15.160:0/4143825651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4e33a5d00 0x55d4e33a91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.777+0000 7f1f6f9b7740 1 --2- 172.21.15.160:0/4143825651 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d4e33a0b90 0x55d4e339f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.777+0000 7f1f6f9b7740 1 --2- 172.21.15.160:0/4143825651 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4e33a01a0 0x55d4e339ed50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.777+0000 7f1f6f9b7740 1 -- 172.21.15.160:0/4143825651 >> 172.21.15.160:0/4143825651 conn(0x55d4e3368580 msgr2=0x55d4e33aa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:42:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.777+0000 7f1f6f9b7740 1 -- 172.21.15.160:0/4143825651 shutdown_connections 2024-03-07T07:42:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.777+0000 7f1f6f9b7740 1 -- 172.21.15.160:0/4143825651 wait complete. 2024-03-07T07:42:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.777+0000 7f1f6f9b7740 1 -- start start 2024-03-07T07:42:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.777+0000 7f1f6f9b7740 1 Processor -- start 2024-03-07T07:42:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.777+0000 7f1f6f9b7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4e33a01a0 0x55d4e340ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.777+0000 7f1f6f9b7740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d4e33a0b90 0x55d4e340d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.777+0000 7f1f6f9b7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4e33a5d00 0x55d4e3414270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.777+0000 7f1f6f9b7740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f681a70f0 con 0x55d4e33a0b90 2024-03-07T07:42:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.777+0000 7f1f6f9b7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f681a8410 con 0x55d4e33a01a0 2024-03-07T07:42:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.777+0000 7f1f6f9b7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f681a7750 con 0x55d4e33a5d00 2024-03-07T07:42:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.777+0000 7f1f6f9b7740 10 client.0 _ll_register_callbacks cb 0x7f1f5c009660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-07T07:42:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.777+0000 7f1f6659c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4e33a01a0 0x55d4e340ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.777+0000 7f1f6659c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4e33a01a0 0x55d4e340ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.160:40948/0 (socket says 172.21.15.160:40948) 2024-03-07T07:42:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.777+0000 7f1f6659c640 1 -- 172.21.15.160:0/2263905520 learned_addr learned my addr 172.21.15.160:0/2263905520 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:42:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.778+0000 7f1f65d9b640 1 --2- 172.21.15.160:0/2263905520 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d4e33a0b90 0x55d4e340d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.778+0000 7f1f66d9d640 1 --2- 172.21.15.160:0/2263905520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4e33a5d00 0x55d4e3414270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.778+0000 7f1f6659c640 1 -- 172.21.15.160:0/2263905520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4e33a5d00 msgr2=0x55d4e3414270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:42:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.778+0000 7f1f6659c640 1 --2- 172.21.15.160:0/2263905520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4e33a5d00 0x55d4e3414270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.778+0000 7f1f6659c640 1 -- 172.21.15.160:0/2263905520 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d4e33a0b90 msgr2=0x55d4e340d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:42:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.778+0000 7f1f6659c640 1 --2- 172.21.15.160:0/2263905520 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d4e33a0b90 0x55d4e340d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.778+0000 7f1f6659c640 1 -- 172.21.15.160:0/2263905520 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d4e3414860 con 0x55d4e33a01a0 2024-03-07T07:42:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.778+0000 7f1f6659c640 1 --2- 172.21.15.160:0/2263905520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4e33a01a0 0x55d4e340ccf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1f580dc010 tx=0x7f1f58009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f2747111abaee5b server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:42:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.778+0000 7f1f4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-03-07T07:42:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.778+0000 7f1f4dffb640 1 -- 172.21.15.160:0/2263905520 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f580d4070 con 0x55d4e33a01a0 2024-03-07T07:42:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.778+0000 7f1f4dffb640 1 -- 172.21.15.160:0/2263905520 <== mon.2 v2:172.21.15.177:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1f580035f0 con 0x55d4e33a01a0 2024-03-07T07:42:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.778+0000 7f1f6f9b7740 1 -- 172.21.15.160:0/2263905520 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1f580bc690 con 0x55d4e33a01a0 2024-03-07T07:42:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.778+0000 7f1f4dffb640 1 -- 172.21.15.160:0/2263905520 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f580c4860 con 0x55d4e33a01a0 2024-03-07T07:42:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.778+0000 7f1f6f9b7740 10 client.25235 fetch_fsmap learned FSMap version 22 2024-03-07T07:42:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.778+0000 7f1f6f9b7740 1 -- 172.21.15.160:0/2263905520 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55d4e3415b70 con 0x55d4e33a01a0 2024-03-07T07:42:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.778+0000 7f1f4dffb640 1 -- 172.21.15.160:0/2263905520 <== mon.2 v2:172.21.15.177:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1f580bc690 con 0x55d4e33a01a0 2024-03-07T07:42:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.779+0000 7f1f4dffb640 1 -- 172.21.15.160:0/2263905520 <== mon.2 v2:172.21.15.177:3300/0 5 ==== fsmap(e 22) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f1f580bb550 con 0x55d4e33a01a0 2024-03-07T07:42:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.779+0000 7f1f6f9b7740 10 client.25235 fetch_fsmap finished waiting for FSMap version 22 2024-03-07T07:42:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.779+0000 7f1f6f9b7740 10 client.25235 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-07T07:42:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.779+0000 7f1f6f9b7740 20 client.25235 populate_metadata read hostname 'smithi160' 2024-03-07T07:42:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.779+0000 7f1f6f9b7740 1 --2- 172.21.15.160:0/2263905520 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x55d4e34164c0 0x55d4e34188e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.779+0000 7f1f6f9b7740 4 client.25235 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-07T07:42:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.779+0000 7f1f6f9b7740 1 -- 172.21.15.160:0/2263905520 --> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d4e330ce50 con 0x55d4e34164c0 2024-03-07T07:42:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.779+0000 7f1f65d9b640 1 --2- 172.21.15.160:0/2263905520 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x55d4e34164c0 0x55d4e34188e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.779+0000 7f1f4dffb640 10 client.25235 ms_handle_connect on v2:172.21.15.178:6838/2947354024 2024-03-07T07:42:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.779+0000 7f1f65d9b640 1 --2- 172.21.15.160:0/2263905520 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x55d4e34164c0 0x55d4e34188e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=464ff23d3c128b5a server_cookie=429384e4bba7677 in_seq=0 out_seq=0 2024-03-07T07:42:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.789+0000 7f1f4dffb640 1 -- 172.21.15.160:0/2263905520 <== mds.0 v2:172.21.15.178:6838/2947354024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d4e330ce50 con 0x55d4e34164c0 2024-03-07T07:42:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.789+0000 7f1f4dffb640 10 client.25235 handle_command_reply: tid=0 2024-03-07T07:42:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.790+0000 7f1f6f9b7740 1 client.25235 shutdown 2024-03-07T07:42:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.790+0000 7f1f6f9b7740 1 -- 172.21.15.160:0/2263905520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4e33a01a0 msgr2=0x55d4e340ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:42:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.790+0000 7f1f6f9b7740 1 --2- 172.21.15.160:0/2263905520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4e33a01a0 0x55d4e340ccf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1f580dc010 tx=0x7f1f58009680 comp rx=0 tx=0).stop 2024-03-07T07:42:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.790+0000 7f1f6f9b7740 1 -- 172.21.15.160:0/2263905520 shutdown_connections 2024-03-07T07:42:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.790+0000 7f1f6f9b7740 1 --2- 172.21.15.160:0/2263905520 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x55d4e34164c0 0x55d4e34188e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.790+0000 7f1f6f9b7740 1 --2- 172.21.15.160:0/2263905520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4e33a5d00 0x55d4e3414270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.790+0000 7f1f6f9b7740 1 --2- 172.21.15.160:0/2263905520 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d4e33a0b90 0x55d4e340d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.790+0000 7f1f6f9b7740 1 --2- 172.21.15.160:0/2263905520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4e33a01a0 0x55d4e340ccf0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.790+0000 7f1f4dffb640 0 client.25235 ms_handle_reset on v2:172.21.15.178:6838/2947354024 2024-03-07T07:42:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.790+0000 7f1f6f9b7740 1 -- 172.21.15.160:0/2263905520 >> 172.21.15.160:0/2263905520 conn(0x55d4e3368580 msgr2=0x55d4e339ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:42:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.790+0000 7f1f6f9b7740 1 -- 172.21.15.160:0/2263905520 shutdown_connections 2024-03-07T07:42:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.790+0000 7f1f6f9b7740 1 -- 172.21.15.160:0/2263905520 wait complete. 2024-03-07T07:42:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.790+0000 7f1f6f9b7740 20 client.25235 trim_cache size 0 max 16384 2024-03-07T07:42:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stdout:{ 2024-03-07T07:42:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stdout: "message": "", 2024-03-07T07:42:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stdout: "return_code": 0 2024-03-07T07:42:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stdout:} 2024-03-07T07:42:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.790+0000 7f1f6cbd2640 1 -- 172.21.15.160:0/3578963852 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f1f500a1e20 msgr2=0x7f1f500a42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:42:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.790+0000 7f1f6cbd2640 1 --2- 172.21.15.160:0/3578963852 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f1f500a1e20 0x7f1f500a42e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1f68198e00 tx=0x7f1f5c005f90 comp rx=0 tx=0).stop 2024-03-07T07:42:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.790+0000 7f1f6cbd2640 1 -- 172.21.15.160:0/3578963852 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f680ff6f0 msgr2=0x7f1f681978f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:42:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.790+0000 7f1f6cbd2640 1 --2- 172.21.15.160:0/3578963852 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f680ff6f0 0x7f1f681978f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1f58000c00 tx=0x7f1f5802c640 comp rx=0 tx=0).stop 2024-03-07T07:42:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.790+0000 7f1f6cbd2640 1 -- 172.21.15.160:0/3578963852 shutdown_connections 2024-03-07T07:42:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.790+0000 7f1f6cbd2640 1 --2- 172.21.15.160:0/3578963852 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f1f500a1e20 0x7f1f500a42e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.790+0000 7f1f6cbd2640 1 --2- 172.21.15.160:0/3578963852 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f68102c80 0x7f1f6819eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.790+0000 7f1f6cbd2640 1 --2- 172.21.15.160:0/3578963852 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f681000b0 0x7f1f68197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.790+0000 7f1f6cbd2640 1 --2- 172.21.15.160:0/3578963852 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f680ff6f0 0x7f1f681978f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.790+0000 7f1f6cbd2640 1 -- 172.21.15.160:0/3578963852 >> 172.21.15.160:0/3578963852 conn(0x7f1f680fb2b0 msgr2=0x7f1f68105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:42:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.791+0000 7f1f6cbd2640 1 -- 172.21.15.160:0/3578963852 shutdown_connections 2024-03-07T07:42:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:43.791+0000 7f1f6cbd2640 1 -- 172.21.15.160:0/3578963852 wait complete. 2024-03-07T07:42:43.805 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:42:43 smithi178 ceph-mon[36650]: pgmap v394: 97 pgs: 97 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 779 KiB/s wr, 12 op/s 2024-03-07T07:42:43.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:42:43 smithi177 ceph-mon[36578]: pgmap v394: 97 pgs: 97 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 779 KiB/s wr, 12 op/s 2024-03-07T07:42:45.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:42:45 smithi160 ceph-mon[31963]: pgmap v395: 97 pgs: 97 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.2 MiB/s wr, 13 op/s 2024-03-07T07:42:45.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:42:45 smithi178 ceph-mon[36650]: pgmap v395: 97 pgs: 97 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.2 MiB/s wr, 13 op/s 2024-03-07T07:42:45.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:42:45 smithi177 ceph-mon[36578]: pgmap v395: 97 pgs: 97 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.2 MiB/s wr, 13 op/s 2024-03-07T07:42:47.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:42:47 smithi160 ceph-mon[31963]: pgmap v396: 97 pgs: 97 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 868 KiB/s wr, 14 op/s 2024-03-07T07:42:47.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:42:47 smithi178 ceph-mon[36650]: pgmap v396: 97 pgs: 97 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 868 KiB/s wr, 14 op/s 2024-03-07T07:42:47.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:42:47 smithi177 ceph-mon[36578]: pgmap v396: 97 pgs: 97 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 868 KiB/s wr, 14 op/s 2024-03-07T07:42:49.659 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:42:49 smithi177 ceph-mon[36578]: pgmap v397: 97 pgs: 97 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 861 KiB/s wr, 10 op/s 2024-03-07T07:42:49.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:42:49 smithi160 ceph-mon[31963]: pgmap v397: 97 pgs: 97 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 861 KiB/s wr, 10 op/s 2024-03-07T07:42:49.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:42:49 smithi178 ceph-mon[36650]: pgmap v397: 97 pgs: 97 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 861 KiB/s wr, 10 op/s 2024-03-07T07:42:51.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:42:51 smithi160 ceph-mon[31963]: pgmap v398: 97 pgs: 97 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.3 MiB/s wr, 14 op/s 2024-03-07T07:42:51.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:42:51 smithi178 ceph-mon[36650]: pgmap v398: 97 pgs: 97 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.3 MiB/s wr, 14 op/s 2024-03-07T07:42:51.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:42:51 smithi177 ceph-mon[36578]: pgmap v398: 97 pgs: 97 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.3 MiB/s wr, 14 op/s 2024-03-07T07:42:53.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:42:53 smithi160 ceph-mon[31963]: pgmap v399: 97 pgs: 97 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 861 KiB/s wr, 10 op/s 2024-03-07T07:42:53.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:42:53 smithi178 ceph-mon[36650]: pgmap v399: 97 pgs: 97 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 861 KiB/s wr, 10 op/s 2024-03-07T07:42:53.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:42:53 smithi177 ceph-mon[36578]: pgmap v399: 97 pgs: 97 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 861 KiB/s wr, 10 op/s 2024-03-07T07:42:55.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:42:55 smithi160 ceph-mon[31963]: pgmap v400: 97 pgs: 97 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 1.3 MiB/s wr, 11 op/s 2024-03-07T07:42:55.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:42:55 smithi178 ceph-mon[36650]: pgmap v400: 97 pgs: 97 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 1.3 MiB/s wr, 11 op/s 2024-03-07T07:42:55.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:42:55 smithi177 ceph-mon[36578]: pgmap v400: 97 pgs: 97 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 1.3 MiB/s wr, 11 op/s 2024-03-07T07:42:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.884+0000 7f4efefb8640 1 -- 172.21.15.160:0/3232323353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ef81016c0 msgr2=0x7f4ef8101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:42:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.884+0000 7f4efefb8640 1 --2- 172.21.15.160:0/3232323353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ef81016c0 0x7f4ef8101ae0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4eec009860 tx=0x7f4eec02bb70 comp rx=0 tx=0).stop 2024-03-07T07:42:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.884+0000 7f4efefb8640 1 -- 172.21.15.160:0/3232323353 shutdown_connections 2024-03-07T07:42:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.884+0000 7f4efefb8640 1 --2- 172.21.15.160:0/3232323353 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ef8103270 0x7f4ef8109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.884+0000 7f4efefb8640 1 --2- 172.21.15.160:0/3232323353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ef81028b0 0x7f4ef8102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.884+0000 7f4efefb8640 1 --2- 172.21.15.160:0/3232323353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ef81016c0 0x7f4ef8101ae0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.884+0000 7f4efefb8640 1 -- 172.21.15.160:0/3232323353 >> 172.21.15.160:0/3232323353 conn(0x7f4ef80fcc30 msgr2=0x7f4ef80ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:42:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.884+0000 7f4efefb8640 1 -- 172.21.15.160:0/3232323353 shutdown_connections 2024-03-07T07:42:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.885+0000 7f4efefb8640 1 -- 172.21.15.160:0/3232323353 wait complete. 2024-03-07T07:42:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.885+0000 7f4efefb8640 1 Processor -- start 2024-03-07T07:42:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.885+0000 7f4efefb8640 1 -- start start 2024-03-07T07:42:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.885+0000 7f4efefb8640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ef81016c0 0x7f4ef8199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.885+0000 7f4efefb8640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ef81028b0 0x7f4ef819a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.885+0000 7f4efefb8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ef8103270 0x7f4ef81a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.885+0000 7f4efefb8640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef810d340 con 0x7f4ef81016c0 2024-03-07T07:42:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.885+0000 7f4efefb8640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef810d1c0 con 0x7f4ef81028b0 2024-03-07T07:42:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.885+0000 7f4efefb8640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef810d4c0 con 0x7f4ef8103270 2024-03-07T07:42:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.885+0000 7f4efcd54640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ef81016c0 0x7f4ef8199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.885+0000 7f4efcd54640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ef81016c0 0x7f4ef8199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41608/0 (socket says 172.21.15.160:41608) 2024-03-07T07:42:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.885+0000 7f4efcd54640 1 -- 172.21.15.160:0/1105591713 learned_addr learned my addr 172.21.15.160:0/1105591713 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:42:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.886+0000 7f4efd555640 1 --2- 172.21.15.160:0/1105591713 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ef8103270 0x7f4ef81a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.886+0000 7f4ef7fff640 1 --2- 172.21.15.160:0/1105591713 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ef81028b0 0x7f4ef819a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.886+0000 7f4efcd54640 1 -- 172.21.15.160:0/1105591713 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ef8103270 msgr2=0x7f4ef81a10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:42:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.886+0000 7f4efcd54640 1 --2- 172.21.15.160:0/1105591713 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ef8103270 0x7f4ef81a10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.886+0000 7f4efcd54640 1 -- 172.21.15.160:0/1105591713 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ef81028b0 msgr2=0x7f4ef819a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:42:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.886+0000 7f4efcd54640 1 --2- 172.21.15.160:0/1105591713 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ef81028b0 0x7f4ef819a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.886+0000 7f4efcd54640 1 -- 172.21.15.160:0/1105591713 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef81a1760 con 0x7f4ef81016c0 2024-03-07T07:42:56.890 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.886+0000 7f4ef7fff640 1 --2- 172.21.15.160:0/1105591713 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ef81028b0 0x7f4ef819a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:42:56.890 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.886+0000 7f4efd555640 1 --2- 172.21.15.160:0/1105591713 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ef8103270 0x7f4ef81a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:42:56.890 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.886+0000 7f4efcd54640 1 --2- 172.21.15.160:0/1105591713 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ef81016c0 0x7f4ef8199b40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4eec0349e0 tx=0x7f4eec032430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:42:56.890 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.886+0000 7f4ef5ffb640 1 -- 172.21.15.160:0/1105591713 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eec041020 con 0x7f4ef81016c0 2024-03-07T07:42:56.890 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.886+0000 7f4ef5ffb640 1 -- 172.21.15.160:0/1105591713 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4eec02c9f0 con 0x7f4ef81016c0 2024-03-07T07:42:56.890 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.886+0000 7f4ef5ffb640 1 -- 172.21.15.160:0/1105591713 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eec039040 con 0x7f4ef81016c0 2024-03-07T07:42:56.890 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.886+0000 7f4efefb8640 1 -- 172.21.15.160:0/1105591713 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ef81a19f0 con 0x7f4ef81016c0 2024-03-07T07:42:56.890 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.886+0000 7f4efefb8640 1 -- 172.21.15.160:0/1105591713 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ef81a1ee0 con 0x7f4ef81016c0 2024-03-07T07:42:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.889+0000 7f4ef5ffb640 1 -- 172.21.15.160:0/1105591713 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 20) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f4eec044020 con 0x7f4ef81016c0 2024-03-07T07:42:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.889+0000 7f4ef5ffb640 1 --2- 172.21.15.160:0/1105591713 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f4ee80a1e20 0x7f4ee80a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.889+0000 7f4ef7fff640 1 --2- 172.21.15.160:0/1105591713 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f4ee80a1e20 0x7f4ee80a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.889+0000 7f4ef5ffb640 1 -- 172.21.15.160:0/1105591713 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f4eec0ef5e0 con 0x7f4ef81016c0 2024-03-07T07:42:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.890+0000 7f4ef7fff640 1 --2- 172.21.15.160:0/1105591713 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f4ee80a1e20 0x7f4ee80a42e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4ef819b050 tx=0x7f4ee4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:42:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.890+0000 7f4f01da1740 1 Processor -- start 2024-03-07T07:42:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.890+0000 7f4f01da1740 1 -- start start 2024-03-07T07:42:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.890+0000 7f4f01da1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558799a0d1a0 0x558799a0d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.890+0000 7f4f01da1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558799a0db90 0x558799a12690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.890+0000 7f4f01da1740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558799a12d00 0x558799a15190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.890+0000 7f4f01da1740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587999725c0 con 0x558799a12d00 2024-03-07T07:42:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.890+0000 7f4f01da1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55879991b900 con 0x558799a0d1a0 2024-03-07T07:42:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.890+0000 7f4f01da1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55879996e660 con 0x558799a0db90 2024-03-07T07:42:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.890+0000 7f4efd555640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558799a12d00 0x558799a15190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.890+0000 7f4efd555640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558799a12d00 0x558799a15190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41622/0 (socket says 172.21.15.160:41622) 2024-03-07T07:42:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.890+0000 7f4efd555640 1 -- 172.21.15.160:0/1305282107 learned_addr learned my addr 172.21.15.160:0/1305282107 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:42:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.890+0000 7f4efcd54640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558799a0d1a0 0x558799a0d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.890+0000 7f4ef7fff640 1 --2- 172.21.15.160:0/1305282107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558799a0db90 0x558799a12690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.891+0000 7f4efcd54640 1 -- 172.21.15.160:0/1305282107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558799a0db90 msgr2=0x558799a12690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:42:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.891+0000 7f4efcd54640 1 --2- 172.21.15.160:0/1305282107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558799a0db90 0x558799a12690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.891+0000 7f4efcd54640 1 -- 172.21.15.160:0/1305282107 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558799a12d00 msgr2=0x558799a15190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:42:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.891+0000 7f4efcd54640 1 --2- 172.21.15.160:0/1305282107 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558799a12d00 0x558799a15190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.891+0000 7f4efcd54640 1 -- 172.21.15.160:0/1305282107 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558799687210 con 0x558799a0d1a0 2024-03-07T07:42:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.891+0000 7f4efcd54640 1 --2- 172.21.15.160:0/1305282107 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558799a0d1a0 0x558799a0d5c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4eec02c180 tx=0x7f4eec0045f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=409755d11caa0a44 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:42:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.891+0000 7f4ee2ffd640 1 -- 172.21.15.160:0/1305282107 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eec0bc310 con 0x558799a0d1a0 2024-03-07T07:42:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.891+0000 7f4ee2ffd640 1 -- 172.21.15.160:0/1305282107 <== mon.2 v2:172.21.15.177:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4eec0c0460 con 0x558799a0d1a0 2024-03-07T07:42:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.891+0000 7f4ee2ffd640 1 -- 172.21.15.160:0/1305282107 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eec0b8980 con 0x558799a0d1a0 2024-03-07T07:42:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.892+0000 7f4f01da1740 1 -- 172.21.15.160:0/1305282107 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558799a0d1a0 msgr2=0x558799a0d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:42:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.892+0000 7f4f01da1740 1 --2- 172.21.15.160:0/1305282107 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558799a0d1a0 0x558799a0d5c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4eec02c180 tx=0x7f4eec0045f0 comp rx=0 tx=0).stop 2024-03-07T07:42:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.892+0000 7f4f01da1740 1 -- 172.21.15.160:0/1305282107 shutdown_connections 2024-03-07T07:42:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.892+0000 7f4f01da1740 1 --2- 172.21.15.160:0/1305282107 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558799a12d00 0x558799a15190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.892+0000 7f4f01da1740 1 --2- 172.21.15.160:0/1305282107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558799a0db90 0x558799a12690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.892+0000 7f4f01da1740 1 --2- 172.21.15.160:0/1305282107 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558799a0d1a0 0x558799a0d5c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.892+0000 7f4f01da1740 1 -- 172.21.15.160:0/1305282107 >> 172.21.15.160:0/1305282107 conn(0x5587999d5580 msgr2=0x558799a08b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:42:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.892+0000 7f4f01da1740 1 -- 172.21.15.160:0/1305282107 shutdown_connections 2024-03-07T07:42:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.892+0000 7f4f01da1740 1 -- 172.21.15.160:0/1305282107 wait complete. 2024-03-07T07:42:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.892+0000 7f4f01da1740 1 -- start start 2024-03-07T07:42:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.892+0000 7f4f01da1740 1 Processor -- start 2024-03-07T07:42:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.893+0000 7f4f01da1740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558799a0d1a0 0x558799a170b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.893+0000 7f4f01da1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558799a0db90 0x558799a175f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.893+0000 7f4f01da1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558799a12d00 0x558799a1e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.893+0000 7f4f01da1740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55879991b900 con 0x558799a0d1a0 2024-03-07T07:42:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.893+0000 7f4f01da1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55879992ed60 con 0x558799a12d00 2024-03-07T07:42:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.893+0000 7f4f01da1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x558799a15a70 con 0x558799a0db90 2024-03-07T07:42:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.893+0000 7f4f01da1740 10 client.0 _ll_register_callbacks cb 0x558799686a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-07T07:42:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.893+0000 7f4efcd54640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558799a0d1a0 0x558799a170b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.893+0000 7f4efd555640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558799a12d00 0x558799a1e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.893+0000 7f4ef7fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558799a0db90 0x558799a175f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.893+0000 7f4efcd54640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558799a0d1a0 0x558799a170b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41638/0 (socket says 172.21.15.160:41638) 2024-03-07T07:42:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.893+0000 7f4efd555640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558799a12d00 0x558799a1e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.160:60442/0 (socket says 172.21.15.160:60442) 2024-03-07T07:42:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.893+0000 7f4efcd54640 1 -- 172.21.15.160:0/1462580261 learned_addr learned my addr 172.21.15.160:0/1462580261 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:42:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.893+0000 7f4ef7fff640 1 --2- 172.21.15.160:0/1462580261 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558799a0db90 0x558799a175f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.160:38696/0 (socket says 172.21.15.160:38696) 2024-03-07T07:42:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.893+0000 7f4efcd54640 1 -- 172.21.15.160:0/1462580261 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558799a0db90 msgr2=0x558799a175f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:42:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.893+0000 7f4efcd54640 1 --2- 172.21.15.160:0/1462580261 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558799a0db90 0x558799a175f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.893+0000 7f4efcd54640 1 -- 172.21.15.160:0/1462580261 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558799a12d00 msgr2=0x558799a1e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:42:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.893+0000 7f4efcd54640 1 --2- 172.21.15.160:0/1462580261 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558799a12d00 0x558799a1e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.893+0000 7f4efcd54640 1 -- 172.21.15.160:0/1462580261 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558799753620 con 0x558799a0d1a0 2024-03-07T07:42:56.902 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.893+0000 7f4ef7fff640 1 --2- 172.21.15.160:0/1462580261 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558799a0db90 0x558799a175f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:42:56.902 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.893+0000 7f4efd555640 1 --2- 172.21.15.160:0/1462580261 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558799a12d00 0x558799a1e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:42:56.902 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.893+0000 7f4efcd54640 1 --2- 172.21.15.160:0/1462580261 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558799a0d1a0 0x558799a170b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4eec0bc2a0 tx=0x7f4eec0c4ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a674b193d4b87ac server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:42:56.902 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.893+0000 7f4ed7fff640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-03-07T07:42:56.902 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.893+0000 7f4ed7fff640 1 -- 172.21.15.160:0/1462580261 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eec0b8980 con 0x558799a0d1a0 2024-03-07T07:42:56.902 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.893+0000 7f4ed7fff640 1 -- 172.21.15.160:0/1462580261 <== mon.0 v2:172.21.15.160:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f4eec032640 con 0x558799a0d1a0 2024-03-07T07:42:56.902 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.893+0000 7f4f01da1740 1 -- 172.21.15.160:0/1462580261 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4eec0c0460 con 0x558799a0d1a0 2024-03-07T07:42:56.902 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.894+0000 7f4ed7fff640 1 -- 172.21.15.160:0/1462580261 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4eec0b8b20 con 0x558799a0d1a0 2024-03-07T07:42:56.902 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.894+0000 7f4f01da1740 10 client.15411 fetch_fsmap learned FSMap version 22 2024-03-07T07:42:56.902 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.894+0000 7f4f01da1740 1 -- 172.21.15.160:0/1462580261 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x558799a219a0 con 0x558799a0d1a0 2024-03-07T07:42:56.903 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.894+0000 7f4ed7fff640 1 -- 172.21.15.160:0/1462580261 <== mon.0 v2:172.21.15.160:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4eec0c0460 con 0x558799a0d1a0 2024-03-07T07:42:56.903 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.894+0000 7f4ed7fff640 1 -- 172.21.15.160:0/1462580261 <== mon.0 v2:172.21.15.160:3300/0 5 ==== fsmap(e 22) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f4eec09f020 con 0x558799a0d1a0 2024-03-07T07:42:56.903 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.894+0000 7f4f01da1740 10 client.15411 fetch_fsmap finished waiting for FSMap version 22 2024-03-07T07:42:56.903 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.894+0000 7f4f01da1740 10 client.15411 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-07T07:42:56.903 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.894+0000 7f4f01da1740 20 client.15411 populate_metadata read hostname 'smithi160' 2024-03-07T07:42:56.903 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.894+0000 7f4f01da1740 1 --2- 172.21.15.160:0/1462580261 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x558799a23030 0x558799a25420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:56.903 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.894+0000 7f4f01da1740 4 client.15411 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-07T07:42:56.903 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.894+0000 7f4f01da1740 1 -- 172.21.15.160:0/1462580261 --> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55879997fb00 con 0x558799a23030 2024-03-07T07:42:56.903 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.894+0000 7f4ef7fff640 1 --2- 172.21.15.160:0/1462580261 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x558799a23030 0x558799a25420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:56.903 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.895+0000 7f4ed7fff640 10 client.15411 ms_handle_connect on v2:172.21.15.178:6838/2947354024 2024-03-07T07:42:56.903 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.895+0000 7f4ef7fff640 1 --2- 172.21.15.160:0/1462580261 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x558799a23030 0x558799a25420 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29f1727dad61ff19 server_cookie=10bdaa1fe4e1e013 in_seq=0 out_seq=0 2024-03-07T07:42:56.903 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.895+0000 7f4ed7fff640 1 -- 172.21.15.160:0/1462580261 <== mds.0 v2:172.21.15.178:6838/2947354024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55879997fb00 con 0x558799a23030 2024-03-07T07:42:56.903 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.895+0000 7f4ed7fff640 10 client.15411 handle_command_reply: tid=0 2024-03-07T07:42:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.895+0000 7f4f01da1740 1 client.15411 shutdown 2024-03-07T07:42:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.895+0000 7f4f01da1740 1 -- 172.21.15.160:0/1462580261 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558799a0d1a0 msgr2=0x558799a170b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:42:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.895+0000 7f4f01da1740 1 --2- 172.21.15.160:0/1462580261 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558799a0d1a0 0x558799a170b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4eec0bc2a0 tx=0x7f4eec0c4ba0 comp rx=0 tx=0).stop 2024-03-07T07:42:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.896+0000 7f4f01da1740 1 -- 172.21.15.160:0/1462580261 shutdown_connections 2024-03-07T07:42:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.896+0000 7f4f01da1740 1 --2- 172.21.15.160:0/1462580261 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x558799a23030 0x558799a25420 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.896+0000 7f4f01da1740 1 --2- 172.21.15.160:0/1462580261 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558799a12d00 0x558799a1e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.896+0000 7f4ed7fff640 0 client.15411 ms_handle_reset on v2:172.21.15.178:6838/2947354024 2024-03-07T07:42:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.896+0000 7f4f01da1740 1 --2- 172.21.15.160:0/1462580261 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558799a0db90 0x558799a175f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.896+0000 7f4f01da1740 1 --2- 172.21.15.160:0/1462580261 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558799a0d1a0 0x558799a170b0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.896+0000 7f4f01da1740 1 -- 172.21.15.160:0/1462580261 >> 172.21.15.160:0/1462580261 conn(0x5587999d5580 msgr2=0x558799a07e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:42:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.896+0000 7f4f01da1740 1 -- 172.21.15.160:0/1462580261 shutdown_connections 2024-03-07T07:42:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.896+0000 7f4f01da1740 1 -- 172.21.15.160:0/1462580261 wait complete. 2024-03-07T07:42:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.896+0000 7f4f01da1740 20 client.15411 trim_cache size 0 max 16384 2024-03-07T07:42:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.910+0000 7f4f01da1740 1 Processor -- start 2024-03-07T07:42:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.910+0000 7f4f01da1740 1 -- start start 2024-03-07T07:42:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.910+0000 7f4f01da1740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558799a0d1a0 0x558799a16cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.910+0000 7f4f01da1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558799a0db90 0x558799a1e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.910+0000 7f4f01da1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558799a12d00 0x558799a17200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.910+0000 7f4f01da1740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef81ad1a0 con 0x558799a0d1a0 2024-03-07T07:42:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.910+0000 7f4f01da1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef81aaca0 con 0x558799a12d00 2024-03-07T07:42:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.910+0000 7f4f01da1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef81a8d20 con 0x558799a0db90 2024-03-07T07:42:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.910+0000 7f4ef7fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558799a0db90 0x558799a1e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.910+0000 7f4efcd54640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558799a0d1a0 0x558799a16cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.910+0000 7f4efd555640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558799a12d00 0x558799a17200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.910+0000 7f4efd555640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558799a12d00 0x558799a17200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.160:60446/0 (socket says 172.21.15.160:60446) 2024-03-07T07:42:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.910+0000 7f4efcd54640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558799a0d1a0 0x558799a16cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41648/0 (socket says 172.21.15.160:41648) 2024-03-07T07:42:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.910+0000 7f4ef7fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558799a0db90 0x558799a1e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.160:38704/0 (socket says 172.21.15.160:38704) 2024-03-07T07:42:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.910+0000 7f4efd555640 1 -- 172.21.15.160:0/1702841951 learned_addr learned my addr 172.21.15.160:0/1702841951 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:42:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.910+0000 7f4efd555640 1 -- 172.21.15.160:0/1702841951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558799a0db90 msgr2=0x558799a1e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:42:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.910+0000 7f4efd555640 1 --2- 172.21.15.160:0/1702841951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558799a0db90 0x558799a1e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.910+0000 7f4efd555640 1 -- 172.21.15.160:0/1702841951 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558799a0d1a0 msgr2=0x558799a16cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:42:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.910+0000 7f4efd555640 1 --2- 172.21.15.160:0/1702841951 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558799a0d1a0 0x558799a16cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.910+0000 7f4efd555640 1 -- 172.21.15.160:0/1702841951 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558799a25790 con 0x558799a12d00 2024-03-07T07:42:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.910+0000 7f4efd555640 1 --2- 172.21.15.160:0/1702841951 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558799a12d00 0x558799a17200 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4ef000ebe0 tx=0x7f4ef00095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b67a4e8b55aad2e server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:42:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.911+0000 7f4ee27fc640 1 -- 172.21.15.160:0/1702841951 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0004b20 con 0x558799a12d00 2024-03-07T07:42:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.911+0000 7f4ee27fc640 1 -- 172.21.15.160:0/1702841951 <== mon.2 v2:172.21.15.177:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ef0003e50 con 0x558799a12d00 2024-03-07T07:42:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.911+0000 7f4ee27fc640 1 -- 172.21.15.160:0/1702841951 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0005020 con 0x558799a12d00 2024-03-07T07:42:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.912+0000 7f4f01da1740 1 -- 172.21.15.160:0/1702841951 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558799a12d00 msgr2=0x558799a17200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:42:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.912+0000 7f4f01da1740 1 --2- 172.21.15.160:0/1702841951 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558799a12d00 0x558799a17200 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4ef000ebe0 tx=0x7f4ef00095c0 comp rx=0 tx=0).stop 2024-03-07T07:42:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.912+0000 7f4f01da1740 1 -- 172.21.15.160:0/1702841951 shutdown_connections 2024-03-07T07:42:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.912+0000 7f4f01da1740 1 --2- 172.21.15.160:0/1702841951 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558799a12d00 0x558799a17200 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.912+0000 7f4f01da1740 1 --2- 172.21.15.160:0/1702841951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558799a0db90 0x558799a1e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.912+0000 7f4f01da1740 1 --2- 172.21.15.160:0/1702841951 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558799a0d1a0 0x558799a16cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.912+0000 7f4f01da1740 1 -- 172.21.15.160:0/1702841951 >> 172.21.15.160:0/1702841951 conn(0x5587999d5580 msgr2=0x558799a0bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:42:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.912+0000 7f4f01da1740 1 -- 172.21.15.160:0/1702841951 shutdown_connections 2024-03-07T07:42:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.912+0000 7f4f01da1740 1 -- 172.21.15.160:0/1702841951 wait complete. 2024-03-07T07:42:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.912+0000 7f4f01da1740 1 -- start start 2024-03-07T07:42:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.912+0000 7f4f01da1740 1 Processor -- start 2024-03-07T07:42:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.913+0000 7f4f01da1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558799a0d1a0 0x558799a1bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.913+0000 7f4f01da1740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558799a0db90 0x558799a1c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.913+0000 7f4f01da1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558799a12d00 0x558799a1d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.913+0000 7f4f01da1740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef81a9320 con 0x558799a0db90 2024-03-07T07:42:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.913+0000 7f4f01da1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef81aa640 con 0x558799a0d1a0 2024-03-07T07:42:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.913+0000 7f4f01da1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef81a9980 con 0x558799a12d00 2024-03-07T07:42:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.913+0000 7f4ef7fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558799a0db90 0x558799a1c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.913+0000 7f4f01da1740 10 client.0 _ll_register_callbacks cb 0x7f4ef000e450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-07T07:42:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.913+0000 7f4ef7fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558799a0db90 0x558799a1c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41664/0 (socket says 172.21.15.160:41664) 2024-03-07T07:42:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.913+0000 7f4ef7fff640 1 -- 172.21.15.160:0/3647488427 learned_addr learned my addr 172.21.15.160:0/3647488427 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:42:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.913+0000 7f4efd555640 1 --2- 172.21.15.160:0/3647488427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558799a12d00 0x558799a1d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.913+0000 7f4efcd54640 1 --2- 172.21.15.160:0/3647488427 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558799a0d1a0 0x558799a1bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.913+0000 7f4ef7fff640 1 -- 172.21.15.160:0/3647488427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558799a12d00 msgr2=0x558799a1d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:42:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.913+0000 7f4ef7fff640 1 --2- 172.21.15.160:0/3647488427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558799a12d00 0x558799a1d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.913+0000 7f4ef7fff640 1 -- 172.21.15.160:0/3647488427 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558799a0d1a0 msgr2=0x558799a1bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:42:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.913+0000 7f4ef7fff640 1 --2- 172.21.15.160:0/3647488427 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558799a0d1a0 0x558799a1bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.913+0000 7f4ef7fff640 1 -- 172.21.15.160:0/3647488427 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558799a1d710 con 0x558799a0db90 2024-03-07T07:42:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.913+0000 7f4efcd54640 1 --2- 172.21.15.160:0/3647488427 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558799a0d1a0 0x558799a1bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:42:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.913+0000 7f4efd555640 1 --2- 172.21.15.160:0/3647488427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558799a12d00 0x558799a1d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:42:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.913+0000 7f4ef7fff640 1 --2- 172.21.15.160:0/3647488427 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558799a0db90 0x558799a1c110 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4009e80 tx=0x7f4ee4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97c799600330dcfb server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:42:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.913+0000 7f4ed7fff640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-03-07T07:42:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.914+0000 7f4ed7fff640 1 -- 172.21.15.160:0/3647488427 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee402c0b0 con 0x558799a0db90 2024-03-07T07:42:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.914+0000 7f4ed7fff640 1 -- 172.21.15.160:0/3647488427 <== mon.0 v2:172.21.15.160:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f4ee402c250 con 0x558799a0db90 2024-03-07T07:42:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.914+0000 7f4f01da1740 1 -- 172.21.15.160:0/3647488427 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ef0003e50 con 0x558799a0db90 2024-03-07T07:42:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.914+0000 7f4ed7fff640 1 -- 172.21.15.160:0/3647488427 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ee400f480 con 0x558799a0db90 2024-03-07T07:42:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.914+0000 7f4f01da1740 10 client.15417 fetch_fsmap learned FSMap version 22 2024-03-07T07:42:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.914+0000 7f4f01da1740 1 -- 172.21.15.160:0/3647488427 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x558799a7e8c0 con 0x558799a0db90 2024-03-07T07:42:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.914+0000 7f4ed7fff640 1 -- 172.21.15.160:0/3647488427 <== mon.0 v2:172.21.15.160:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ef0003e50 con 0x558799a0db90 2024-03-07T07:42:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.914+0000 7f4ed7fff640 1 -- 172.21.15.160:0/3647488427 <== mon.0 v2:172.21.15.160:3300/0 5 ==== fsmap(e 22) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f4ee401a040 con 0x558799a0db90 2024-03-07T07:42:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.914+0000 7f4f01da1740 10 client.15417 fetch_fsmap finished waiting for FSMap version 22 2024-03-07T07:42:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.914+0000 7f4f01da1740 10 client.15417 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-07T07:42:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.914+0000 7f4f01da1740 20 client.15417 populate_metadata read hostname 'smithi160' 2024-03-07T07:42:56.920 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.914+0000 7f4f01da1740 1 --2- 172.21.15.160:0/3647488427 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x558799a7f1a0 0x558799a815c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:42:56.920 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.914+0000 7f4f01da1740 4 client.15417 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-07T07:42:56.920 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.914+0000 7f4f01da1740 1 -- 172.21.15.160:0/3647488427 --> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558799979e50 con 0x558799a7f1a0 2024-03-07T07:42:56.920 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.914+0000 7f4efcd54640 1 --2- 172.21.15.160:0/3647488427 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x558799a7f1a0 0x558799a815c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:42:56.920 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.915+0000 7f4ed7fff640 10 client.15417 ms_handle_connect on v2:172.21.15.178:6838/2947354024 2024-03-07T07:42:56.920 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.915+0000 7f4efcd54640 1 --2- 172.21.15.160:0/3647488427 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x558799a7f1a0 0x558799a815c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faa79645365ad3d6 server_cookie=501162fb0aae6648 in_seq=0 out_seq=0 2024-03-07T07:42:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.923+0000 7f4ed7fff640 1 -- 172.21.15.160:0/3647488427 <== mds.0 v2:172.21.15.178:6838/2947354024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558799979e50 con 0x558799a7f1a0 2024-03-07T07:42:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.923+0000 7f4ed7fff640 10 client.15417 handle_command_reply: tid=0 2024-03-07T07:42:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.923+0000 7f4f01da1740 1 client.15417 shutdown 2024-03-07T07:42:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.923+0000 7f4f01da1740 1 -- 172.21.15.160:0/3647488427 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558799a0db90 msgr2=0x558799a1c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:42:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.923+0000 7f4f01da1740 1 --2- 172.21.15.160:0/3647488427 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558799a0db90 0x558799a1c110 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4009e80 tx=0x7f4ee4004730 comp rx=0 tx=0).stop 2024-03-07T07:42:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.923+0000 7f4f01da1740 1 -- 172.21.15.160:0/3647488427 shutdown_connections 2024-03-07T07:42:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.924+0000 7f4f01da1740 1 --2- 172.21.15.160:0/3647488427 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x558799a7f1a0 0x558799a815c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.924+0000 7f4f01da1740 1 --2- 172.21.15.160:0/3647488427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558799a12d00 0x558799a1d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.924+0000 7f4ed7fff640 0 client.15417 ms_handle_reset on v2:172.21.15.178:6838/2947354024 2024-03-07T07:42:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.924+0000 7f4f01da1740 1 --2- 172.21.15.160:0/3647488427 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558799a0db90 0x558799a1c110 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.924+0000 7f4f01da1740 1 --2- 172.21.15.160:0/3647488427 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558799a0d1a0 0x558799a1bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.924+0000 7f4f01da1740 1 -- 172.21.15.160:0/3647488427 >> 172.21.15.160:0/3647488427 conn(0x5587999d5580 msgr2=0x558799a16cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:42:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.924+0000 7f4f01da1740 1 -- 172.21.15.160:0/3647488427 shutdown_connections 2024-03-07T07:42:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.924+0000 7f4f01da1740 1 -- 172.21.15.160:0/3647488427 wait complete. 2024-03-07T07:42:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.924+0000 7f4f01da1740 20 client.15417 trim_cache size 0 max 16384 2024-03-07T07:42:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stdout:{ 2024-03-07T07:42:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stdout: "message": "", 2024-03-07T07:42:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stdout: "return_code": 0 2024-03-07T07:42:56.926 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stdout:} 2024-03-07T07:42:56.926 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.924+0000 7f4efefb8640 1 -- 172.21.15.160:0/1105591713 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f4ee80a1e20 msgr2=0x7f4ee80a42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:42:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.924+0000 7f4efefb8640 1 --2- 172.21.15.160:0/1105591713 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f4ee80a1e20 0x7f4ee80a42e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4ef819b050 tx=0x7f4ee4005f90 comp rx=0 tx=0).stop 2024-03-07T07:42:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.924+0000 7f4efefb8640 1 -- 172.21.15.160:0/1105591713 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ef81016c0 msgr2=0x7f4ef8199b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:42:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.924+0000 7f4efefb8640 1 --2- 172.21.15.160:0/1105591713 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ef81016c0 0x7f4ef8199b40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4eec0349e0 tx=0x7f4eec032430 comp rx=0 tx=0).stop 2024-03-07T07:42:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.924+0000 7f4efefb8640 1 -- 172.21.15.160:0/1105591713 shutdown_connections 2024-03-07T07:42:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.924+0000 7f4efefb8640 1 --2- 172.21.15.160:0/1105591713 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f4ee80a1e20 0x7f4ee80a42e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.924+0000 7f4efefb8640 1 --2- 172.21.15.160:0/1105591713 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ef8103270 0x7f4ef81a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.924+0000 7f4efefb8640 1 --2- 172.21.15.160:0/1105591713 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ef81028b0 0x7f4ef819a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.924+0000 7f4efefb8640 1 --2- 172.21.15.160:0/1105591713 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ef81016c0 0x7f4ef8199b40 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:42:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.924+0000 7f4efefb8640 1 -- 172.21.15.160:0/1105591713 >> 172.21.15.160:0/1105591713 conn(0x7f4ef80fcc30 msgr2=0x7f4ef8073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:42:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.925+0000 7f4efefb8640 1 -- 172.21.15.160:0/1105591713 shutdown_connections 2024-03-07T07:42:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:42:56.925+0000 7f4efefb8640 1 -- 172.21.15.160:0/1105591713 wait complete. 2024-03-07T07:42:57.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:42:57 smithi160 ceph-mon[31963]: pgmap v401: 97 pgs: 97 active+clean; 1.9 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 864 KiB/s wr, 12 op/s 2024-03-07T07:42:57.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:42:57 smithi178 ceph-mon[36650]: pgmap v401: 97 pgs: 97 active+clean; 1.9 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 864 KiB/s wr, 12 op/s 2024-03-07T07:42:57.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:42:57 smithi177 ceph-mon[36578]: pgmap v401: 97 pgs: 97 active+clean; 1.9 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 864 KiB/s wr, 12 op/s 2024-03-07T07:42:59.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:42:59 smithi160 ceph-mon[31963]: pgmap v402: 97 pgs: 97 active+clean; 1.9 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 860 KiB/s wr, 8 op/s 2024-03-07T07:42:59.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:42:59 smithi178 ceph-mon[36650]: pgmap v402: 97 pgs: 97 active+clean; 1.9 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 860 KiB/s wr, 8 op/s 2024-03-07T07:42:59.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:42:59 smithi177 ceph-mon[36578]: pgmap v402: 97 pgs: 97 active+clean; 1.9 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 860 KiB/s wr, 8 op/s 2024-03-07T07:43:00.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:43:00 smithi160 ceph-mon[31963]: pgmap v403: 97 pgs: 97 active+clean; 1.9 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.3 MiB/s wr, 14 op/s 2024-03-07T07:43:00.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:43:00 smithi178 ceph-mon[36650]: pgmap v403: 97 pgs: 97 active+clean; 1.9 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.3 MiB/s wr, 14 op/s 2024-03-07T07:43:00.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:43:00 smithi177 ceph-mon[36578]: pgmap v403: 97 pgs: 97 active+clean; 1.9 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.3 MiB/s wr, 14 op/s 2024-03-07T07:43:03.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:43:03 smithi160 ceph-mon[31963]: pgmap v404: 97 pgs: 97 active+clean; 54 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.2 MiB/s wr, 10 op/s 2024-03-07T07:43:03.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:43:03 smithi178 ceph-mon[36650]: pgmap v404: 97 pgs: 97 active+clean; 54 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.2 MiB/s wr, 10 op/s 2024-03-07T07:43:03.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:43:03 smithi177 ceph-mon[36578]: pgmap v404: 97 pgs: 97 active+clean; 54 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.2 MiB/s wr, 10 op/s 2024-03-07T07:43:05.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:43:05 smithi160 ceph-mon[31963]: pgmap v405: 97 pgs: 97 active+clean; 158 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 14 MiB/s wr, 14 op/s 2024-03-07T07:43:05.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:43:05 smithi178 ceph-mon[36650]: pgmap v405: 97 pgs: 97 active+clean; 158 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 14 MiB/s wr, 14 op/s 2024-03-07T07:43:05.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:43:05 smithi177 ceph-mon[36578]: pgmap v405: 97 pgs: 97 active+clean; 158 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 14 MiB/s wr, 14 op/s 2024-03-07T07:43:07.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:43:07 smithi160 ceph-mon[31963]: pgmap v406: 97 pgs: 97 active+clean; 182 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 16 op/s 2024-03-07T07:43:07.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:43:07 smithi178 ceph-mon[36650]: pgmap v406: 97 pgs: 97 active+clean; 182 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 16 op/s 2024-03-07T07:43:07.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:43:07 smithi177 ceph-mon[36578]: pgmap v406: 97 pgs: 97 active+clean; 182 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 16 op/s 2024-03-07T07:43:09.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:43:09 smithi160 ceph-mon[31963]: pgmap v407: 97 pgs: 97 active+clean; 214 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 22 MiB/s wr, 15 op/s 2024-03-07T07:43:09.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:43:09 smithi178 ceph-mon[36650]: pgmap v407: 97 pgs: 97 active+clean; 214 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 22 MiB/s wr, 15 op/s 2024-03-07T07:43:09.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:43:09 smithi177 ceph-mon[36578]: pgmap v407: 97 pgs: 97 active+clean; 214 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 22 MiB/s wr, 15 op/s 2024-03-07T07:43:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.020+0000 7f9445816640 1 -- 172.21.15.160:0/541233614 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9440109540 msgr2=0x7f944010b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:43:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.020+0000 7f9445816640 1 --2- 172.21.15.160:0/541233614 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9440109540 0x7f944010b9e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9438009ca0 tx=0x7f943802c180 comp rx=0 tx=0).stop 2024-03-07T07:43:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.020+0000 7f9445816640 1 -- 172.21.15.160:0/541233614 shutdown_connections 2024-03-07T07:43:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.020+0000 7f9445816640 1 --2- 172.21.15.160:0/541233614 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9440109540 0x7f944010b9e0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.020+0000 7f9445816640 1 --2- 172.21.15.160:0/541233614 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9440074060 0x7f94400744c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.020+0000 7f9445816640 1 --2- 172.21.15.160:0/541233614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9440075640 0x7f9440073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.020+0000 7f9445816640 1 -- 172.21.15.160:0/541233614 >> 172.21.15.160:0/541233614 conn(0x7f94400fcc20 msgr2=0x7f94400ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:43:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.020+0000 7f9445816640 1 -- 172.21.15.160:0/541233614 shutdown_connections 2024-03-07T07:43:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.021+0000 7f9445816640 1 -- 172.21.15.160:0/541233614 wait complete. 2024-03-07T07:43:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.021+0000 7f9445816640 1 Processor -- start 2024-03-07T07:43:10.024 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.022+0000 7f9445816640 1 -- start start 2024-03-07T07:43:10.024 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.022+0000 7f9445816640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9440074060 0x7f9440199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:10.024 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.022+0000 7f9445816640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9440075640 0x7f944019a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.022+0000 7f9445816640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9440109540 0x7f94401a10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.022+0000 7f9445816640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944010f530 con 0x7f9440075640 2024-03-07T07:43:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.022+0000 7f9445816640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944010f3b0 con 0x7f9440074060 2024-03-07T07:43:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.022+0000 7f9445816640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944010f6b0 con 0x7f9440109540 2024-03-07T07:43:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.022+0000 7f943e7fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9440075640 0x7f944019a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.022+0000 7f943e7fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9440075640 0x7f944019a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44326/0 (socket says 172.21.15.160:44326) 2024-03-07T07:43:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.022+0000 7f943effd640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9440074060 0x7f9440199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.022+0000 7f943e7fc640 1 -- 172.21.15.160:0/2051217144 learned_addr learned my addr 172.21.15.160:0/2051217144 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:43:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.022+0000 7f943f7fe640 1 --2- 172.21.15.160:0/2051217144 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9440109540 0x7f94401a10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.022+0000 7f943effd640 1 -- 172.21.15.160:0/2051217144 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9440109540 msgr2=0x7f94401a10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:43:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.022+0000 7f943effd640 1 --2- 172.21.15.160:0/2051217144 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9440109540 0x7f94401a10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.022+0000 7f943effd640 1 -- 172.21.15.160:0/2051217144 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9440075640 msgr2=0x7f944019a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:43:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.022+0000 7f943effd640 1 --2- 172.21.15.160:0/2051217144 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9440075640 0x7f944019a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.022+0000 7f943effd640 1 -- 172.21.15.160:0/2051217144 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94401a1720 con 0x7f9440074060 2024-03-07T07:43:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.022+0000 7f943e7fc640 1 --2- 172.21.15.160:0/2051217144 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9440075640 0x7f944019a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:43:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.022+0000 7f943effd640 1 --2- 172.21.15.160:0/2051217144 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9440074060 0x7f9440199b00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f943400e9f0 tx=0x7f943400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:43:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.023+0000 7f9444814640 1 -- 172.21.15.160:0/2051217144 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943400ccc0 con 0x7f9440074060 2024-03-07T07:43:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.023+0000 7f9444814640 1 -- 172.21.15.160:0/2051217144 <== mon.2 v2:172.21.15.177:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9434005020 con 0x7f9440074060 2024-03-07T07:43:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.023+0000 7f9445816640 1 -- 172.21.15.160:0/2051217144 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94401a1a10 con 0x7f9440074060 2024-03-07T07:43:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.023+0000 7f9444814640 1 -- 172.21.15.160:0/2051217144 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94340049e0 con 0x7f9440074060 2024-03-07T07:43:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.023+0000 7f9445816640 1 -- 172.21.15.160:0/2051217144 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94401a1f10 con 0x7f9440074060 2024-03-07T07:43:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.025+0000 7f9444814640 1 -- 172.21.15.160:0/2051217144 <== mon.2 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 20) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f9434005560 con 0x7f9440074060 2024-03-07T07:43:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.026+0000 7f94485fe740 1 Processor -- start 2024-03-07T07:43:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.026+0000 7f94485fe740 1 -- start start 2024-03-07T07:43:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.026+0000 7f94485fe740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c92ad381a0 0x55c92ad385c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.026+0000 7f9444814640 1 --2- 172.21.15.160:0/2051217144 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f94280a1e20 0x7f94280a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.026+0000 7f94485fe740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c92ad38b90 0x55c92ad3d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.026+0000 7f94485fe740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c92ad3dd00 0x55c92ad40190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.026+0000 7f94485fe740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c92ac9d5c0 con 0x55c92ad3dd00 2024-03-07T07:43:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.026+0000 7f94485fe740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c92ac46900 con 0x55c92ad381a0 2024-03-07T07:43:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.026+0000 7f94485fe740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c92ac99660 con 0x55c92ad38b90 2024-03-07T07:43:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.026+0000 7f943f7fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c92ad3dd00 0x55c92ad40190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.026+0000 7f943effd640 1 --2- 172.21.15.160:0/2051217144 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f94280a1e20 0x7f94280a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.026+0000 7f943f7fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c92ad3dd00 0x55c92ad40190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44332/0 (socket says 172.21.15.160:44332) 2024-03-07T07:43:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.026+0000 7f943f7fe640 1 -- 172.21.15.160:0/1676656224 learned_addr learned my addr 172.21.15.160:0/1676656224 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:43:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.026+0000 7f943e7fc640 1 --2- 172.21.15.160:0/1676656224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c92ad38b90 0x55c92ad3d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.026+0000 7f943f7fe640 1 --2- 172.21.15.160:0/1676656224 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c92ad381a0 0x55c92ad385c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.026+0000 7f9444814640 1 -- 172.21.15.160:0/2051217144 <== mon.2 v2:172.21.15.177:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f94340cdd70 con 0x7f9440074060 2024-03-07T07:43:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.027+0000 7f943e7fc640 1 -- 172.21.15.160:0/1676656224 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c92ad381a0 msgr2=0x55c92ad385c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:43:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.027+0000 7f943e7fc640 1 --2- 172.21.15.160:0/1676656224 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c92ad381a0 0x55c92ad385c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.027+0000 7f943e7fc640 1 -- 172.21.15.160:0/1676656224 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c92ad3dd00 msgr2=0x55c92ad40190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:43:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.027+0000 7f943e7fc640 1 --2- 172.21.15.160:0/1676656224 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c92ad3dd00 0x55c92ad40190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.027+0000 7f943e7fc640 1 -- 172.21.15.160:0/1676656224 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c92a9b2210 con 0x55c92ad38b90 2024-03-07T07:43:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.027+0000 7f943f7fe640 1 --2- 172.21.15.160:0/1676656224 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c92ad381a0 0x55c92ad385c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:43:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.027+0000 7f943effd640 1 --2- 172.21.15.160:0/2051217144 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f94280a1e20 0x7f94280a42e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f943409a7a0 tx=0x7f9434096e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:43:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.027+0000 7f943e7fc640 1 --2- 172.21.15.160:0/1676656224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c92ad38b90 0x55c92ad3d690 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9430009fd0 tx=0x7f94300119a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa2cf303315b839d server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:43:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.027+0000 7f9425ffb640 1 -- 172.21.15.160:0/1676656224 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9430005020 con 0x55c92ad38b90 2024-03-07T07:43:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.027+0000 7f9425ffb640 1 -- 172.21.15.160:0/1676656224 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f94300047a0 con 0x55c92ad38b90 2024-03-07T07:43:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.027+0000 7f9425ffb640 1 -- 172.21.15.160:0/1676656224 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943000f920 con 0x55c92ad38b90 2024-03-07T07:43:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.028+0000 7f94485fe740 1 -- 172.21.15.160:0/1676656224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c92ad38b90 msgr2=0x55c92ad3d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:43:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.028+0000 7f94485fe740 1 --2- 172.21.15.160:0/1676656224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c92ad38b90 0x55c92ad3d690 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9430009fd0 tx=0x7f94300119a0 comp rx=0 tx=0).stop 2024-03-07T07:43:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.028+0000 7f94485fe740 1 -- 172.21.15.160:0/1676656224 shutdown_connections 2024-03-07T07:43:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.028+0000 7f94485fe740 1 --2- 172.21.15.160:0/1676656224 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c92ad3dd00 0x55c92ad40190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.028+0000 7f94485fe740 1 --2- 172.21.15.160:0/1676656224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c92ad38b90 0x55c92ad3d690 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.028+0000 7f94485fe740 1 --2- 172.21.15.160:0/1676656224 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c92ad381a0 0x55c92ad385c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.028+0000 7f94485fe740 1 -- 172.21.15.160:0/1676656224 >> 172.21.15.160:0/1676656224 conn(0x55c92ad00580 msgr2=0x55c92ad33b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:43:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.028+0000 7f94485fe740 1 -- 172.21.15.160:0/1676656224 shutdown_connections 2024-03-07T07:43:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.028+0000 7f94485fe740 1 -- 172.21.15.160:0/1676656224 wait complete. 2024-03-07T07:43:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.028+0000 7f94485fe740 1 -- start start 2024-03-07T07:43:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.029+0000 7f94485fe740 1 Processor -- start 2024-03-07T07:43:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.029+0000 7f94485fe740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c92ad381a0 0x55c92ad424e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.029+0000 7f94485fe740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c92ad38b90 0x55c92ad42a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.029+0000 7f94485fe740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c92ad3dd00 0x55c92ad49a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.029+0000 7f94485fe740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c92ad40cb0 con 0x55c92ad3dd00 2024-03-07T07:43:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.029+0000 7f94485fe740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c92ac59d60 con 0x55c92ad38b90 2024-03-07T07:43:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.029+0000 7f94485fe740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c92ad40e30 con 0x55c92ad381a0 2024-03-07T07:43:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.029+0000 7f94485fe740 10 client.0 _ll_register_callbacks cb 0x55c92a9b1a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-07T07:43:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.029+0000 7f943f7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c92ad381a0 0x55c92ad424e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.029+0000 7f943f7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c92ad381a0 0x55c92ad424e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.160:36314/0 (socket says 172.21.15.160:36314) 2024-03-07T07:43:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.029+0000 7f943f7fe640 1 -- 172.21.15.160:0/1647878655 learned_addr learned my addr 172.21.15.160:0/1647878655 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:43:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.029+0000 7f943e7fc640 1 --2- 172.21.15.160:0/1647878655 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c92ad38b90 0x55c92ad42a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.029+0000 7f943f7fe640 1 --2- 172.21.15.160:0/1647878655 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c92ad3dd00 0x55c92ad49a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.029+0000 7f943f7fe640 1 -- 172.21.15.160:0/1647878655 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c92ad38b90 msgr2=0x55c92ad42a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:43:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.029+0000 7f943f7fe640 1 --2- 172.21.15.160:0/1647878655 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c92ad38b90 0x55c92ad42a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.029+0000 7f943f7fe640 1 -- 172.21.15.160:0/1647878655 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c92ad3dd00 msgr2=0x55c92ad49a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:43:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.029+0000 7f943f7fe640 1 --2- 172.21.15.160:0/1647878655 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c92ad3dd00 0x55c92ad49a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.029+0000 7f943f7fe640 1 -- 172.21.15.160:0/1647878655 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c92aa7e620 con 0x55c92ad381a0 2024-03-07T07:43:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.029+0000 7f943e7fc640 1 --2- 172.21.15.160:0/1647878655 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c92ad38b90 0x55c92ad42a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:43:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.030+0000 7f943f7fe640 1 --2- 172.21.15.160:0/1647878655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c92ad381a0 0x55c92ad424e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f943802c790 tx=0x7f943802c180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4289962f23a0c16f server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:43:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.030+0000 7f941effd640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-03-07T07:43:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.030+0000 7f941effd640 1 -- 172.21.15.160:0/1647878655 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9438039040 con 0x55c92ad381a0 2024-03-07T07:43:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.030+0000 7f941effd640 1 -- 172.21.15.160:0/1647878655 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9438032b40 con 0x55c92ad381a0 2024-03-07T07:43:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.030+0000 7f94485fe740 1 -- 172.21.15.160:0/1647878655 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f94300047a0 con 0x55c92ad381a0 2024-03-07T07:43:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.030+0000 7f941effd640 1 -- 172.21.15.160:0/1647878655 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9438032d40 con 0x55c92ad381a0 2024-03-07T07:43:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.030+0000 7f94485fe740 10 client.25303 fetch_fsmap learned FSMap version 22 2024-03-07T07:43:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.030+0000 7f94485fe740 1 -- 172.21.15.160:0/1647878655 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55c92ad4ce40 con 0x55c92ad381a0 2024-03-07T07:43:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.030+0000 7f941effd640 1 -- 172.21.15.160:0/1647878655 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f94300047a0 con 0x55c92ad381a0 2024-03-07T07:43:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.030+0000 7f941effd640 1 -- 172.21.15.160:0/1647878655 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 22) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f943803f070 con 0x55c92ad381a0 2024-03-07T07:43:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.030+0000 7f94485fe740 10 client.25303 fetch_fsmap finished waiting for FSMap version 22 2024-03-07T07:43:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.030+0000 7f94485fe740 10 client.25303 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-07T07:43:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.030+0000 7f94485fe740 20 client.25303 populate_metadata read hostname 'smithi160' 2024-03-07T07:43:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.030+0000 7f94485fe740 1 --2- 172.21.15.160:0/1647878655 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x55c92ad4e470 0x55c92ad50880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.030+0000 7f94485fe740 4 client.25303 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-07T07:43:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.030+0000 7f94485fe740 1 -- 172.21.15.160:0/1647878655 --> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c92acaab00 con 0x55c92ad4e470 2024-03-07T07:43:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.030+0000 7f943e7fc640 1 --2- 172.21.15.160:0/1647878655 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x55c92ad4e470 0x55c92ad50880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.031+0000 7f941effd640 10 client.25303 ms_handle_connect on v2:172.21.15.178:6838/2947354024 2024-03-07T07:43:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.031+0000 7f943e7fc640 1 --2- 172.21.15.160:0/1647878655 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x55c92ad4e470 0x55c92ad50880 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=157ec59b15c3c781 server_cookie=ec99d3a69bc92675 in_seq=0 out_seq=0 2024-03-07T07:43:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.032+0000 7f941effd640 1 -- 172.21.15.160:0/1647878655 <== mds.0 v2:172.21.15.178:6838/2947354024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c92acaab00 con 0x55c92ad4e470 2024-03-07T07:43:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.032+0000 7f941effd640 10 client.25303 handle_command_reply: tid=0 2024-03-07T07:43:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.032+0000 7f94485fe740 1 client.25303 shutdown 2024-03-07T07:43:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.032+0000 7f94485fe740 1 -- 172.21.15.160:0/1647878655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c92ad381a0 msgr2=0x55c92ad424e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:43:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.032+0000 7f94485fe740 1 --2- 172.21.15.160:0/1647878655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c92ad381a0 0x55c92ad424e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f943802c790 tx=0x7f943802c180 comp rx=0 tx=0).stop 2024-03-07T07:43:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.032+0000 7f94485fe740 1 -- 172.21.15.160:0/1647878655 shutdown_connections 2024-03-07T07:43:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.032+0000 7f94485fe740 1 --2- 172.21.15.160:0/1647878655 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x55c92ad4e470 0x55c92ad50880 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.032+0000 7f94485fe740 1 --2- 172.21.15.160:0/1647878655 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c92ad3dd00 0x55c92ad49a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.032+0000 7f941effd640 0 client.25303 ms_handle_reset on v2:172.21.15.178:6838/2947354024 2024-03-07T07:43:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.032+0000 7f94485fe740 1 --2- 172.21.15.160:0/1647878655 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c92ad38b90 0x55c92ad42a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.032+0000 7f94485fe740 1 --2- 172.21.15.160:0/1647878655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c92ad381a0 0x55c92ad424e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.032+0000 7f94485fe740 1 -- 172.21.15.160:0/1647878655 >> 172.21.15.160:0/1647878655 conn(0x55c92ad00580 msgr2=0x55c92ad32e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:43:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.032+0000 7f94485fe740 1 -- 172.21.15.160:0/1647878655 shutdown_connections 2024-03-07T07:43:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.032+0000 7f94485fe740 1 -- 172.21.15.160:0/1647878655 wait complete. 2024-03-07T07:43:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.032+0000 7f94485fe740 20 client.25303 trim_cache size 0 max 16384 2024-03-07T07:43:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.046+0000 7f94485fe740 1 Processor -- start 2024-03-07T07:43:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.046+0000 7f94485fe740 1 -- start start 2024-03-07T07:43:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.046+0000 7f94485fe740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c92ad381a0 0x55c92ad42980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.046+0000 7f94485fe740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c92ad38b90 0x55c92ad499f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.046+0000 7f94485fe740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c92ad3dd00 0x55c92ad32580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.046+0000 7f94485fe740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94401ad0d0 con 0x55c92ad381a0 2024-03-07T07:43:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.046+0000 7f94485fe740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94401aabd0 con 0x55c92ad3dd00 2024-03-07T07:43:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.046+0000 7f94485fe740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94401a8c50 con 0x55c92ad38b90 2024-03-07T07:43:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.046+0000 7f943f7fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c92ad381a0 0x55c92ad42980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.046+0000 7f943e7fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c92ad38b90 0x55c92ad499f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.046+0000 7f943e7fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c92ad38b90 0x55c92ad499f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.160:36318/0 (socket says 172.21.15.160:36318) 2024-03-07T07:43:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.046+0000 7f943f7fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c92ad381a0 0x55c92ad42980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44364/0 (socket says 172.21.15.160:44364) 2024-03-07T07:43:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.046+0000 7f943e7fc640 1 -- 172.21.15.160:0/1059868618 learned_addr learned my addr 172.21.15.160:0/1059868618 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:43:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.046+0000 7f943f7fe640 1 --2- 172.21.15.160:0/1059868618 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c92ad3dd00 0x55c92ad32580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.047+0000 7f943e7fc640 1 -- 172.21.15.160:0/1059868618 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c92ad3dd00 msgr2=0x55c92ad32580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:43:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.047+0000 7f943e7fc640 1 --2- 172.21.15.160:0/1059868618 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c92ad3dd00 0x55c92ad32580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.047+0000 7f943e7fc640 1 -- 172.21.15.160:0/1059868618 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c92ad381a0 msgr2=0x55c92ad42980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:43:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.047+0000 7f943e7fc640 1 --2- 172.21.15.160:0/1059868618 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c92ad381a0 0x55c92ad42980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.047+0000 7f943e7fc640 1 -- 172.21.15.160:0/1059868618 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c92ad37db0 con 0x55c92ad38b90 2024-03-07T07:43:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.047+0000 7f943e7fc640 1 --2- 172.21.15.160:0/1059868618 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c92ad38b90 0x55c92ad499f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9430005fd0 tx=0x7f9430009550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=266d44a105b4e441 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:43:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.047+0000 7f94257fa640 1 -- 172.21.15.160:0/1059868618 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9430011020 con 0x55c92ad38b90 2024-03-07T07:43:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.047+0000 7f94257fa640 1 -- 172.21.15.160:0/1059868618 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f94300111c0 con 0x55c92ad38b90 2024-03-07T07:43:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.047+0000 7f94257fa640 1 -- 172.21.15.160:0/1059868618 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9430004660 con 0x55c92ad38b90 2024-03-07T07:43:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.048+0000 7f94485fe740 1 -- 172.21.15.160:0/1059868618 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c92ad38b90 msgr2=0x55c92ad499f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:43:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.048+0000 7f94485fe740 1 --2- 172.21.15.160:0/1059868618 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c92ad38b90 0x55c92ad499f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9430005fd0 tx=0x7f9430009550 comp rx=0 tx=0).stop 2024-03-07T07:43:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.048+0000 7f94485fe740 1 -- 172.21.15.160:0/1059868618 shutdown_connections 2024-03-07T07:43:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.048+0000 7f94485fe740 1 --2- 172.21.15.160:0/1059868618 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c92ad3dd00 0x55c92ad32580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.048+0000 7f94485fe740 1 --2- 172.21.15.160:0/1059868618 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c92ad38b90 0x55c92ad499f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.048+0000 7f94485fe740 1 --2- 172.21.15.160:0/1059868618 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c92ad381a0 0x55c92ad42980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.048+0000 7f94485fe740 1 -- 172.21.15.160:0/1059868618 >> 172.21.15.160:0/1059868618 conn(0x55c92ad00580 msgr2=0x55c92ad373c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:43:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.048+0000 7f94485fe740 1 -- 172.21.15.160:0/1059868618 shutdown_connections 2024-03-07T07:43:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.048+0000 7f94485fe740 1 -- 172.21.15.160:0/1059868618 wait complete. 2024-03-07T07:43:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.048+0000 7f94485fe740 1 -- start start 2024-03-07T07:43:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.048+0000 7f94485fe740 1 Processor -- start 2024-03-07T07:43:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.048+0000 7f94485fe740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c92ad381a0 0x55c92ad47660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.048+0000 7f94485fe740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c92ad38b90 0x55c92ad47ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.048+0000 7f94485fe740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c92ad3dd00 0x55c92ad48bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.048+0000 7f94485fe740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94401a9250 con 0x55c92ad3dd00 2024-03-07T07:43:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.048+0000 7f94485fe740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94401aa570 con 0x55c92ad381a0 2024-03-07T07:43:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.048+0000 7f94485fe740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94401a98b0 con 0x55c92ad38b90 2024-03-07T07:43:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.049+0000 7f94485fe740 10 client.0 _ll_register_callbacks cb 0x7f9438009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-07T07:43:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.049+0000 7f943f7fe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c92ad381a0 0x55c92ad47660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.049+0000 7f943e7fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c92ad38b90 0x55c92ad47ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.049+0000 7f943f7fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c92ad3dd00 0x55c92ad48bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.049+0000 7f943e7fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c92ad38b90 0x55c92ad47ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.160:36326/0 (socket says 172.21.15.160:36326) 2024-03-07T07:43:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.049+0000 7f943e7fc640 1 -- 172.21.15.160:0/3803782125 learned_addr learned my addr 172.21.15.160:0/3803782125 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:43:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.049+0000 7f943f7fe640 1 --2- 172.21.15.160:0/3803782125 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c92ad3dd00 0x55c92ad48bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44374/0 (socket says 172.21.15.160:44374) 2024-03-07T07:43:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.049+0000 7f943e7fc640 1 -- 172.21.15.160:0/3803782125 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c92ad381a0 msgr2=0x55c92ad47660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:43:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.049+0000 7f943e7fc640 1 --2- 172.21.15.160:0/3803782125 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c92ad381a0 0x55c92ad47660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.049+0000 7f943e7fc640 1 -- 172.21.15.160:0/3803782125 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c92ad3dd00 msgr2=0x55c92ad48bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:43:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.049+0000 7f943e7fc640 1 --2- 172.21.15.160:0/3803782125 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c92ad3dd00 0x55c92ad48bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.049+0000 7f943e7fc640 1 -- 172.21.15.160:0/3803782125 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c92ad491a0 con 0x55c92ad38b90 2024-03-07T07:43:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.049+0000 7f943e7fc640 1 --2- 172.21.15.160:0/3803782125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c92ad38b90 0x55c92ad47ba0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9430009fd0 tx=0x7f9430005050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae05885ba262fca4 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:43:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.049+0000 7f941effd640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-03-07T07:43:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.049+0000 7f941effd640 1 -- 172.21.15.160:0/3803782125 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9430005480 con 0x55c92ad38b90 2024-03-07T07:43:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.049+0000 7f941effd640 1 -- 172.21.15.160:0/3803782125 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f943000f5c0 con 0x55c92ad38b90 2024-03-07T07:43:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.049+0000 7f94485fe740 1 -- 172.21.15.160:0/3803782125 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f94300111c0 con 0x55c92ad38b90 2024-03-07T07:43:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.050+0000 7f941effd640 1 -- 172.21.15.160:0/3803782125 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9430005620 con 0x55c92ad38b90 2024-03-07T07:43:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.050+0000 7f94485fe740 10 client.25309 fetch_fsmap learned FSMap version 22 2024-03-07T07:43:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.050+0000 7f94485fe740 1 -- 172.21.15.160:0/3803782125 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55c92adaa210 con 0x55c92ad38b90 2024-03-07T07:43:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.050+0000 7f941effd640 1 -- 172.21.15.160:0/3803782125 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f94300111c0 con 0x55c92ad38b90 2024-03-07T07:43:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.050+0000 7f941effd640 1 -- 172.21.15.160:0/3803782125 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 22) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f9430030790 con 0x55c92ad38b90 2024-03-07T07:43:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.050+0000 7f94485fe740 10 client.25309 fetch_fsmap finished waiting for FSMap version 22 2024-03-07T07:43:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.050+0000 7f94485fe740 10 client.25309 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-07T07:43:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.050+0000 7f94485fe740 20 client.25309 populate_metadata read hostname 'smithi160' 2024-03-07T07:43:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.050+0000 7f94485fe740 1 --2- 172.21.15.160:0/3803782125 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x55c92adaaa80 0x55c92adacea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.050+0000 7f94485fe740 4 client.25309 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-07T07:43:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.050+0000 7f94485fe740 1 -- 172.21.15.160:0/3803782125 --> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c92aca4e50 con 0x55c92adaaa80 2024-03-07T07:43:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.050+0000 7f943f7fe640 1 --2- 172.21.15.160:0/3803782125 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x55c92adaaa80 0x55c92adacea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.050+0000 7f941effd640 10 client.25309 ms_handle_connect on v2:172.21.15.178:6838/2947354024 2024-03-07T07:43:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.051+0000 7f943f7fe640 1 --2- 172.21.15.160:0/3803782125 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x55c92adaaa80 0x55c92adacea0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93eb655236e6e49f server_cookie=eb56def37b36d06c in_seq=0 out_seq=0 2024-03-07T07:43:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.058+0000 7f941effd640 1 -- 172.21.15.160:0/3803782125 <== mds.0 v2:172.21.15.178:6838/2947354024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c92aca4e50 con 0x55c92adaaa80 2024-03-07T07:43:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.058+0000 7f941effd640 10 client.25309 handle_command_reply: tid=0 2024-03-07T07:43:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.058+0000 7f94485fe740 1 client.25309 shutdown 2024-03-07T07:43:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.059+0000 7f94485fe740 1 -- 172.21.15.160:0/3803782125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c92ad38b90 msgr2=0x55c92ad47ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:43:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.059+0000 7f94485fe740 1 --2- 172.21.15.160:0/3803782125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c92ad38b90 0x55c92ad47ba0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9430009fd0 tx=0x7f9430005050 comp rx=0 tx=0).stop 2024-03-07T07:43:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.059+0000 7f94485fe740 1 -- 172.21.15.160:0/3803782125 shutdown_connections 2024-03-07T07:43:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.059+0000 7f94485fe740 1 --2- 172.21.15.160:0/3803782125 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x55c92adaaa80 0x55c92adacea0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.059+0000 7f94485fe740 1 --2- 172.21.15.160:0/3803782125 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c92ad3dd00 0x55c92ad48bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.059+0000 7f941effd640 0 client.25309 ms_handle_reset on v2:172.21.15.178:6838/2947354024 2024-03-07T07:43:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.059+0000 7f94485fe740 1 --2- 172.21.15.160:0/3803782125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c92ad38b90 0x55c92ad47ba0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.059+0000 7f94485fe740 1 --2- 172.21.15.160:0/3803782125 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c92ad381a0 0x55c92ad47660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.059+0000 7f94485fe740 1 -- 172.21.15.160:0/3803782125 >> 172.21.15.160:0/3803782125 conn(0x55c92ad00580 msgr2=0x55c92ad373c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:43:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.059+0000 7f94485fe740 1 -- 172.21.15.160:0/3803782125 shutdown_connections 2024-03-07T07:43:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.059+0000 7f94485fe740 1 -- 172.21.15.160:0/3803782125 wait complete. 2024-03-07T07:43:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.059+0000 7f94485fe740 20 client.25309 trim_cache size 0 max 16384 2024-03-07T07:43:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stdout:{ 2024-03-07T07:43:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stdout: "message": "", 2024-03-07T07:43:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stdout: "return_code": 0 2024-03-07T07:43:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stdout:} 2024-03-07T07:43:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.059+0000 7f9445816640 1 -- 172.21.15.160:0/2051217144 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f94280a1e20 msgr2=0x7f94280a42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:43:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.059+0000 7f9445816640 1 --2- 172.21.15.160:0/2051217144 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f94280a1e20 0x7f94280a42e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f943409a7a0 tx=0x7f9434096e70 comp rx=0 tx=0).stop 2024-03-07T07:43:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.059+0000 7f9445816640 1 -- 172.21.15.160:0/2051217144 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9440074060 msgr2=0x7f9440199b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:43:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.059+0000 7f9445816640 1 --2- 172.21.15.160:0/2051217144 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9440074060 0x7f9440199b00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f943400e9f0 tx=0x7f943400eeb0 comp rx=0 tx=0).stop 2024-03-07T07:43:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.060+0000 7f9445816640 1 -- 172.21.15.160:0/2051217144 shutdown_connections 2024-03-07T07:43:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.060+0000 7f9445816640 1 --2- 172.21.15.160:0/2051217144 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f94280a1e20 0x7f94280a42e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.060+0000 7f9445816640 1 --2- 172.21.15.160:0/2051217144 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9440109540 0x7f94401a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.060+0000 7f9445816640 1 --2- 172.21.15.160:0/2051217144 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9440075640 0x7f944019a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.060+0000 7f9445816640 1 --2- 172.21.15.160:0/2051217144 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9440074060 0x7f9440199b00 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.060+0000 7f9445816640 1 -- 172.21.15.160:0/2051217144 >> 172.21.15.160:0/2051217144 conn(0x7f94400fcc20 msgr2=0x7f94401054d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:43:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.060+0000 7f9445816640 1 -- 172.21.15.160:0/2051217144 shutdown_connections 2024-03-07T07:43:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:10.060+0000 7f9445816640 1 -- 172.21.15.160:0/2051217144 wait complete. 2024-03-07T07:43:11.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:43:11 smithi160 ceph-mon[31963]: pgmap v408: 97 pgs: 97 active+clean; 242 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 30 MiB/s wr, 21 op/s 2024-03-07T07:43:11.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:43:11 smithi178 ceph-mon[36650]: pgmap v408: 97 pgs: 97 active+clean; 242 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 30 MiB/s wr, 21 op/s 2024-03-07T07:43:11.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:43:11 smithi177 ceph-mon[36578]: pgmap v408: 97 pgs: 97 active+clean; 242 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 30 MiB/s wr, 21 op/s 2024-03-07T07:43:13.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:43:13 smithi160 ceph-mon[31963]: pgmap v409: 97 pgs: 97 active+clean; 258 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 31 MiB/s wr, 16 op/s 2024-03-07T07:43:13.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:43:13 smithi178 ceph-mon[36650]: pgmap v409: 97 pgs: 97 active+clean; 258 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 31 MiB/s wr, 16 op/s 2024-03-07T07:43:13.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:43:13 smithi177 ceph-mon[36578]: pgmap v409: 97 pgs: 97 active+clean; 258 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 31 MiB/s wr, 16 op/s 2024-03-07T07:43:15.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:43:15 smithi160 ceph-mon[31963]: pgmap v410: 97 pgs: 97 active+clean; 258 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 27 MiB/s wr, 16 op/s 2024-03-07T07:43:15.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:43:15 smithi178 ceph-mon[36650]: pgmap v410: 97 pgs: 97 active+clean; 258 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 27 MiB/s wr, 16 op/s 2024-03-07T07:43:15.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:43:15 smithi177 ceph-mon[36578]: pgmap v410: 97 pgs: 97 active+clean; 258 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 27 MiB/s wr, 16 op/s 2024-03-07T07:43:16.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:43:16 smithi178 ceph-mon[36650]: pgmap v411: 97 pgs: 97 active+clean; 258 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 MiB/s wr, 17 op/s 2024-03-07T07:43:16.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:43:16 smithi177 ceph-mon[36578]: pgmap v411: 97 pgs: 97 active+clean; 258 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 MiB/s wr, 17 op/s 2024-03-07T07:43:16.927 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:43:16 smithi160 ceph-mon[31963]: pgmap v411: 97 pgs: 97 active+clean; 258 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 MiB/s wr, 17 op/s 2024-03-07T07:43:19.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:43:19 smithi160 ceph-mon[31963]: pgmap v412: 97 pgs: 97 active+clean; 258 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-03-07T07:43:19.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:43:19 smithi178 ceph-mon[36650]: pgmap v412: 97 pgs: 97 active+clean; 258 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-03-07T07:43:19.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:43:19 smithi177 ceph-mon[36578]: pgmap v412: 97 pgs: 97 active+clean; 258 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-03-07T07:43:21.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:43:21 smithi160 ceph-mon[31963]: pgmap v413: 97 pgs: 97 active+clean; 258 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 16 op/s 2024-03-07T07:43:21.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:43:21 smithi178 ceph-mon[36650]: pgmap v413: 97 pgs: 97 active+clean; 258 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 16 op/s 2024-03-07T07:43:21.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:43:21 smithi177 ceph-mon[36578]: pgmap v413: 97 pgs: 97 active+clean; 258 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 16 op/s 2024-03-07T07:43:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.152+0000 7f2a98fb1640 1 -- 172.21.15.160:0/1152856179 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a94100580 msgr2=0x7f2a941009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:43:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.152+0000 7f2a98fb1640 1 --2- 172.21.15.160:0/1152856179 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a94100580 0x7f2a941009a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2a84009900 tx=0x7f2a8402bc30 comp rx=0 tx=0).stop 2024-03-07T07:43:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.152+0000 7f2a98fb1640 1 -- 172.21.15.160:0/1152856179 shutdown_connections 2024-03-07T07:43:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.152+0000 7f2a98fb1640 1 --2- 172.21.15.160:0/1152856179 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a94102130 0x7f2a941089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.152+0000 7f2a98fb1640 1 --2- 172.21.15.160:0/1152856179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a94101770 0x7f2a94101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.152+0000 7f2a98fb1640 1 --2- 172.21.15.160:0/1152856179 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a94100580 0x7f2a941009a0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.152+0000 7f2a98fb1640 1 -- 172.21.15.160:0/1152856179 >> 172.21.15.160:0/1152856179 conn(0x7f2a940fbaf0 msgr2=0x7f2a940fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:43:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.153+0000 7f2a98fb1640 1 -- 172.21.15.160:0/1152856179 shutdown_connections 2024-03-07T07:43:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.153+0000 7f2a98fb1640 1 -- 172.21.15.160:0/1152856179 wait complete. 2024-03-07T07:43:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.153+0000 7f2a98fb1640 1 Processor -- start 2024-03-07T07:43:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.153+0000 7f2a98fb1640 1 -- start start 2024-03-07T07:43:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.153+0000 7f2a98fb1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a94100580 0x7f2a94197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.153+0000 7f2a98fb1640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a94101770 0x7f2a94197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.153+0000 7f2a98fb1640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a94102130 0x7f2a9419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.153+0000 7f2a98fb1640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9410c1e0 con 0x7f2a94101770 2024-03-07T07:43:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.153+0000 7f2a98fb1640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9410c060 con 0x7f2a94102130 2024-03-07T07:43:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.153+0000 7f2a98fb1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9410c360 con 0x7f2a94100580 2024-03-07T07:43:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.154+0000 7f2a91d9b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a94101770 0x7f2a94197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.154+0000 7f2a91d9b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a94101770 0x7f2a94197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:51310/0 (socket says 172.21.15.160:51310) 2024-03-07T07:43:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.154+0000 7f2a91d9b640 1 -- 172.21.15.160:0/3816898789 learned_addr learned my addr 172.21.15.160:0/3816898789 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:43:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.154+0000 7f2a92d9d640 1 --2- 172.21.15.160:0/3816898789 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a94102130 0x7f2a9419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.154+0000 7f2a9259c640 1 --2- 172.21.15.160:0/3816898789 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a94100580 0x7f2a94197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.154+0000 7f2a92d9d640 1 -- 172.21.15.160:0/3816898789 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a94100580 msgr2=0x7f2a94197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:43:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.154+0000 7f2a92d9d640 1 --2- 172.21.15.160:0/3816898789 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a94100580 0x7f2a94197960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.154+0000 7f2a92d9d640 1 -- 172.21.15.160:0/3816898789 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a94101770 msgr2=0x7f2a94197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:43:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.154+0000 7f2a92d9d640 1 --2- 172.21.15.160:0/3816898789 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a94101770 0x7f2a94197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.154+0000 7f2a92d9d640 1 -- 172.21.15.160:0/3816898789 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a9419f530 con 0x7f2a94102130 2024-03-07T07:43:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.154+0000 7f2a91d9b640 1 --2- 172.21.15.160:0/3816898789 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a94101770 0x7f2a94197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:43:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.154+0000 7f2a92d9d640 1 --2- 172.21.15.160:0/3816898789 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a94102130 0x7f2a9419eec0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c009fd0 tx=0x7f2a8c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:43:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.154+0000 7f2a837fe640 1 -- 172.21.15.160:0/3816898789 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c00ce20 con 0x7f2a94102130 2024-03-07T07:43:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.154+0000 7f2a98fb1640 1 -- 172.21.15.160:0/3816898789 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a9419f820 con 0x7f2a94102130 2024-03-07T07:43:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.154+0000 7f2a837fe640 1 -- 172.21.15.160:0/3816898789 <== mon.2 v2:172.21.15.177:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a8c005020 con 0x7f2a94102130 2024-03-07T07:43:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.154+0000 7f2a837fe640 1 -- 172.21.15.160:0/3816898789 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c010720 con 0x7f2a94102130 2024-03-07T07:43:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.154+0000 7f2a98fb1640 1 -- 172.21.15.160:0/3816898789 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a9419fd20 con 0x7f2a94102130 2024-03-07T07:43:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.156+0000 7f2a837fe640 1 -- 172.21.15.160:0/3816898789 <== mon.2 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 20) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f2a8c005540 con 0x7f2a94102130 2024-03-07T07:43:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.157+0000 7f2a9bd9a740 1 Processor -- start 2024-03-07T07:43:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.157+0000 7f2a9bd9a740 1 -- start start 2024-03-07T07:43:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.157+0000 7f2a9bd9a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560e47f101a0 0x560e47f105c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.157+0000 7f2a837fe640 1 --2- 172.21.15.160:0/3816898789 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2a7c0a1e20 0x7f2a7c0a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.157+0000 7f2a9bd9a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560e47f10b90 0x560e47f15690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.157+0000 7f2a9bd9a740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560e47f15d00 0x560e47f18190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.157+0000 7f2a9bd9a740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e47e755c0 con 0x560e47f15d00 2024-03-07T07:43:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.157+0000 7f2a9bd9a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e47e1e900 con 0x560e47f101a0 2024-03-07T07:43:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.157+0000 7f2a9bd9a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e47e71660 con 0x560e47f10b90 2024-03-07T07:43:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.157+0000 7f2a91d9b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560e47f101a0 0x560e47f105c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.157+0000 7f2a92d9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560e47f15d00 0x560e47f18190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.157+0000 7f2a9259c640 1 --2- 172.21.15.160:0/3816898789 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2a7c0a1e20 0x7f2a7c0a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.157+0000 7f2a91d9b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560e47f101a0 0x560e47f105c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.160:51826/0 (socket says 172.21.15.160:51826) 2024-03-07T07:43:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.157+0000 7f2a91d9b640 1 -- 172.21.15.160:0/4023424056 learned_addr learned my addr 172.21.15.160:0/4023424056 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:43:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.157+0000 7f2a91d9b640 1 --2- 172.21.15.160:0/4023424056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560e47f10b90 0x560e47f15690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.157+0000 7f2a837fe640 1 -- 172.21.15.160:0/3816898789 <== mon.2 v2:172.21.15.177:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2a8c0ce490 con 0x7f2a94102130 2024-03-07T07:43:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.158+0000 7f2a91d9b640 1 -- 172.21.15.160:0/4023424056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560e47f10b90 msgr2=0x560e47f15690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:43:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.158+0000 7f2a91d9b640 1 --2- 172.21.15.160:0/4023424056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560e47f10b90 0x560e47f15690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.158+0000 7f2a91d9b640 1 -- 172.21.15.160:0/4023424056 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560e47f15d00 msgr2=0x560e47f18190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:43:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.158+0000 7f2a91d9b640 1 --2- 172.21.15.160:0/4023424056 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560e47f15d00 0x560e47f18190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.158+0000 7f2a91d9b640 1 -- 172.21.15.160:0/4023424056 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e47b8a210 con 0x560e47f101a0 2024-03-07T07:43:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.158+0000 7f2a91d9b640 1 --2- 172.21.15.160:0/4023424056 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560e47f101a0 0x560e47f105c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2a88011c70 tx=0x7f2a88009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f597da3bb294d28 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:43:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.158+0000 7f2a9259c640 1 --2- 172.21.15.160:0/3816898789 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2a7c0a1e20 0x7f2a7c0a42e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2a84000c00 tx=0x7f2a84005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:43:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.158+0000 7f2a80ff9640 1 -- 172.21.15.160:0/4023424056 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8800fc50 con 0x560e47f101a0 2024-03-07T07:43:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.158+0000 7f2a80ff9640 1 -- 172.21.15.160:0/4023424056 <== mon.2 v2:172.21.15.177:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a8800fdf0 con 0x560e47f101a0 2024-03-07T07:43:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.158+0000 7f2a80ff9640 1 -- 172.21.15.160:0/4023424056 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8800e6a0 con 0x560e47f101a0 2024-03-07T07:43:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.159+0000 7f2a9bd9a740 1 -- 172.21.15.160:0/4023424056 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560e47f101a0 msgr2=0x560e47f105c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:43:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.159+0000 7f2a9bd9a740 1 --2- 172.21.15.160:0/4023424056 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560e47f101a0 0x560e47f105c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2a88011c70 tx=0x7f2a88009c60 comp rx=0 tx=0).stop 2024-03-07T07:43:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.159+0000 7f2a9bd9a740 1 -- 172.21.15.160:0/4023424056 shutdown_connections 2024-03-07T07:43:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.159+0000 7f2a9bd9a740 1 --2- 172.21.15.160:0/4023424056 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560e47f15d00 0x560e47f18190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.159+0000 7f2a9bd9a740 1 --2- 172.21.15.160:0/4023424056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560e47f10b90 0x560e47f15690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.159+0000 7f2a9bd9a740 1 --2- 172.21.15.160:0/4023424056 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560e47f101a0 0x560e47f105c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.159+0000 7f2a9bd9a740 1 -- 172.21.15.160:0/4023424056 >> 172.21.15.160:0/4023424056 conn(0x560e47ed8580 msgr2=0x560e47f0bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:43:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.159+0000 7f2a9bd9a740 1 -- 172.21.15.160:0/4023424056 shutdown_connections 2024-03-07T07:43:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.159+0000 7f2a9bd9a740 1 -- 172.21.15.160:0/4023424056 wait complete. 2024-03-07T07:43:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.159+0000 7f2a9bd9a740 1 -- start start 2024-03-07T07:43:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.159+0000 7f2a9bd9a740 1 Processor -- start 2024-03-07T07:43:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.160+0000 7f2a9bd9a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560e47f101a0 0x560e47f1a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.160+0000 7f2a9bd9a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560e47f10b90 0x560e47f1aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.160+0000 7f2a9bd9a740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560e47f15d00 0x560e47f21ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.160+0000 7f2a9bd9a740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e47f18cb0 con 0x560e47f15d00 2024-03-07T07:43:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.160+0000 7f2a9bd9a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e47e31d60 con 0x560e47f10b90 2024-03-07T07:43:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.160+0000 7f2a9bd9a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e47f18e30 con 0x560e47f101a0 2024-03-07T07:43:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.160+0000 7f2a9bd9a740 10 client.0 _ll_register_callbacks cb 0x560e47b89a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-07T07:43:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.160+0000 7f2a91d9b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560e47f10b90 0x560e47f1aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.160+0000 7f2a92d9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560e47f15d00 0x560e47f21ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.160+0000 7f2a91d9b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560e47f10b90 0x560e47f1aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.160:51840/0 (socket says 172.21.15.160:51840) 2024-03-07T07:43:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.160+0000 7f2a91d9b640 1 -- 172.21.15.160:0/2349054075 learned_addr learned my addr 172.21.15.160:0/2349054075 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:43:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.160+0000 7f2a92d9d640 1 --2- 172.21.15.160:0/2349054075 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560e47f15d00 0x560e47f21ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:51320/0 (socket says 172.21.15.160:51320) 2024-03-07T07:43:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.160+0000 7f2a9259c640 1 --2- 172.21.15.160:0/2349054075 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560e47f101a0 0x560e47f1a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.160+0000 7f2a91d9b640 1 -- 172.21.15.160:0/2349054075 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560e47f101a0 msgr2=0x560e47f1a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:43:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.160+0000 7f2a91d9b640 1 --2- 172.21.15.160:0/2349054075 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560e47f101a0 0x560e47f1a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.160+0000 7f2a91d9b640 1 -- 172.21.15.160:0/2349054075 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560e47f15d00 msgr2=0x560e47f21ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:43:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.160+0000 7f2a91d9b640 1 --2- 172.21.15.160:0/2349054075 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560e47f15d00 0x560e47f21ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.160+0000 7f2a91d9b640 1 -- 172.21.15.160:0/2349054075 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560e47c56620 con 0x560e47f10b90 2024-03-07T07:43:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.160+0000 7f2a91d9b640 1 --2- 172.21.15.160:0/2349054075 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560e47f10b90 0x560e47f1aa80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2a880055d0 tx=0x7f2a88005600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d81c360367ced5ad server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:43:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.160+0000 7f2a79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-03-07T07:43:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.160+0000 7f2a79ffb640 1 -- 172.21.15.160:0/2349054075 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a88025050 con 0x560e47f10b90 2024-03-07T07:43:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.160+0000 7f2a79ffb640 1 -- 172.21.15.160:0/2349054075 <== mon.2 v2:172.21.15.177:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2a88004020 con 0x560e47f10b90 2024-03-07T07:43:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.160+0000 7f2a9bd9a740 1 -- 172.21.15.160:0/2349054075 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a8800fdf0 con 0x560e47f10b90 2024-03-07T07:43:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.161+0000 7f2a79ffb640 1 -- 172.21.15.160:0/2349054075 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a8800fc50 con 0x560e47f10b90 2024-03-07T07:43:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.161+0000 7f2a9bd9a740 10 client.25283 fetch_fsmap learned FSMap version 22 2024-03-07T07:43:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.161+0000 7f2a9bd9a740 1 -- 172.21.15.160:0/2349054075 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x560e47f24e30 con 0x560e47f10b90 2024-03-07T07:43:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.161+0000 7f2a79ffb640 1 -- 172.21.15.160:0/2349054075 <== mon.2 v2:172.21.15.177:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a8800fdf0 con 0x560e47f10b90 2024-03-07T07:43:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.161+0000 7f2a79ffb640 1 -- 172.21.15.160:0/2349054075 <== mon.2 v2:172.21.15.177:3300/0 5 ==== fsmap(e 22) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f2a88032020 con 0x560e47f10b90 2024-03-07T07:43:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.161+0000 7f2a9bd9a740 10 client.25283 fetch_fsmap finished waiting for FSMap version 22 2024-03-07T07:43:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.161+0000 7f2a9bd9a740 10 client.25283 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-07T07:43:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.161+0000 7f2a9bd9a740 20 client.25283 populate_metadata read hostname 'smithi160' 2024-03-07T07:43:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.161+0000 7f2a9bd9a740 1 --2- 172.21.15.160:0/2349054075 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x560e47f26230 0x560e47f28620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.161+0000 7f2a9bd9a740 4 client.25283 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-07T07:43:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.161+0000 7f2a9bd9a740 1 -- 172.21.15.160:0/2349054075 --> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560e47e82b00 con 0x560e47f26230 2024-03-07T07:43:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.161+0000 7f2a9259c640 1 --2- 172.21.15.160:0/2349054075 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x560e47f26230 0x560e47f28620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.162+0000 7f2a79ffb640 10 client.25283 ms_handle_connect on v2:172.21.15.178:6838/2947354024 2024-03-07T07:43:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.162+0000 7f2a9259c640 1 --2- 172.21.15.160:0/2349054075 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x560e47f26230 0x560e47f28620 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43b833a955afb5e9 server_cookie=5044284914525908 in_seq=0 out_seq=0 2024-03-07T07:43:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.162+0000 7f2a79ffb640 1 -- 172.21.15.160:0/2349054075 <== mds.0 v2:172.21.15.178:6838/2947354024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560e47e82b00 con 0x560e47f26230 2024-03-07T07:43:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.162+0000 7f2a79ffb640 10 client.25283 handle_command_reply: tid=0 2024-03-07T07:43:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.162+0000 7f2a9bd9a740 1 client.25283 shutdown 2024-03-07T07:43:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.162+0000 7f2a9bd9a740 1 -- 172.21.15.160:0/2349054075 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560e47f10b90 msgr2=0x560e47f1aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:43:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.162+0000 7f2a9bd9a740 1 --2- 172.21.15.160:0/2349054075 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560e47f10b90 0x560e47f1aa80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2a880055d0 tx=0x7f2a88005600 comp rx=0 tx=0).stop 2024-03-07T07:43:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.162+0000 7f2a9bd9a740 1 -- 172.21.15.160:0/2349054075 shutdown_connections 2024-03-07T07:43:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.162+0000 7f2a9bd9a740 1 --2- 172.21.15.160:0/2349054075 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x560e47f26230 0x560e47f28620 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.162+0000 7f2a9bd9a740 1 --2- 172.21.15.160:0/2349054075 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560e47f15d00 0x560e47f21ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.162+0000 7f2a9bd9a740 1 --2- 172.21.15.160:0/2349054075 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560e47f10b90 0x560e47f1aa80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.162+0000 7f2a9bd9a740 1 --2- 172.21.15.160:0/2349054075 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560e47f101a0 0x560e47f1a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.162+0000 7f2a79ffb640 0 client.25283 ms_handle_reset on v2:172.21.15.178:6838/2947354024 2024-03-07T07:43:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.162+0000 7f2a9bd9a740 1 -- 172.21.15.160:0/2349054075 >> 172.21.15.160:0/2349054075 conn(0x560e47ed8580 msgr2=0x560e47f0ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:43:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.162+0000 7f2a9bd9a740 1 -- 172.21.15.160:0/2349054075 shutdown_connections 2024-03-07T07:43:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.162+0000 7f2a9bd9a740 1 -- 172.21.15.160:0/2349054075 wait complete. 2024-03-07T07:43:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.163+0000 7f2a9bd9a740 20 client.25283 trim_cache size 0 max 16384 2024-03-07T07:43:23.178 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.176+0000 7f2a9bd9a740 1 Processor -- start 2024-03-07T07:43:23.178 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.177+0000 7f2a9bd9a740 1 -- start start 2024-03-07T07:43:23.178 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.177+0000 7f2a9bd9a740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560e47f101a0 0x560e47f0ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:23.178 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.177+0000 7f2a9bd9a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560e47f10b90 0x560e47f21a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:23.178 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.177+0000 7f2a9bd9a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560e47f15d00 0x560e47f0f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:23.178 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.177+0000 7f2a9bd9a740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a941a8cd0 con 0x560e47f101a0 2024-03-07T07:43:23.178 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.177+0000 7f2a9bd9a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a941a67d0 con 0x560e47f10b90 2024-03-07T07:43:23.178 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.177+0000 7f2a9bd9a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a941a4850 con 0x560e47f15d00 2024-03-07T07:43:23.178 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.177+0000 7f2a9259c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560e47f101a0 0x560e47f0ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:23.178 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.177+0000 7f2a9259c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560e47f101a0 0x560e47f0ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:51336/0 (socket says 172.21.15.160:51336) 2024-03-07T07:43:23.178 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.177+0000 7f2a92d9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560e47f15d00 0x560e47f0f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.177+0000 7f2a91d9b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560e47f10b90 0x560e47f21a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.177+0000 7f2a9259c640 1 -- 172.21.15.160:0/1990963837 learned_addr learned my addr 172.21.15.160:0/1990963837 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:43:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.177+0000 7f2a91d9b640 1 -- 172.21.15.160:0/1990963837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560e47f15d00 msgr2=0x560e47f0f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:43:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.177+0000 7f2a91d9b640 1 --2- 172.21.15.160:0/1990963837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560e47f15d00 0x560e47f0f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.177+0000 7f2a91d9b640 1 -- 172.21.15.160:0/1990963837 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560e47f101a0 msgr2=0x560e47f0ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:43:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.177+0000 7f2a91d9b640 1 --2- 172.21.15.160:0/1990963837 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560e47f101a0 0x560e47f0ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.177+0000 7f2a91d9b640 1 -- 172.21.15.160:0/1990963837 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e47f10580 con 0x560e47f10b90 2024-03-07T07:43:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.177+0000 7f2a9259c640 1 --2- 172.21.15.160:0/1990963837 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560e47f101a0 0x560e47f0ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:43:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.177+0000 7f2a91d9b640 1 --2- 172.21.15.160:0/1990963837 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560e47f10b90 0x560e47f21a80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2a88001eb0 tx=0x7f2a880119e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d34109435893872 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:43:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.178+0000 7f2a7bfff640 1 -- 172.21.15.160:0/1990963837 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a88039030 con 0x560e47f10b90 2024-03-07T07:43:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.178+0000 7f2a7bfff640 1 -- 172.21.15.160:0/1990963837 <== mon.2 v2:172.21.15.177:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a8801f440 con 0x560e47f10b90 2024-03-07T07:43:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.178+0000 7f2a7bfff640 1 -- 172.21.15.160:0/1990963837 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a88011040 con 0x560e47f10b90 2024-03-07T07:43:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.178+0000 7f2a9bd9a740 1 -- 172.21.15.160:0/1990963837 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560e47f10b90 msgr2=0x560e47f21a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:43:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.178+0000 7f2a9bd9a740 1 --2- 172.21.15.160:0/1990963837 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560e47f10b90 0x560e47f21a80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2a88001eb0 tx=0x7f2a880119e0 comp rx=0 tx=0).stop 2024-03-07T07:43:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.178+0000 7f2a9bd9a740 1 -- 172.21.15.160:0/1990963837 shutdown_connections 2024-03-07T07:43:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.178+0000 7f2a9bd9a740 1 --2- 172.21.15.160:0/1990963837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560e47f15d00 0x560e47f0f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.178+0000 7f2a9bd9a740 1 --2- 172.21.15.160:0/1990963837 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560e47f10b90 0x560e47f21a80 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.178+0000 7f2a9bd9a740 1 --2- 172.21.15.160:0/1990963837 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560e47f101a0 0x560e47f0ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.178+0000 7f2a9bd9a740 1 -- 172.21.15.160:0/1990963837 >> 172.21.15.160:0/1990963837 conn(0x560e47ed8580 msgr2=0x560e47f11ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:43:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.178+0000 7f2a9bd9a740 1 -- 172.21.15.160:0/1990963837 shutdown_connections 2024-03-07T07:43:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.178+0000 7f2a9bd9a740 1 -- 172.21.15.160:0/1990963837 wait complete. 2024-03-07T07:43:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.179+0000 7f2a9bd9a740 1 -- start start 2024-03-07T07:43:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.179+0000 7f2a9bd9a740 1 Processor -- start 2024-03-07T07:43:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.179+0000 7f2a9bd9a740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560e47f101a0 0x560e47f1f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.179+0000 7f2a9bd9a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560e47f10b90 0x560e47f1fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.179+0000 7f2a9bd9a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560e47f15d00 0x560e47f20d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.179+0000 7f2a9bd9a740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a941a4e50 con 0x560e47f101a0 2024-03-07T07:43:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.179+0000 7f2a9bd9a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a941a6170 con 0x560e47f15d00 2024-03-07T07:43:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.179+0000 7f2a9bd9a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a941a54b0 con 0x560e47f10b90 2024-03-07T07:43:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.179+0000 7f2a9bd9a740 10 client.0 _ll_register_callbacks cb 0x7f2a88011760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-07T07:43:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.179+0000 7f2a9259c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560e47f101a0 0x560e47f1f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.179+0000 7f2a9259c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560e47f101a0 0x560e47f1f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:51352/0 (socket says 172.21.15.160:51352) 2024-03-07T07:43:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.179+0000 7f2a9259c640 1 -- 172.21.15.160:0/527867290 learned_addr learned my addr 172.21.15.160:0/527867290 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:43:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.179+0000 7f2a92d9d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560e47f15d00 0x560e47f20d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.179+0000 7f2a91d9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560e47f10b90 0x560e47f1fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.179+0000 7f2a9259c640 1 -- 172.21.15.160:0/527867290 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560e47f10b90 msgr2=0x560e47f1fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:43:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.179+0000 7f2a9259c640 1 --2- 172.21.15.160:0/527867290 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560e47f10b90 0x560e47f1fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.179+0000 7f2a9259c640 1 -- 172.21.15.160:0/527867290 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560e47f15d00 msgr2=0x560e47f20d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:43:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.179+0000 7f2a9259c640 1 --2- 172.21.15.160:0/527867290 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560e47f15d00 0x560e47f20d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.179+0000 7f2a9259c640 1 -- 172.21.15.160:0/527867290 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560e47f21380 con 0x560e47f101a0 2024-03-07T07:43:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.179+0000 7f2a91d9b640 1 --2- 172.21.15.160:0/527867290 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560e47f10b90 0x560e47f1fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:43:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.179+0000 7f2a92d9d640 1 --2- 172.21.15.160:0/527867290 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560e47f15d00 0x560e47f20d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:43:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.180+0000 7f2a9259c640 1 --2- 172.21.15.160:0/527867290 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560e47f101a0 0x560e47f1f840 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2a84036040 tx=0x7f2a8402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36035629d20f20d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:43:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.180+0000 7f2a79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-03-07T07:43:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.180+0000 7f2a79ffb640 1 -- 172.21.15.160:0/527867290 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a84004450 con 0x560e47f101a0 2024-03-07T07:43:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.180+0000 7f2a79ffb640 1 -- 172.21.15.160:0/527867290 <== mon.0 v2:172.21.15.160:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2a840035f0 con 0x560e47f101a0 2024-03-07T07:43:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.180+0000 7f2a9bd9a740 1 -- 172.21.15.160:0/527867290 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560e47e7ce50 con 0x560e47f101a0 2024-03-07T07:43:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.180+0000 7f2a79ffb640 1 -- 172.21.15.160:0/527867290 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a840037f0 con 0x560e47f101a0 2024-03-07T07:43:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.180+0000 7f2a9bd9a740 10 client.15453 fetch_fsmap learned FSMap version 22 2024-03-07T07:43:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.180+0000 7f2a9bd9a740 1 -- 172.21.15.160:0/527867290 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x560e47f82120 con 0x560e47f101a0 2024-03-07T07:43:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.180+0000 7f2a79ffb640 1 -- 172.21.15.160:0/527867290 <== mon.0 v2:172.21.15.160:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560e47e7ce50 con 0x560e47f101a0 2024-03-07T07:43:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.180+0000 7f2a79ffb640 1 -- 172.21.15.160:0/527867290 <== mon.0 v2:172.21.15.160:3300/0 5 ==== fsmap(e 22) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f2a84034440 con 0x560e47f101a0 2024-03-07T07:43:23.186 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.180+0000 7f2a9bd9a740 10 client.15453 fetch_fsmap finished waiting for FSMap version 22 2024-03-07T07:43:23.186 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.180+0000 7f2a9bd9a740 10 client.15453 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-07T07:43:23.186 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.180+0000 7f2a9bd9a740 20 client.15453 populate_metadata read hostname 'smithi160' 2024-03-07T07:43:23.186 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.180+0000 7f2a9bd9a740 1 --2- 172.21.15.160:0/527867290 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x560e47f82b00 0x560e47f84f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:23.186 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.180+0000 7f2a9bd9a740 4 client.15453 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-07T07:43:23.186 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.180+0000 7f2a9bd9a740 1 -- 172.21.15.160:0/527867290 --> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560e47dfb6e0 con 0x560e47f82b00 2024-03-07T07:43:23.186 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.181+0000 7f2a91d9b640 1 --2- 172.21.15.160:0/527867290 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x560e47f82b00 0x560e47f84f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:23.186 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.181+0000 7f2a79ffb640 10 client.15453 ms_handle_connect on v2:172.21.15.178:6838/2947354024 2024-03-07T07:43:23.186 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.181+0000 7f2a91d9b640 1 --2- 172.21.15.160:0/527867290 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x560e47f82b00 0x560e47f84f20 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=202fa5bbaf957f03 server_cookie=ff1bd7997d63b6af in_seq=0 out_seq=0 2024-03-07T07:43:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.189+0000 7f2a79ffb640 1 -- 172.21.15.160:0/527867290 <== mds.0 v2:172.21.15.178:6838/2947354024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560e47dfb6e0 con 0x560e47f82b00 2024-03-07T07:43:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.189+0000 7f2a79ffb640 10 client.15453 handle_command_reply: tid=0 2024-03-07T07:43:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.189+0000 7f2a9bd9a740 1 client.15453 shutdown 2024-03-07T07:43:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.189+0000 7f2a9bd9a740 1 -- 172.21.15.160:0/527867290 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560e47f101a0 msgr2=0x560e47f1f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:43:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.189+0000 7f2a9bd9a740 1 --2- 172.21.15.160:0/527867290 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560e47f101a0 0x560e47f1f840 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2a84036040 tx=0x7f2a8402bc30 comp rx=0 tx=0).stop 2024-03-07T07:43:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.190+0000 7f2a9bd9a740 1 -- 172.21.15.160:0/527867290 shutdown_connections 2024-03-07T07:43:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.190+0000 7f2a9bd9a740 1 --2- 172.21.15.160:0/527867290 >> [v2:172.21.15.178:6838/2947354024,v1:172.21.15.178:6839/2947354024] conn(0x560e47f82b00 0x560e47f84f20 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.190+0000 7f2a9bd9a740 1 --2- 172.21.15.160:0/527867290 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560e47f15d00 0x560e47f20d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.190+0000 7f2a9bd9a740 1 --2- 172.21.15.160:0/527867290 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560e47f10b90 0x560e47f1fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.190+0000 7f2a79ffb640 0 client.15453 ms_handle_reset on v2:172.21.15.178:6838/2947354024 2024-03-07T07:43:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.190+0000 7f2a9bd9a740 1 --2- 172.21.15.160:0/527867290 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560e47f101a0 0x560e47f1f840 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.190+0000 7f2a9bd9a740 1 -- 172.21.15.160:0/527867290 >> 172.21.15.160:0/527867290 conn(0x560e47ed8580 msgr2=0x560e47f0ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:43:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.190+0000 7f2a9bd9a740 1 -- 172.21.15.160:0/527867290 shutdown_connections 2024-03-07T07:43:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.190+0000 7f2a9bd9a740 1 -- 172.21.15.160:0/527867290 wait complete. 2024-03-07T07:43:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.190+0000 7f2a9bd9a740 20 client.15453 trim_cache size 0 max 16384 2024-03-07T07:43:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stdout:{ 2024-03-07T07:43:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stdout: "message": "", 2024-03-07T07:43:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stdout: "return_code": 0 2024-03-07T07:43:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stdout:} 2024-03-07T07:43:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.190+0000 7f2a98fb1640 1 -- 172.21.15.160:0/3816898789 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2a7c0a1e20 msgr2=0x7f2a7c0a42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:43:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.190+0000 7f2a98fb1640 1 --2- 172.21.15.160:0/3816898789 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2a7c0a1e20 0x7f2a7c0a42e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2a84000c00 tx=0x7f2a84005fd0 comp rx=0 tx=0).stop 2024-03-07T07:43:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.190+0000 7f2a98fb1640 1 -- 172.21.15.160:0/3816898789 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a94102130 msgr2=0x7f2a9419eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:43:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.190+0000 7f2a98fb1640 1 --2- 172.21.15.160:0/3816898789 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a94102130 0x7f2a9419eec0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c009fd0 tx=0x7f2a8c00ef10 comp rx=0 tx=0).stop 2024-03-07T07:43:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.190+0000 7f2a98fb1640 1 -- 172.21.15.160:0/3816898789 shutdown_connections 2024-03-07T07:43:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.190+0000 7f2a98fb1640 1 --2- 172.21.15.160:0/3816898789 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2a7c0a1e20 0x7f2a7c0a42e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.190+0000 7f2a98fb1640 1 --2- 172.21.15.160:0/3816898789 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a94102130 0x7f2a9419eec0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.190+0000 7f2a98fb1640 1 --2- 172.21.15.160:0/3816898789 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a94101770 0x7f2a94197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.190+0000 7f2a98fb1640 1 --2- 172.21.15.160:0/3816898789 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a94100580 0x7f2a94197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.190+0000 7f2a98fb1640 1 -- 172.21.15.160:0/3816898789 >> 172.21.15.160:0/3816898789 conn(0x7f2a940fbaf0 msgr2=0x7f2a940fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:43:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.190+0000 7f2a98fb1640 1 -- 172.21.15.160:0/3816898789 shutdown_connections 2024-03-07T07:43:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi160.front.sepia.ceph.com.smithi160.stderr:2024-03-07T07:43:23.190+0000 7f2a98fb1640 1 -- 172.21.15.160:0/3816898789 wait complete. 2024-03-07T07:43:23.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:43:23 smithi160 ceph-mon[31963]: pgmap v414: 97 pgs: 97 active+clean; 258 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 2.5 MiB/s wr, 10 op/s 2024-03-07T07:43:23.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:43:23 smithi178 ceph-mon[36650]: pgmap v414: 97 pgs: 97 active+clean; 258 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 2.5 MiB/s wr, 10 op/s 2024-03-07T07:43:23.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:43:23 smithi177 ceph-mon[36578]: pgmap v414: 97 pgs: 97 active+clean; 258 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 2.5 MiB/s wr, 10 op/s 2024-03-07T07:43:25.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:43:25 smithi160 ceph-mon[31963]: pgmap v415: 97 pgs: 97 active+clean; 258 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 1.3 MiB/s wr, 11 op/s 2024-03-07T07:43:25.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:43:25 smithi178 ceph-mon[36650]: pgmap v415: 97 pgs: 97 active+clean; 258 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 1.3 MiB/s wr, 11 op/s 2024-03-07T07:43:25.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:43:25 smithi177 ceph-mon[36578]: pgmap v415: 97 pgs: 97 active+clean; 258 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 1.3 MiB/s wr, 11 op/s 2024-03-07T07:43:27.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:43:27 smithi160 ceph-mon[31963]: pgmap v416: 97 pgs: 97 active+clean; 258 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 864 KiB/s wr, 13 op/s 2024-03-07T07:43:27.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:43:27 smithi178 ceph-mon[36650]: pgmap v416: 97 pgs: 97 active+clean; 258 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 864 KiB/s wr, 13 op/s 2024-03-07T07:43:27.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:43:27 smithi177 ceph-mon[36578]: pgmap v416: 97 pgs: 97 active+clean; 258 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 864 KiB/s wr, 13 op/s 2024-03-07T07:43:29.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:43:29 smithi160 ceph-mon[31963]: pgmap v417: 97 pgs: 97 active+clean; 258 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 860 KiB/s wr, 10 op/s 2024-03-07T07:43:29.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:43:29 smithi178 ceph-mon[36650]: pgmap v417: 97 pgs: 97 active+clean; 258 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 860 KiB/s wr, 10 op/s 2024-03-07T07:43:29.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:43:29 smithi177 ceph-mon[36578]: pgmap v417: 97 pgs: 97 active+clean; 258 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 860 KiB/s wr, 10 op/s 2024-03-07T07:43:30.364 INFO:tasks.workunit.client.0.smithi160.stdout:setting up random write file 2024-03-07T07:43:30.364 INFO:tasks.workunit.client.0.smithi160.stdout:done setting up random write file 2024-03-07T07:43:30.364 INFO:tasks.workunit.client.0.smithi160.stdout:starting fsync run 2024-03-07T07:43:30.364 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0078s 2024-03-07T07:43:30.364 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0057s 2024-03-07T07:43:30.364 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0053s 2024-03-07T07:43:30.364 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0051s 2024-03-07T07:43:30.364 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0054s 2024-03-07T07:43:30.365 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0051s 2024-03-07T07:43:30.365 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0004s fsync time: 0.0058s 2024-03-07T07:43:30.365 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0053s 2024-03-07T07:43:30.365 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0051s 2024-03-07T07:43:30.365 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0048s 2024-03-07T07:43:30.365 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0045s 2024-03-07T07:43:30.365 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0051s 2024-03-07T07:43:30.365 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0053s 2024-03-07T07:43:30.365 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0047s 2024-03-07T07:43:30.365 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0048s 2024-03-07T07:43:30.365 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0048s 2024-03-07T07:43:30.365 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0050s 2024-03-07T07:43:30.365 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0046s 2024-03-07T07:43:30.365 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0049s 2024-03-07T07:43:30.366 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0048s 2024-03-07T07:43:30.366 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0043s 2024-03-07T07:43:30.366 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0047s 2024-03-07T07:43:30.366 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0046s 2024-03-07T07:43:30.366 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0049s 2024-03-07T07:43:30.366 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0044s 2024-03-07T07:43:30.366 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0053s 2024-03-07T07:43:30.366 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0045s 2024-03-07T07:43:30.366 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0051s 2024-03-07T07:43:30.366 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0046s 2024-03-07T07:43:30.366 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0048s 2024-03-07T07:43:30.366 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0044s 2024-03-07T07:43:30.366 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0047s 2024-03-07T07:43:30.366 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0048s 2024-03-07T07:43:30.367 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0043s 2024-03-07T07:43:30.367 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0051s 2024-03-07T07:43:30.367 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0045s 2024-03-07T07:43:30.367 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0049s 2024-03-07T07:43:30.367 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0044s 2024-03-07T07:43:30.367 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0054s 2024-03-07T07:43:30.367 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0047s 2024-03-07T07:43:30.367 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0048s 2024-03-07T07:43:30.367 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0045s 2024-03-07T07:43:30.367 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0049s 2024-03-07T07:43:30.367 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0047s 2024-03-07T07:43:30.367 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0054s 2024-03-07T07:43:30.367 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0051s 2024-03-07T07:43:30.367 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0049s 2024-03-07T07:43:30.368 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0054s 2024-03-07T07:43:30.368 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0048s 2024-03-07T07:43:30.368 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0049s 2024-03-07T07:43:30.368 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0045s 2024-03-07T07:43:30.368 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0043s 2024-03-07T07:43:30.368 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0045s 2024-03-07T07:43:30.368 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0055s 2024-03-07T07:43:30.368 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0045s 2024-03-07T07:43:30.368 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0048s 2024-03-07T07:43:30.368 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0049s 2024-03-07T07:43:30.368 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0049s 2024-03-07T07:43:30.368 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0050s 2024-03-07T07:43:30.368 INFO:tasks.workunit.client.0.smithi160.stdout:write time: 0.0003s fsync time: 0.0044s 2024-03-07T07:43:30.369 INFO:tasks.workunit.client.0.smithi160.stdout:run done 61 fsyncs total, killing random writer 2024-03-07T07:43:30.369 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2024-03-07T07:43:30.369 INFO:tasks.workunit.client.0.smithi160.stdout:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2024-03-07T07:43:30.369 INFO:tasks.workunit.client.0.smithi160.stderr:+ whereis lsof 2024-03-07T07:43:30.467 INFO:tasks.workunit.client.0.smithi160.stdout:lsof: /usr/bin/lsof /usr/share/man/man1/lsof.1.gz 2024-03-07T07:43:30.467 INFO:tasks.workunit.client.0.smithi160.stderr:+ lsof 2024-03-07T07:43:30.482 INFO:tasks.workunit.client.0.smithi160.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/c21ee0b0a82b35455cb4e426e4c945c9db012b0bb9a637fcd60fa27a453c4960/merged 2024-03-07T07:43:30.482 INFO:tasks.workunit.client.0.smithi160.stderr: Output information may be incomplete. 2024-03-07T07:43:30.482 INFO:tasks.workunit.client.0.smithi160.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/fe87c35a44a44b7933b693331deb001b683bdc88b43e8d32a9c53a66d9769c1c/merged 2024-03-07T07:43:30.482 INFO:tasks.workunit.client.0.smithi160.stderr: Output information may be incomplete. 2024-03-07T07:43:30.482 INFO:tasks.workunit.client.0.smithi160.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/f7d8ab2035cebc5f7a2b2868e191792e798f8acf0ce41177b2b55c6d711df454/merged 2024-03-07T07:43:30.482 INFO:tasks.workunit.client.0.smithi160.stderr: Output information may be incomplete. 2024-03-07T07:43:30.482 INFO:tasks.workunit.client.0.smithi160.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/0c5297397963481512be49dac6b004dc443eca134cf545eb53ade2c72d32135b/merged 2024-03-07T07:43:30.482 INFO:tasks.workunit.client.0.smithi160.stderr: Output information may be incomplete. 2024-03-07T07:43:30.482 INFO:tasks.workunit.client.0.smithi160.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/fb513e3295cc53eb3181596b4c4ad8c637f1abcffb4d6870abbaa8c7520eb316/merged 2024-03-07T07:43:30.483 INFO:tasks.workunit.client.0.smithi160.stderr: Output information may be incomplete. 2024-03-07T07:43:30.483 INFO:tasks.workunit.client.0.smithi160.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/4d79eb329f96254417196b0fea1e5d9afe7a085e78e1bd6b47a2afa52452b452/merged 2024-03-07T07:43:30.483 INFO:tasks.workunit.client.0.smithi160.stderr: Output information may be incomplete. 2024-03-07T07:43:30.483 INFO:tasks.workunit.client.0.smithi160.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/6b55ee42ea4f3b6928ad5e4f83f1881f7f5865227b8be2487f09fee69a12fdbe/merged 2024-03-07T07:43:30.483 INFO:tasks.workunit.client.0.smithi160.stderr: Output information may be incomplete. 2024-03-07T07:43:30.483 INFO:tasks.workunit.client.0.smithi160.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/431dc959e6c38360463eda31dd167ce16080dcdd534078e2f8f42d5ebdb3406f/merged 2024-03-07T07:43:30.483 INFO:tasks.workunit.client.0.smithi160.stderr: Output information may be incomplete. 2024-03-07T07:43:30.483 INFO:tasks.workunit.client.0.smithi160.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/31125510f9ac88602a877e2601748d7a8af0b81893a77569f137e439ed6c1bd6/merged 2024-03-07T07:43:30.483 INFO:tasks.workunit.client.0.smithi160.stderr: Output information may be incomplete. 2024-03-07T07:43:30.483 INFO:tasks.workunit.client.0.smithi160.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/bee9dd03443bf8e5a5a22d5dfddcf8da12241d1cf8460920157fbef2f58429c9/merged 2024-03-07T07:43:30.483 INFO:tasks.workunit.client.0.smithi160.stderr: Output information may be incomplete. 2024-03-07T07:43:30.537 INFO:tasks.workunit.client.0.smithi160.stdout:COMMAND PID TID TASKCMD USER FD TYPE DEVICE SIZE/OFF NODE NAME 2024-03-07T07:43:30.537 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 1 root cwd unknown /proc/1/cwd (readlink: Permission denied) 2024-03-07T07:43:30.537 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 1 root rtd unknown /proc/1/root (readlink: Permission denied) 2024-03-07T07:43:30.537 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 1 root txt unknown /proc/1/exe (readlink: Permission denied) 2024-03-07T07:43:30.537 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 1 root NOFD /proc/1/fd (opendir: Permission denied) 2024-03-07T07:43:30.537 INFO:tasks.workunit.client.0.smithi160.stdout:kthreadd 2 root cwd unknown /proc/2/cwd (readlink: Permission denied) 2024-03-07T07:43:30.537 INFO:tasks.workunit.client.0.smithi160.stdout:kthreadd 2 root rtd unknown /proc/2/root (readlink: Permission denied) 2024-03-07T07:43:30.537 INFO:tasks.workunit.client.0.smithi160.stdout:kthreadd 2 root txt unknown /proc/2/exe (readlink: Permission denied) 2024-03-07T07:43:30.537 INFO:tasks.workunit.client.0.smithi160.stdout:kthreadd 2 root NOFD /proc/2/fd (opendir: Permission denied) 2024-03-07T07:43:30.538 INFO:tasks.workunit.client.0.smithi160.stdout:rcu_gp 3 root cwd unknown /proc/3/cwd (readlink: Permission denied) 2024-03-07T07:43:30.538 INFO:tasks.workunit.client.0.smithi160.stdout:rcu_gp 3 root rtd unknown /proc/3/root (readlink: Permission denied) 2024-03-07T07:43:30.538 INFO:tasks.workunit.client.0.smithi160.stdout:rcu_gp 3 root txt unknown /proc/3/exe (readlink: Permission denied) 2024-03-07T07:43:30.538 INFO:tasks.workunit.client.0.smithi160.stdout:rcu_gp 3 root NOFD /proc/3/fd (opendir: Permission denied) 2024-03-07T07:43:30.538 INFO:tasks.workunit.client.0.smithi160.stdout:rcu_par_g 4 root cwd unknown /proc/4/cwd (readlink: Permission denied) 2024-03-07T07:43:30.538 INFO:tasks.workunit.client.0.smithi160.stdout:rcu_par_g 4 root rtd unknown /proc/4/root (readlink: Permission denied) 2024-03-07T07:43:30.538 INFO:tasks.workunit.client.0.smithi160.stdout:rcu_par_g 4 root txt unknown /proc/4/exe (readlink: Permission denied) 2024-03-07T07:43:30.538 INFO:tasks.workunit.client.0.smithi160.stdout:rcu_par_g 4 root NOFD /proc/4/fd (opendir: Permission denied) 2024-03-07T07:43:30.538 INFO:tasks.workunit.client.0.smithi160.stdout:slub_flus 5 root cwd unknown /proc/5/cwd (readlink: Permission denied) 2024-03-07T07:43:30.538 INFO:tasks.workunit.client.0.smithi160.stdout:slub_flus 5 root rtd unknown /proc/5/root (readlink: Permission denied) 2024-03-07T07:43:30.538 INFO:tasks.workunit.client.0.smithi160.stdout:slub_flus 5 root txt unknown /proc/5/exe (readlink: Permission denied) 2024-03-07T07:43:30.538 INFO:tasks.workunit.client.0.smithi160.stdout:slub_flus 5 root NOFD /proc/5/fd (opendir: Permission denied) 2024-03-07T07:43:30.538 INFO:tasks.workunit.client.0.smithi160.stdout:netns 6 root cwd unknown /proc/6/cwd (readlink: Permission denied) 2024-03-07T07:43:30.539 INFO:tasks.workunit.client.0.smithi160.stdout:netns 6 root rtd unknown /proc/6/root (readlink: Permission denied) 2024-03-07T07:43:30.539 INFO:tasks.workunit.client.0.smithi160.stdout:netns 6 root txt unknown /proc/6/exe (readlink: Permission denied) 2024-03-07T07:43:30.539 INFO:tasks.workunit.client.0.smithi160.stdout:netns 6 root NOFD /proc/6/fd (opendir: Permission denied) 2024-03-07T07:43:30.539 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/0 8 root cwd unknown /proc/8/cwd (readlink: Permission denied) 2024-03-07T07:43:30.539 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/0 8 root rtd unknown /proc/8/root (readlink: Permission denied) 2024-03-07T07:43:30.539 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/0 8 root txt unknown /proc/8/exe (readlink: Permission denied) 2024-03-07T07:43:30.539 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/0 8 root NOFD /proc/8/fd (opendir: Permission denied) 2024-03-07T07:43:30.539 INFO:tasks.workunit.client.0.smithi160.stdout:mm_percpu 10 root cwd unknown /proc/10/cwd (readlink: Permission denied) 2024-03-07T07:43:30.539 INFO:tasks.workunit.client.0.smithi160.stdout:mm_percpu 10 root rtd unknown /proc/10/root (readlink: Permission denied) 2024-03-07T07:43:30.539 INFO:tasks.workunit.client.0.smithi160.stdout:mm_percpu 10 root txt unknown /proc/10/exe (readlink: Permission denied) 2024-03-07T07:43:30.539 INFO:tasks.workunit.client.0.smithi160.stdout:mm_percpu 10 root NOFD /proc/10/fd (opendir: Permission denied) 2024-03-07T07:43:30.539 INFO:tasks.workunit.client.0.smithi160.stdout:rcu_tasks 12 root cwd unknown /proc/12/cwd (readlink: Permission denied) 2024-03-07T07:43:30.539 INFO:tasks.workunit.client.0.smithi160.stdout:rcu_tasks 12 root rtd unknown /proc/12/root (readlink: Permission denied) 2024-03-07T07:43:30.539 INFO:tasks.workunit.client.0.smithi160.stdout:rcu_tasks 12 root txt unknown /proc/12/exe (readlink: Permission denied) 2024-03-07T07:43:30.540 INFO:tasks.workunit.client.0.smithi160.stdout:rcu_tasks 12 root NOFD /proc/12/fd (opendir: Permission denied) 2024-03-07T07:43:30.540 INFO:tasks.workunit.client.0.smithi160.stdout:rcu_tasks 13 root cwd unknown /proc/13/cwd (readlink: Permission denied) 2024-03-07T07:43:30.540 INFO:tasks.workunit.client.0.smithi160.stdout:rcu_tasks 13 root rtd unknown /proc/13/root (readlink: Permission denied) 2024-03-07T07:43:30.540 INFO:tasks.workunit.client.0.smithi160.stdout:rcu_tasks 13 root txt unknown /proc/13/exe (readlink: Permission denied) 2024-03-07T07:43:30.540 INFO:tasks.workunit.client.0.smithi160.stdout:rcu_tasks 13 root NOFD /proc/13/fd (opendir: Permission denied) 2024-03-07T07:43:30.540 INFO:tasks.workunit.client.0.smithi160.stdout:rcu_tasks 14 root cwd unknown /proc/14/cwd (readlink: Permission denied) 2024-03-07T07:43:30.540 INFO:tasks.workunit.client.0.smithi160.stdout:rcu_tasks 14 root rtd unknown /proc/14/root (readlink: Permission denied) 2024-03-07T07:43:30.540 INFO:tasks.workunit.client.0.smithi160.stdout:rcu_tasks 14 root txt unknown /proc/14/exe (readlink: Permission denied) 2024-03-07T07:43:30.540 INFO:tasks.workunit.client.0.smithi160.stdout:rcu_tasks 14 root NOFD /proc/14/fd (opendir: Permission denied) 2024-03-07T07:43:30.540 INFO:tasks.workunit.client.0.smithi160.stdout:ksoftirqd 15 root cwd unknown /proc/15/cwd (readlink: Permission denied) 2024-03-07T07:43:30.540 INFO:tasks.workunit.client.0.smithi160.stdout:ksoftirqd 15 root rtd unknown /proc/15/root (readlink: Permission denied) 2024-03-07T07:43:30.540 INFO:tasks.workunit.client.0.smithi160.stdout:ksoftirqd 15 root txt unknown /proc/15/exe (readlink: Permission denied) 2024-03-07T07:43:30.540 INFO:tasks.workunit.client.0.smithi160.stdout:ksoftirqd 15 root NOFD /proc/15/fd (opendir: Permission denied) 2024-03-07T07:43:30.541 INFO:tasks.workunit.client.0.smithi160.stdout:pr/ttyS1 16 root cwd unknown /proc/16/cwd (readlink: Permission denied) 2024-03-07T07:43:30.541 INFO:tasks.workunit.client.0.smithi160.stdout:pr/ttyS1 16 root rtd unknown /proc/16/root (readlink: Permission denied) 2024-03-07T07:43:30.541 INFO:tasks.workunit.client.0.smithi160.stdout:pr/ttyS1 16 root txt unknown /proc/16/exe (readlink: Permission denied) 2024-03-07T07:43:30.541 INFO:tasks.workunit.client.0.smithi160.stdout:pr/ttyS1 16 root NOFD /proc/16/fd (opendir: Permission denied) 2024-03-07T07:43:30.541 INFO:tasks.workunit.client.0.smithi160.stdout:rcu_preem 17 root cwd unknown /proc/17/cwd (readlink: Permission denied) 2024-03-07T07:43:30.541 INFO:tasks.workunit.client.0.smithi160.stdout:rcu_preem 17 root rtd unknown /proc/17/root (readlink: Permission denied) 2024-03-07T07:43:30.541 INFO:tasks.workunit.client.0.smithi160.stdout:rcu_preem 17 root txt unknown /proc/17/exe (readlink: Permission denied) 2024-03-07T07:43:30.541 INFO:tasks.workunit.client.0.smithi160.stdout:rcu_preem 17 root NOFD /proc/17/fd (opendir: Permission denied) 2024-03-07T07:43:30.541 INFO:tasks.workunit.client.0.smithi160.stdout:migration 18 root cwd unknown /proc/18/cwd (readlink: Permission denied) 2024-03-07T07:43:30.541 INFO:tasks.workunit.client.0.smithi160.stdout:migration 18 root rtd unknown /proc/18/root (readlink: Permission denied) 2024-03-07T07:43:30.541 INFO:tasks.workunit.client.0.smithi160.stdout:migration 18 root txt unknown /proc/18/exe (readlink: Permission denied) 2024-03-07T07:43:30.541 INFO:tasks.workunit.client.0.smithi160.stdout:migration 18 root NOFD /proc/18/fd (opendir: Permission denied) 2024-03-07T07:43:30.541 INFO:tasks.workunit.client.0.smithi160.stdout:idle_inje 19 root cwd unknown /proc/19/cwd (readlink: Permission denied) 2024-03-07T07:43:30.542 INFO:tasks.workunit.client.0.smithi160.stdout:idle_inje 19 root rtd unknown /proc/19/root (readlink: Permission denied) 2024-03-07T07:43:30.542 INFO:tasks.workunit.client.0.smithi160.stdout:idle_inje 19 root txt unknown /proc/19/exe (readlink: Permission denied) 2024-03-07T07:43:30.542 INFO:tasks.workunit.client.0.smithi160.stdout:idle_inje 19 root NOFD /proc/19/fd (opendir: Permission denied) 2024-03-07T07:43:30.542 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/0 20 root cwd unknown /proc/20/cwd (readlink: Permission denied) 2024-03-07T07:43:30.542 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/0 20 root rtd unknown /proc/20/root (readlink: Permission denied) 2024-03-07T07:43:30.542 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/0 20 root txt unknown /proc/20/exe (readlink: Permission denied) 2024-03-07T07:43:30.542 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/0 20 root NOFD /proc/20/fd (opendir: Permission denied) 2024-03-07T07:43:30.542 INFO:tasks.workunit.client.0.smithi160.stdout:cpuhp/0 21 root cwd unknown /proc/21/cwd (readlink: Permission denied) 2024-03-07T07:43:30.542 INFO:tasks.workunit.client.0.smithi160.stdout:cpuhp/0 21 root rtd unknown /proc/21/root (readlink: Permission denied) 2024-03-07T07:43:30.542 INFO:tasks.workunit.client.0.smithi160.stdout:cpuhp/0 21 root txt unknown /proc/21/exe (readlink: Permission denied) 2024-03-07T07:43:30.542 INFO:tasks.workunit.client.0.smithi160.stdout:cpuhp/0 21 root NOFD /proc/21/fd (opendir: Permission denied) 2024-03-07T07:43:30.542 INFO:tasks.workunit.client.0.smithi160.stdout:cpuhp/1 22 root cwd unknown /proc/22/cwd (readlink: Permission denied) 2024-03-07T07:43:30.542 INFO:tasks.workunit.client.0.smithi160.stdout:cpuhp/1 22 root rtd unknown /proc/22/root (readlink: Permission denied) 2024-03-07T07:43:30.542 INFO:tasks.workunit.client.0.smithi160.stdout:cpuhp/1 22 root txt unknown /proc/22/exe (readlink: Permission denied) 2024-03-07T07:43:30.543 INFO:tasks.workunit.client.0.smithi160.stdout:cpuhp/1 22 root NOFD /proc/22/fd (opendir: Permission denied) 2024-03-07T07:43:30.543 INFO:tasks.workunit.client.0.smithi160.stdout:idle_inje 23 root cwd unknown /proc/23/cwd (readlink: Permission denied) 2024-03-07T07:43:30.543 INFO:tasks.workunit.client.0.smithi160.stdout:idle_inje 23 root rtd unknown /proc/23/root (readlink: Permission denied) 2024-03-07T07:43:30.543 INFO:tasks.workunit.client.0.smithi160.stdout:idle_inje 23 root txt unknown /proc/23/exe (readlink: Permission denied) 2024-03-07T07:43:30.543 INFO:tasks.workunit.client.0.smithi160.stdout:idle_inje 23 root NOFD /proc/23/fd (opendir: Permission denied) 2024-03-07T07:43:30.543 INFO:tasks.workunit.client.0.smithi160.stdout:migration 24 root cwd unknown /proc/24/cwd (readlink: Permission denied) 2024-03-07T07:43:30.543 INFO:tasks.workunit.client.0.smithi160.stdout:migration 24 root rtd unknown /proc/24/root (readlink: Permission denied) 2024-03-07T07:43:30.543 INFO:tasks.workunit.client.0.smithi160.stdout:migration 24 root txt unknown /proc/24/exe (readlink: Permission denied) 2024-03-07T07:43:30.543 INFO:tasks.workunit.client.0.smithi160.stdout:migration 24 root NOFD /proc/24/fd (opendir: Permission denied) 2024-03-07T07:43:30.543 INFO:tasks.workunit.client.0.smithi160.stdout:ksoftirqd 25 root cwd unknown /proc/25/cwd (readlink: Permission denied) 2024-03-07T07:43:30.543 INFO:tasks.workunit.client.0.smithi160.stdout:ksoftirqd 25 root rtd unknown /proc/25/root (readlink: Permission denied) 2024-03-07T07:43:30.543 INFO:tasks.workunit.client.0.smithi160.stdout:ksoftirqd 25 root txt unknown /proc/25/exe (readlink: Permission denied) 2024-03-07T07:43:30.543 INFO:tasks.workunit.client.0.smithi160.stdout:ksoftirqd 25 root NOFD /proc/25/fd (opendir: Permission denied) 2024-03-07T07:43:30.543 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/1 27 root cwd unknown /proc/27/cwd (readlink: Permission denied) 2024-03-07T07:43:30.544 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/1 27 root rtd unknown /proc/27/root (readlink: Permission denied) 2024-03-07T07:43:30.544 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/1 27 root txt unknown /proc/27/exe (readlink: Permission denied) 2024-03-07T07:43:30.544 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/1 27 root NOFD /proc/27/fd (opendir: Permission denied) 2024-03-07T07:43:30.544 INFO:tasks.workunit.client.0.smithi160.stdout:cpuhp/2 28 root cwd unknown /proc/28/cwd (readlink: Permission denied) 2024-03-07T07:43:30.544 INFO:tasks.workunit.client.0.smithi160.stdout:cpuhp/2 28 root rtd unknown /proc/28/root (readlink: Permission denied) 2024-03-07T07:43:30.544 INFO:tasks.workunit.client.0.smithi160.stdout:cpuhp/2 28 root txt unknown /proc/28/exe (readlink: Permission denied) 2024-03-07T07:43:30.544 INFO:tasks.workunit.client.0.smithi160.stdout:cpuhp/2 28 root NOFD /proc/28/fd (opendir: Permission denied) 2024-03-07T07:43:30.544 INFO:tasks.workunit.client.0.smithi160.stdout:idle_inje 29 root cwd unknown /proc/29/cwd (readlink: Permission denied) 2024-03-07T07:43:30.544 INFO:tasks.workunit.client.0.smithi160.stdout:idle_inje 29 root rtd unknown /proc/29/root (readlink: Permission denied) 2024-03-07T07:43:30.544 INFO:tasks.workunit.client.0.smithi160.stdout:idle_inje 29 root txt unknown /proc/29/exe (readlink: Permission denied) 2024-03-07T07:43:30.544 INFO:tasks.workunit.client.0.smithi160.stdout:idle_inje 29 root NOFD /proc/29/fd (opendir: Permission denied) 2024-03-07T07:43:30.544 INFO:tasks.workunit.client.0.smithi160.stdout:migration 30 root cwd unknown /proc/30/cwd (readlink: Permission denied) 2024-03-07T07:43:30.544 INFO:tasks.workunit.client.0.smithi160.stdout:migration 30 root rtd unknown /proc/30/root (readlink: Permission denied) 2024-03-07T07:43:30.544 INFO:tasks.workunit.client.0.smithi160.stdout:migration 30 root txt unknown /proc/30/exe (readlink: Permission denied) 2024-03-07T07:43:30.545 INFO:tasks.workunit.client.0.smithi160.stdout:migration 30 root NOFD /proc/30/fd (opendir: Permission denied) 2024-03-07T07:43:30.545 INFO:tasks.workunit.client.0.smithi160.stdout:ksoftirqd 31 root cwd unknown /proc/31/cwd (readlink: Permission denied) 2024-03-07T07:43:30.545 INFO:tasks.workunit.client.0.smithi160.stdout:ksoftirqd 31 root rtd unknown /proc/31/root (readlink: Permission denied) 2024-03-07T07:43:30.545 INFO:tasks.workunit.client.0.smithi160.stdout:ksoftirqd 31 root txt unknown /proc/31/exe (readlink: Permission denied) 2024-03-07T07:43:30.545 INFO:tasks.workunit.client.0.smithi160.stdout:ksoftirqd 31 root NOFD /proc/31/fd (opendir: Permission denied) 2024-03-07T07:43:30.545 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/2 33 root cwd unknown /proc/33/cwd (readlink: Permission denied) 2024-03-07T07:43:30.545 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/2 33 root rtd unknown /proc/33/root (readlink: Permission denied) 2024-03-07T07:43:30.545 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/2 33 root txt unknown /proc/33/exe (readlink: Permission denied) 2024-03-07T07:43:30.545 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/2 33 root NOFD /proc/33/fd (opendir: Permission denied) 2024-03-07T07:43:30.545 INFO:tasks.workunit.client.0.smithi160.stdout:cpuhp/3 34 root cwd unknown /proc/34/cwd (readlink: Permission denied) 2024-03-07T07:43:30.545 INFO:tasks.workunit.client.0.smithi160.stdout:cpuhp/3 34 root rtd unknown /proc/34/root (readlink: Permission denied) 2024-03-07T07:43:30.545 INFO:tasks.workunit.client.0.smithi160.stdout:cpuhp/3 34 root txt unknown /proc/34/exe (readlink: Permission denied) 2024-03-07T07:43:30.545 INFO:tasks.workunit.client.0.smithi160.stdout:cpuhp/3 34 root NOFD /proc/34/fd (opendir: Permission denied) 2024-03-07T07:43:30.546 INFO:tasks.workunit.client.0.smithi160.stdout:idle_inje 35 root cwd unknown /proc/35/cwd (readlink: Permission denied) 2024-03-07T07:43:30.546 INFO:tasks.workunit.client.0.smithi160.stdout:idle_inje 35 root rtd unknown /proc/35/root (readlink: Permission denied) 2024-03-07T07:43:30.546 INFO:tasks.workunit.client.0.smithi160.stdout:idle_inje 35 root txt unknown /proc/35/exe (readlink: Permission denied) 2024-03-07T07:43:30.546 INFO:tasks.workunit.client.0.smithi160.stdout:idle_inje 35 root NOFD /proc/35/fd (opendir: Permission denied) 2024-03-07T07:43:30.546 INFO:tasks.workunit.client.0.smithi160.stdout:migration 36 root cwd unknown /proc/36/cwd (readlink: Permission denied) 2024-03-07T07:43:30.546 INFO:tasks.workunit.client.0.smithi160.stdout:migration 36 root rtd unknown /proc/36/root (readlink: Permission denied) 2024-03-07T07:43:30.546 INFO:tasks.workunit.client.0.smithi160.stdout:migration 36 root txt unknown /proc/36/exe (readlink: Permission denied) 2024-03-07T07:43:30.546 INFO:tasks.workunit.client.0.smithi160.stdout:migration 36 root NOFD /proc/36/fd (opendir: Permission denied) 2024-03-07T07:43:30.546 INFO:tasks.workunit.client.0.smithi160.stdout:ksoftirqd 37 root cwd unknown /proc/37/cwd (readlink: Permission denied) 2024-03-07T07:43:30.546 INFO:tasks.workunit.client.0.smithi160.stdout:ksoftirqd 37 root rtd unknown /proc/37/root (readlink: Permission denied) 2024-03-07T07:43:30.546 INFO:tasks.workunit.client.0.smithi160.stdout:ksoftirqd 37 root txt unknown /proc/37/exe (readlink: Permission denied) 2024-03-07T07:43:30.546 INFO:tasks.workunit.client.0.smithi160.stdout:ksoftirqd 37 root NOFD /proc/37/fd (opendir: Permission denied) 2024-03-07T07:43:30.547 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/3 39 root cwd unknown /proc/39/cwd (readlink: Permission denied) 2024-03-07T07:43:30.547 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/3 39 root rtd unknown /proc/39/root (readlink: Permission denied) 2024-03-07T07:43:30.547 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/3 39 root txt unknown /proc/39/exe (readlink: Permission denied) 2024-03-07T07:43:30.547 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/3 39 root NOFD /proc/39/fd (opendir: Permission denied) 2024-03-07T07:43:30.547 INFO:tasks.workunit.client.0.smithi160.stdout:cpuhp/4 40 root cwd unknown /proc/40/cwd (readlink: Permission denied) 2024-03-07T07:43:30.547 INFO:tasks.workunit.client.0.smithi160.stdout:cpuhp/4 40 root rtd unknown /proc/40/root (readlink: Permission denied) 2024-03-07T07:43:30.547 INFO:tasks.workunit.client.0.smithi160.stdout:cpuhp/4 40 root txt unknown /proc/40/exe (readlink: Permission denied) 2024-03-07T07:43:30.547 INFO:tasks.workunit.client.0.smithi160.stdout:cpuhp/4 40 root NOFD /proc/40/fd (opendir: Permission denied) 2024-03-07T07:43:30.547 INFO:tasks.workunit.client.0.smithi160.stdout:idle_inje 41 root cwd unknown /proc/41/cwd (readlink: Permission denied) 2024-03-07T07:43:30.547 INFO:tasks.workunit.client.0.smithi160.stdout:idle_inje 41 root rtd unknown /proc/41/root (readlink: Permission denied) 2024-03-07T07:43:30.547 INFO:tasks.workunit.client.0.smithi160.stdout:idle_inje 41 root txt unknown /proc/41/exe (readlink: Permission denied) 2024-03-07T07:43:30.547 INFO:tasks.workunit.client.0.smithi160.stdout:idle_inje 41 root NOFD /proc/41/fd (opendir: Permission denied) 2024-03-07T07:43:30.547 INFO:tasks.workunit.client.0.smithi160.stdout:migration 42 root cwd unknown /proc/42/cwd (readlink: Permission denied) 2024-03-07T07:43:30.547 INFO:tasks.workunit.client.0.smithi160.stdout:migration 42 root rtd unknown /proc/42/root (readlink: Permission denied) 2024-03-07T07:43:30.548 INFO:tasks.workunit.client.0.smithi160.stdout:migration 42 root txt unknown /proc/42/exe (readlink: Permission denied) 2024-03-07T07:43:30.548 INFO:tasks.workunit.client.0.smithi160.stdout:migration 42 root NOFD /proc/42/fd (opendir: Permission denied) 2024-03-07T07:43:30.548 INFO:tasks.workunit.client.0.smithi160.stdout:ksoftirqd 43 root cwd unknown /proc/43/cwd (readlink: Permission denied) 2024-03-07T07:43:30.548 INFO:tasks.workunit.client.0.smithi160.stdout:ksoftirqd 43 root rtd unknown /proc/43/root (readlink: Permission denied) 2024-03-07T07:43:30.548 INFO:tasks.workunit.client.0.smithi160.stdout:ksoftirqd 43 root txt unknown /proc/43/exe (readlink: Permission denied) 2024-03-07T07:43:30.548 INFO:tasks.workunit.client.0.smithi160.stdout:ksoftirqd 43 root NOFD /proc/43/fd (opendir: Permission denied) 2024-03-07T07:43:30.548 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/4 45 root cwd unknown /proc/45/cwd (readlink: Permission denied) 2024-03-07T07:43:30.548 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/4 45 root rtd unknown /proc/45/root (readlink: Permission denied) 2024-03-07T07:43:30.548 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/4 45 root txt unknown /proc/45/exe (readlink: Permission denied) 2024-03-07T07:43:30.548 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/4 45 root NOFD /proc/45/fd (opendir: Permission denied) 2024-03-07T07:43:30.548 INFO:tasks.workunit.client.0.smithi160.stdout:cpuhp/5 46 root cwd unknown /proc/46/cwd (readlink: Permission denied) 2024-03-07T07:43:30.548 INFO:tasks.workunit.client.0.smithi160.stdout:cpuhp/5 46 root rtd unknown /proc/46/root (readlink: Permission denied) 2024-03-07T07:43:30.548 INFO:tasks.workunit.client.0.smithi160.stdout:cpuhp/5 46 root txt unknown /proc/46/exe (readlink: Permission denied) 2024-03-07T07:43:30.549 INFO:tasks.workunit.client.0.smithi160.stdout:cpuhp/5 46 root NOFD /proc/46/fd (opendir: Permission denied) 2024-03-07T07:43:30.549 INFO:tasks.workunit.client.0.smithi160.stdout:idle_inje 47 root cwd unknown /proc/47/cwd (readlink: Permission denied) 2024-03-07T07:43:30.549 INFO:tasks.workunit.client.0.smithi160.stdout:idle_inje 47 root rtd unknown /proc/47/root (readlink: Permission denied) 2024-03-07T07:43:30.549 INFO:tasks.workunit.client.0.smithi160.stdout:idle_inje 47 root txt unknown /proc/47/exe (readlink: Permission denied) 2024-03-07T07:43:30.549 INFO:tasks.workunit.client.0.smithi160.stdout:idle_inje 47 root NOFD /proc/47/fd (opendir: Permission denied) 2024-03-07T07:43:30.549 INFO:tasks.workunit.client.0.smithi160.stdout:migration 48 root cwd unknown /proc/48/cwd (readlink: Permission denied) 2024-03-07T07:43:30.549 INFO:tasks.workunit.client.0.smithi160.stdout:migration 48 root rtd unknown /proc/48/root (readlink: Permission denied) 2024-03-07T07:43:30.549 INFO:tasks.workunit.client.0.smithi160.stdout:migration 48 root txt unknown /proc/48/exe (readlink: Permission denied) 2024-03-07T07:43:30.549 INFO:tasks.workunit.client.0.smithi160.stdout:migration 48 root NOFD /proc/48/fd (opendir: Permission denied) 2024-03-07T07:43:30.550 INFO:tasks.workunit.client.0.smithi160.stdout:ksoftirqd 49 root cwd unknown /proc/49/cwd (readlink: Permission denied) 2024-03-07T07:43:30.550 INFO:tasks.workunit.client.0.smithi160.stdout:ksoftirqd 49 root rtd unknown /proc/49/root (readlink: Permission denied) 2024-03-07T07:43:30.550 INFO:tasks.workunit.client.0.smithi160.stdout:ksoftirqd 49 root txt unknown /proc/49/exe (readlink: Permission denied) 2024-03-07T07:43:30.550 INFO:tasks.workunit.client.0.smithi160.stdout:ksoftirqd 49 root NOFD /proc/49/fd (opendir: Permission denied) 2024-03-07T07:43:30.550 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/5 51 root cwd unknown /proc/51/cwd (readlink: Permission denied) 2024-03-07T07:43:30.550 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/5 51 root rtd unknown /proc/51/root (readlink: Permission denied) 2024-03-07T07:43:30.550 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/5 51 root txt unknown /proc/51/exe (readlink: Permission denied) 2024-03-07T07:43:30.550 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/5 51 root NOFD /proc/51/fd (opendir: Permission denied) 2024-03-07T07:43:30.550 INFO:tasks.workunit.client.0.smithi160.stdout:cpuhp/6 52 root cwd unknown /proc/52/cwd (readlink: Permission denied) 2024-03-07T07:43:30.550 INFO:tasks.workunit.client.0.smithi160.stdout:cpuhp/6 52 root rtd unknown /proc/52/root (readlink: Permission denied) 2024-03-07T07:43:30.550 INFO:tasks.workunit.client.0.smithi160.stdout:cpuhp/6 52 root txt unknown /proc/52/exe (readlink: Permission denied) 2024-03-07T07:43:30.550 INFO:tasks.workunit.client.0.smithi160.stdout:cpuhp/6 52 root NOFD /proc/52/fd (opendir: Permission denied) 2024-03-07T07:43:30.550 INFO:tasks.workunit.client.0.smithi160.stdout:idle_inje 53 root cwd unknown /proc/53/cwd (readlink: Permission denied) 2024-03-07T07:43:30.551 INFO:tasks.workunit.client.0.smithi160.stdout:idle_inje 53 root rtd unknown /proc/53/root (readlink: Permission denied) 2024-03-07T07:43:30.551 INFO:tasks.workunit.client.0.smithi160.stdout:idle_inje 53 root txt unknown /proc/53/exe (readlink: Permission denied) 2024-03-07T07:43:30.551 INFO:tasks.workunit.client.0.smithi160.stdout:idle_inje 53 root NOFD /proc/53/fd (opendir: Permission denied) 2024-03-07T07:43:30.551 INFO:tasks.workunit.client.0.smithi160.stdout:migration 54 root cwd unknown /proc/54/cwd (readlink: Permission denied) 2024-03-07T07:43:30.551 INFO:tasks.workunit.client.0.smithi160.stdout:migration 54 root rtd unknown /proc/54/root (readlink: Permission denied) 2024-03-07T07:43:30.551 INFO:tasks.workunit.client.0.smithi160.stdout:migration 54 root txt unknown /proc/54/exe (readlink: Permission denied) 2024-03-07T07:43:30.551 INFO:tasks.workunit.client.0.smithi160.stdout:migration 54 root NOFD /proc/54/fd (opendir: Permission denied) 2024-03-07T07:43:30.551 INFO:tasks.workunit.client.0.smithi160.stdout:ksoftirqd 55 root cwd unknown /proc/55/cwd (readlink: Permission denied) 2024-03-07T07:43:30.551 INFO:tasks.workunit.client.0.smithi160.stdout:ksoftirqd 55 root rtd unknown /proc/55/root (readlink: Permission denied) 2024-03-07T07:43:30.551 INFO:tasks.workunit.client.0.smithi160.stdout:ksoftirqd 55 root txt unknown /proc/55/exe (readlink: Permission denied) 2024-03-07T07:43:30.551 INFO:tasks.workunit.client.0.smithi160.stdout:ksoftirqd 55 root NOFD /proc/55/fd (opendir: Permission denied) 2024-03-07T07:43:30.551 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/6 57 root cwd unknown /proc/57/cwd (readlink: Permission denied) 2024-03-07T07:43:30.551 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/6 57 root rtd unknown /proc/57/root (readlink: Permission denied) 2024-03-07T07:43:30.551 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/6 57 root txt unknown /proc/57/exe (readlink: Permission denied) 2024-03-07T07:43:30.552 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/6 57 root NOFD /proc/57/fd (opendir: Permission denied) 2024-03-07T07:43:30.552 INFO:tasks.workunit.client.0.smithi160.stdout:cpuhp/7 58 root cwd unknown /proc/58/cwd (readlink: Permission denied) 2024-03-07T07:43:30.552 INFO:tasks.workunit.client.0.smithi160.stdout:cpuhp/7 58 root rtd unknown /proc/58/root (readlink: Permission denied) 2024-03-07T07:43:30.552 INFO:tasks.workunit.client.0.smithi160.stdout:cpuhp/7 58 root txt unknown /proc/58/exe (readlink: Permission denied) 2024-03-07T07:43:30.552 INFO:tasks.workunit.client.0.smithi160.stdout:cpuhp/7 58 root NOFD /proc/58/fd (opendir: Permission denied) 2024-03-07T07:43:30.552 INFO:tasks.workunit.client.0.smithi160.stdout:idle_inje 59 root cwd unknown /proc/59/cwd (readlink: Permission denied) 2024-03-07T07:43:30.552 INFO:tasks.workunit.client.0.smithi160.stdout:idle_inje 59 root rtd unknown /proc/59/root (readlink: Permission denied) 2024-03-07T07:43:30.552 INFO:tasks.workunit.client.0.smithi160.stdout:idle_inje 59 root txt unknown /proc/59/exe (readlink: Permission denied) 2024-03-07T07:43:30.552 INFO:tasks.workunit.client.0.smithi160.stdout:idle_inje 59 root NOFD /proc/59/fd (opendir: Permission denied) 2024-03-07T07:43:30.552 INFO:tasks.workunit.client.0.smithi160.stdout:migration 60 root cwd unknown /proc/60/cwd (readlink: Permission denied) 2024-03-07T07:43:30.552 INFO:tasks.workunit.client.0.smithi160.stdout:migration 60 root rtd unknown /proc/60/root (readlink: Permission denied) 2024-03-07T07:43:30.552 INFO:tasks.workunit.client.0.smithi160.stdout:migration 60 root txt unknown /proc/60/exe (readlink: Permission denied) 2024-03-07T07:43:30.552 INFO:tasks.workunit.client.0.smithi160.stdout:migration 60 root NOFD /proc/60/fd (opendir: Permission denied) 2024-03-07T07:43:30.552 INFO:tasks.workunit.client.0.smithi160.stdout:ksoftirqd 61 root cwd unknown /proc/61/cwd (readlink: Permission denied) 2024-03-07T07:43:30.553 INFO:tasks.workunit.client.0.smithi160.stdout:ksoftirqd 61 root rtd unknown /proc/61/root (readlink: Permission denied) 2024-03-07T07:43:30.553 INFO:tasks.workunit.client.0.smithi160.stdout:ksoftirqd 61 root txt unknown /proc/61/exe (readlink: Permission denied) 2024-03-07T07:43:30.553 INFO:tasks.workunit.client.0.smithi160.stdout:ksoftirqd 61 root NOFD /proc/61/fd (opendir: Permission denied) 2024-03-07T07:43:30.553 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/7 63 root cwd unknown /proc/63/cwd (readlink: Permission denied) 2024-03-07T07:43:30.553 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/7 63 root rtd unknown /proc/63/root (readlink: Permission denied) 2024-03-07T07:43:30.553 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/7 63 root txt unknown /proc/63/exe (readlink: Permission denied) 2024-03-07T07:43:30.553 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/7 63 root NOFD /proc/63/fd (opendir: Permission denied) 2024-03-07T07:43:30.553 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/u 65 root cwd unknown /proc/65/cwd (readlink: Permission denied) 2024-03-07T07:43:30.553 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/u 65 root rtd unknown /proc/65/root (readlink: Permission denied) 2024-03-07T07:43:30.553 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/u 65 root txt unknown /proc/65/exe (readlink: Permission denied) 2024-03-07T07:43:30.553 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/u 65 root NOFD /proc/65/fd (opendir: Permission denied) 2024-03-07T07:43:30.553 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/u 66 root cwd unknown /proc/66/cwd (readlink: Permission denied) 2024-03-07T07:43:30.553 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/u 66 root rtd unknown /proc/66/root (readlink: Permission denied) 2024-03-07T07:43:30.553 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/u 66 root txt unknown /proc/66/exe (readlink: Permission denied) 2024-03-07T07:43:30.554 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/u 66 root NOFD /proc/66/fd (opendir: Permission denied) 2024-03-07T07:43:30.554 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/u 69 root cwd unknown /proc/69/cwd (readlink: Permission denied) 2024-03-07T07:43:30.554 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/u 69 root rtd unknown /proc/69/root (readlink: Permission denied) 2024-03-07T07:43:30.554 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/u 69 root txt unknown /proc/69/exe (readlink: Permission denied) 2024-03-07T07:43:30.554 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/u 69 root NOFD /proc/69/fd (opendir: Permission denied) 2024-03-07T07:43:30.554 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/u 70 root cwd unknown /proc/70/cwd (readlink: Permission denied) 2024-03-07T07:43:30.554 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/u 70 root rtd unknown /proc/70/root (readlink: Permission denied) 2024-03-07T07:43:30.554 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/u 70 root txt unknown /proc/70/exe (readlink: Permission denied) 2024-03-07T07:43:30.554 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/u 70 root NOFD /proc/70/fd (opendir: Permission denied) 2024-03-07T07:43:30.554 INFO:tasks.workunit.client.0.smithi160.stdout:kdevtmpfs 71 root cwd unknown /proc/71/cwd (readlink: Permission denied) 2024-03-07T07:43:30.554 INFO:tasks.workunit.client.0.smithi160.stdout:kdevtmpfs 71 root rtd unknown /proc/71/root (readlink: Permission denied) 2024-03-07T07:43:30.554 INFO:tasks.workunit.client.0.smithi160.stdout:kdevtmpfs 71 root txt unknown /proc/71/exe (readlink: Permission denied) 2024-03-07T07:43:30.554 INFO:tasks.workunit.client.0.smithi160.stdout:kdevtmpfs 71 root NOFD /proc/71/fd (opendir: Permission denied) 2024-03-07T07:43:30.554 INFO:tasks.workunit.client.0.smithi160.stdout:inet_frag 72 root cwd unknown /proc/72/cwd (readlink: Permission denied) 2024-03-07T07:43:30.555 INFO:tasks.workunit.client.0.smithi160.stdout:inet_frag 72 root rtd unknown /proc/72/root (readlink: Permission denied) 2024-03-07T07:43:30.555 INFO:tasks.workunit.client.0.smithi160.stdout:inet_frag 72 root txt unknown /proc/72/exe (readlink: Permission denied) 2024-03-07T07:43:30.555 INFO:tasks.workunit.client.0.smithi160.stdout:inet_frag 72 root NOFD /proc/72/fd (opendir: Permission denied) 2024-03-07T07:43:30.555 INFO:tasks.workunit.client.0.smithi160.stdout:kauditd 73 root cwd unknown /proc/73/cwd (readlink: Permission denied) 2024-03-07T07:43:30.555 INFO:tasks.workunit.client.0.smithi160.stdout:kauditd 73 root rtd unknown /proc/73/root (readlink: Permission denied) 2024-03-07T07:43:30.555 INFO:tasks.workunit.client.0.smithi160.stdout:kauditd 73 root txt unknown /proc/73/exe (readlink: Permission denied) 2024-03-07T07:43:30.555 INFO:tasks.workunit.client.0.smithi160.stdout:kauditd 73 root NOFD /proc/73/fd (opendir: Permission denied) 2024-03-07T07:43:30.555 INFO:tasks.workunit.client.0.smithi160.stdout:khungtask 75 root cwd unknown /proc/75/cwd (readlink: Permission denied) 2024-03-07T07:43:30.555 INFO:tasks.workunit.client.0.smithi160.stdout:khungtask 75 root rtd unknown /proc/75/root (readlink: Permission denied) 2024-03-07T07:43:30.555 INFO:tasks.workunit.client.0.smithi160.stdout:khungtask 75 root txt unknown /proc/75/exe (readlink: Permission denied) 2024-03-07T07:43:30.555 INFO:tasks.workunit.client.0.smithi160.stdout:khungtask 75 root NOFD /proc/75/fd (opendir: Permission denied) 2024-03-07T07:43:30.555 INFO:tasks.workunit.client.0.smithi160.stdout:oom_reape 76 root cwd unknown /proc/76/cwd (readlink: Permission denied) 2024-03-07T07:43:30.555 INFO:tasks.workunit.client.0.smithi160.stdout:oom_reape 76 root rtd unknown /proc/76/root (readlink: Permission denied) 2024-03-07T07:43:30.555 INFO:tasks.workunit.client.0.smithi160.stdout:oom_reape 76 root txt unknown /proc/76/exe (readlink: Permission denied) 2024-03-07T07:43:30.556 INFO:tasks.workunit.client.0.smithi160.stdout:oom_reape 76 root NOFD /proc/76/fd (opendir: Permission denied) 2024-03-07T07:43:30.556 INFO:tasks.workunit.client.0.smithi160.stdout:writeback 77 root cwd unknown /proc/77/cwd (readlink: Permission denied) 2024-03-07T07:43:30.556 INFO:tasks.workunit.client.0.smithi160.stdout:writeback 77 root rtd unknown /proc/77/root (readlink: Permission denied) 2024-03-07T07:43:30.556 INFO:tasks.workunit.client.0.smithi160.stdout:writeback 77 root txt unknown /proc/77/exe (readlink: Permission denied) 2024-03-07T07:43:30.556 INFO:tasks.workunit.client.0.smithi160.stdout:writeback 77 root NOFD /proc/77/fd (opendir: Permission denied) 2024-03-07T07:43:30.556 INFO:tasks.workunit.client.0.smithi160.stdout:kcompactd 78 root cwd unknown /proc/78/cwd (readlink: Permission denied) 2024-03-07T07:43:30.556 INFO:tasks.workunit.client.0.smithi160.stdout:kcompactd 78 root rtd unknown /proc/78/root (readlink: Permission denied) 2024-03-07T07:43:30.556 INFO:tasks.workunit.client.0.smithi160.stdout:kcompactd 78 root txt unknown /proc/78/exe (readlink: Permission denied) 2024-03-07T07:43:30.556 INFO:tasks.workunit.client.0.smithi160.stdout:kcompactd 78 root NOFD /proc/78/fd (opendir: Permission denied) 2024-03-07T07:43:30.556 INFO:tasks.workunit.client.0.smithi160.stdout:ksmd 79 root cwd unknown /proc/79/cwd (readlink: Permission denied) 2024-03-07T07:43:30.556 INFO:tasks.workunit.client.0.smithi160.stdout:ksmd 79 root rtd unknown /proc/79/root (readlink: Permission denied) 2024-03-07T07:43:30.556 INFO:tasks.workunit.client.0.smithi160.stdout:ksmd 79 root txt unknown /proc/79/exe (readlink: Permission denied) 2024-03-07T07:43:30.556 INFO:tasks.workunit.client.0.smithi160.stdout:ksmd 79 root NOFD /proc/79/fd (opendir: Permission denied) 2024-03-07T07:43:30.556 INFO:tasks.workunit.client.0.smithi160.stdout:khugepage 80 root cwd unknown /proc/80/cwd (readlink: Permission denied) 2024-03-07T07:43:30.557 INFO:tasks.workunit.client.0.smithi160.stdout:khugepage 80 root rtd unknown /proc/80/root (readlink: Permission denied) 2024-03-07T07:43:30.557 INFO:tasks.workunit.client.0.smithi160.stdout:khugepage 80 root txt unknown /proc/80/exe (readlink: Permission denied) 2024-03-07T07:43:30.557 INFO:tasks.workunit.client.0.smithi160.stdout:khugepage 80 root NOFD /proc/80/fd (opendir: Permission denied) 2024-03-07T07:43:30.557 INFO:tasks.workunit.client.0.smithi160.stdout:cryptd 81 root cwd unknown /proc/81/cwd (readlink: Permission denied) 2024-03-07T07:43:30.557 INFO:tasks.workunit.client.0.smithi160.stdout:cryptd 81 root rtd unknown /proc/81/root (readlink: Permission denied) 2024-03-07T07:43:30.557 INFO:tasks.workunit.client.0.smithi160.stdout:cryptd 81 root txt unknown /proc/81/exe (readlink: Permission denied) 2024-03-07T07:43:30.557 INFO:tasks.workunit.client.0.smithi160.stdout:cryptd 81 root NOFD /proc/81/fd (opendir: Permission denied) 2024-03-07T07:43:30.559 INFO:tasks.workunit.client.0.smithi160.stdout:kintegrit 82 root cwd unknown /proc/82/cwd (readlink: Permission denied) 2024-03-07T07:43:30.559 INFO:tasks.workunit.client.0.smithi160.stdout:kintegrit 82 root rtd unknown /proc/82/root (readlink: Permission denied) 2024-03-07T07:43:30.559 INFO:tasks.workunit.client.0.smithi160.stdout:kintegrit 82 root txt unknown /proc/82/exe (readlink: Permission denied) 2024-03-07T07:43:30.559 INFO:tasks.workunit.client.0.smithi160.stdout:kintegrit 82 root NOFD /proc/82/fd (opendir: Permission denied) 2024-03-07T07:43:30.559 INFO:tasks.workunit.client.0.smithi160.stdout:kblockd 83 root cwd unknown /proc/83/cwd (readlink: Permission denied) 2024-03-07T07:43:30.559 INFO:tasks.workunit.client.0.smithi160.stdout:kblockd 83 root rtd unknown /proc/83/root (readlink: Permission denied) 2024-03-07T07:43:30.559 INFO:tasks.workunit.client.0.smithi160.stdout:kblockd 83 root txt unknown /proc/83/exe (readlink: Permission denied) 2024-03-07T07:43:30.559 INFO:tasks.workunit.client.0.smithi160.stdout:kblockd 83 root NOFD /proc/83/fd (opendir: Permission denied) 2024-03-07T07:43:30.559 INFO:tasks.workunit.client.0.smithi160.stdout:blkcg_pun 84 root cwd unknown /proc/84/cwd (readlink: Permission denied) 2024-03-07T07:43:30.559 INFO:tasks.workunit.client.0.smithi160.stdout:blkcg_pun 84 root rtd unknown /proc/84/root (readlink: Permission denied) 2024-03-07T07:43:30.559 INFO:tasks.workunit.client.0.smithi160.stdout:blkcg_pun 84 root txt unknown /proc/84/exe (readlink: Permission denied) 2024-03-07T07:43:30.560 INFO:tasks.workunit.client.0.smithi160.stdout:blkcg_pun 84 root NOFD /proc/84/fd (opendir: Permission denied) 2024-03-07T07:43:30.560 INFO:tasks.workunit.client.0.smithi160.stdout:tpm_dev_w 86 root cwd unknown /proc/86/cwd (readlink: Permission denied) 2024-03-07T07:43:30.560 INFO:tasks.workunit.client.0.smithi160.stdout:tpm_dev_w 86 root rtd unknown /proc/86/root (readlink: Permission denied) 2024-03-07T07:43:30.560 INFO:tasks.workunit.client.0.smithi160.stdout:tpm_dev_w 86 root txt unknown /proc/86/exe (readlink: Permission denied) 2024-03-07T07:43:30.560 INFO:tasks.workunit.client.0.smithi160.stdout:tpm_dev_w 86 root NOFD /proc/86/fd (opendir: Permission denied) 2024-03-07T07:43:30.560 INFO:tasks.workunit.client.0.smithi160.stdout:md 87 root cwd unknown /proc/87/cwd (readlink: Permission denied) 2024-03-07T07:43:30.560 INFO:tasks.workunit.client.0.smithi160.stdout:md 87 root rtd unknown /proc/87/root (readlink: Permission denied) 2024-03-07T07:43:30.560 INFO:tasks.workunit.client.0.smithi160.stdout:md 87 root txt unknown /proc/87/exe (readlink: Permission denied) 2024-03-07T07:43:30.560 INFO:tasks.workunit.client.0.smithi160.stdout:md 87 root NOFD /proc/87/fd (opendir: Permission denied) 2024-03-07T07:43:30.560 INFO:tasks.workunit.client.0.smithi160.stdout:md_bitmap 88 root cwd unknown /proc/88/cwd (readlink: Permission denied) 2024-03-07T07:43:30.560 INFO:tasks.workunit.client.0.smithi160.stdout:md_bitmap 88 root rtd unknown /proc/88/root (readlink: Permission denied) 2024-03-07T07:43:30.560 INFO:tasks.workunit.client.0.smithi160.stdout:md_bitmap 88 root txt unknown /proc/88/exe (readlink: Permission denied) 2024-03-07T07:43:30.560 INFO:tasks.workunit.client.0.smithi160.stdout:md_bitmap 88 root NOFD /proc/88/fd (opendir: Permission denied) 2024-03-07T07:43:30.560 INFO:tasks.workunit.client.0.smithi160.stdout:edac-poll 89 root cwd unknown /proc/89/cwd (readlink: Permission denied) 2024-03-07T07:43:30.561 INFO:tasks.workunit.client.0.smithi160.stdout:edac-poll 89 root rtd unknown /proc/89/root (readlink: Permission denied) 2024-03-07T07:43:30.561 INFO:tasks.workunit.client.0.smithi160.stdout:edac-poll 89 root txt unknown /proc/89/exe (readlink: Permission denied) 2024-03-07T07:43:30.561 INFO:tasks.workunit.client.0.smithi160.stdout:edac-poll 89 root NOFD /proc/89/fd (opendir: Permission denied) 2024-03-07T07:43:30.561 INFO:tasks.workunit.client.0.smithi160.stdout:watchdogd 90 root cwd unknown /proc/90/cwd (readlink: Permission denied) 2024-03-07T07:43:30.561 INFO:tasks.workunit.client.0.smithi160.stdout:watchdogd 90 root rtd unknown /proc/90/root (readlink: Permission denied) 2024-03-07T07:43:30.561 INFO:tasks.workunit.client.0.smithi160.stdout:watchdogd 90 root txt unknown /proc/90/exe (readlink: Permission denied) 2024-03-07T07:43:30.561 INFO:tasks.workunit.client.0.smithi160.stdout:watchdogd 90 root NOFD /proc/90/fd (opendir: Permission denied) 2024-03-07T07:43:30.561 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/2 91 root cwd unknown /proc/91/cwd (readlink: Permission denied) 2024-03-07T07:43:30.561 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/2 91 root rtd unknown /proc/91/root (readlink: Permission denied) 2024-03-07T07:43:30.561 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/2 91 root txt unknown /proc/91/exe (readlink: Permission denied) 2024-03-07T07:43:30.561 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/2 91 root NOFD /proc/91/fd (opendir: Permission denied) 2024-03-07T07:43:30.561 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/2 92 root cwd unknown /proc/92/cwd (readlink: Permission denied) 2024-03-07T07:43:30.561 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/2 92 root rtd unknown /proc/92/root (readlink: Permission denied) 2024-03-07T07:43:30.561 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/2 92 root txt unknown /proc/92/exe (readlink: Permission denied) 2024-03-07T07:43:30.562 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/2 92 root NOFD /proc/92/fd (opendir: Permission denied) 2024-03-07T07:43:30.562 INFO:tasks.workunit.client.0.smithi160.stdout:kswapd0 94 root cwd unknown /proc/94/cwd (readlink: Permission denied) 2024-03-07T07:43:30.562 INFO:tasks.workunit.client.0.smithi160.stdout:kswapd0 94 root rtd unknown /proc/94/root (readlink: Permission denied) 2024-03-07T07:43:30.562 INFO:tasks.workunit.client.0.smithi160.stdout:kswapd0 94 root txt unknown /proc/94/exe (readlink: Permission denied) 2024-03-07T07:43:30.562 INFO:tasks.workunit.client.0.smithi160.stdout:kswapd0 94 root NOFD /proc/94/fd (opendir: Permission denied) 2024-03-07T07:43:30.562 INFO:tasks.workunit.client.0.smithi160.stdout:kthrotld 101 root cwd unknown /proc/101/cwd (readlink: Permission denied) 2024-03-07T07:43:30.562 INFO:tasks.workunit.client.0.smithi160.stdout:kthrotld 101 root rtd unknown /proc/101/root (readlink: Permission denied) 2024-03-07T07:43:30.562 INFO:tasks.workunit.client.0.smithi160.stdout:kthrotld 101 root txt unknown /proc/101/exe (readlink: Permission denied) 2024-03-07T07:43:30.562 INFO:tasks.workunit.client.0.smithi160.stdout:kthrotld 101 root NOFD /proc/101/fd (opendir: Permission denied) 2024-03-07T07:43:30.562 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/3 108 root cwd unknown /proc/108/cwd (readlink: Permission denied) 2024-03-07T07:43:30.562 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/3 108 root rtd unknown /proc/108/root (readlink: Permission denied) 2024-03-07T07:43:30.562 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/3 108 root txt unknown /proc/108/exe (readlink: Permission denied) 2024-03-07T07:43:30.562 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/3 108 root NOFD /proc/108/fd (opendir: Permission denied) 2024-03-07T07:43:30.563 INFO:tasks.workunit.client.0.smithi160.stdout:acpi_ther 109 root cwd unknown /proc/109/cwd (readlink: Permission denied) 2024-03-07T07:43:30.563 INFO:tasks.workunit.client.0.smithi160.stdout:acpi_ther 109 root rtd unknown /proc/109/root (readlink: Permission denied) 2024-03-07T07:43:30.563 INFO:tasks.workunit.client.0.smithi160.stdout:acpi_ther 109 root txt unknown /proc/109/exe (readlink: Permission denied) 2024-03-07T07:43:30.563 INFO:tasks.workunit.client.0.smithi160.stdout:acpi_ther 109 root NOFD /proc/109/fd (opendir: Permission denied) 2024-03-07T07:43:30.563 INFO:tasks.workunit.client.0.smithi160.stdout:kmpath_rd 110 root cwd unknown /proc/110/cwd (readlink: Permission denied) 2024-03-07T07:43:30.563 INFO:tasks.workunit.client.0.smithi160.stdout:kmpath_rd 110 root rtd unknown /proc/110/root (readlink: Permission denied) 2024-03-07T07:43:30.563 INFO:tasks.workunit.client.0.smithi160.stdout:kmpath_rd 110 root txt unknown /proc/110/exe (readlink: Permission denied) 2024-03-07T07:43:30.563 INFO:tasks.workunit.client.0.smithi160.stdout:kmpath_rd 110 root NOFD /proc/110/fd (opendir: Permission denied) 2024-03-07T07:43:30.563 INFO:tasks.workunit.client.0.smithi160.stdout:kaluad 111 root cwd unknown /proc/111/cwd (readlink: Permission denied) 2024-03-07T07:43:30.563 INFO:tasks.workunit.client.0.smithi160.stdout:kaluad 111 root rtd unknown /proc/111/root (readlink: Permission denied) 2024-03-07T07:43:30.563 INFO:tasks.workunit.client.0.smithi160.stdout:kaluad 111 root txt unknown /proc/111/exe (readlink: Permission denied) 2024-03-07T07:43:30.563 INFO:tasks.workunit.client.0.smithi160.stdout:kaluad 111 root NOFD /proc/111/fd (opendir: Permission denied) 2024-03-07T07:43:30.563 INFO:tasks.workunit.client.0.smithi160.stdout:mld 112 root cwd unknown /proc/112/cwd (readlink: Permission denied) 2024-03-07T07:43:30.563 INFO:tasks.workunit.client.0.smithi160.stdout:mld 112 root rtd unknown /proc/112/root (readlink: Permission denied) 2024-03-07T07:43:30.564 INFO:tasks.workunit.client.0.smithi160.stdout:mld 112 root txt unknown /proc/112/exe (readlink: Permission denied) 2024-03-07T07:43:30.564 INFO:tasks.workunit.client.0.smithi160.stdout:mld 112 root NOFD /proc/112/fd (opendir: Permission denied) 2024-03-07T07:43:30.564 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/1 113 root cwd unknown /proc/113/cwd (readlink: Permission denied) 2024-03-07T07:43:30.564 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/1 113 root rtd unknown /proc/113/root (readlink: Permission denied) 2024-03-07T07:43:30.564 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/1 113 root txt unknown /proc/113/exe (readlink: Permission denied) 2024-03-07T07:43:30.564 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/1 113 root NOFD /proc/113/fd (opendir: Permission denied) 2024-03-07T07:43:30.564 INFO:tasks.workunit.client.0.smithi160.stdout:ipv6_addr 114 root cwd unknown /proc/114/cwd (readlink: Permission denied) 2024-03-07T07:43:30.564 INFO:tasks.workunit.client.0.smithi160.stdout:ipv6_addr 114 root rtd unknown /proc/114/root (readlink: Permission denied) 2024-03-07T07:43:30.564 INFO:tasks.workunit.client.0.smithi160.stdout:ipv6_addr 114 root txt unknown /proc/114/exe (readlink: Permission denied) 2024-03-07T07:43:30.564 INFO:tasks.workunit.client.0.smithi160.stdout:ipv6_addr 114 root NOFD /proc/114/fd (opendir: Permission denied) 2024-03-07T07:43:30.564 INFO:tasks.workunit.client.0.smithi160.stdout:kstrp 123 root cwd unknown /proc/123/cwd (readlink: Permission denied) 2024-03-07T07:43:30.564 INFO:tasks.workunit.client.0.smithi160.stdout:kstrp 123 root rtd unknown /proc/123/root (readlink: Permission denied) 2024-03-07T07:43:30.565 INFO:tasks.workunit.client.0.smithi160.stdout:kstrp 123 root txt unknown /proc/123/exe (readlink: Permission denied) 2024-03-07T07:43:30.565 INFO:tasks.workunit.client.0.smithi160.stdout:kstrp 123 root NOFD /proc/123/fd (opendir: Permission denied) 2024-03-07T07:43:30.565 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/7 124 root cwd unknown /proc/124/cwd (readlink: Permission denied) 2024-03-07T07:43:30.565 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/7 124 root rtd unknown /proc/124/root (readlink: Permission denied) 2024-03-07T07:43:30.565 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/7 124 root txt unknown /proc/124/exe (readlink: Permission denied) 2024-03-07T07:43:30.565 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/7 124 root NOFD /proc/124/fd (opendir: Permission denied) 2024-03-07T07:43:30.565 INFO:tasks.workunit.client.0.smithi160.stdout:zswap-shr 130 root cwd unknown /proc/130/cwd (readlink: Permission denied) 2024-03-07T07:43:30.565 INFO:tasks.workunit.client.0.smithi160.stdout:zswap-shr 130 root rtd unknown /proc/130/root (readlink: Permission denied) 2024-03-07T07:43:30.565 INFO:tasks.workunit.client.0.smithi160.stdout:zswap-shr 130 root txt unknown /proc/130/exe (readlink: Permission denied) 2024-03-07T07:43:30.565 INFO:tasks.workunit.client.0.smithi160.stdout:zswap-shr 130 root NOFD /proc/130/fd (opendir: Permission denied) 2024-03-07T07:43:30.565 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/u 131 root cwd unknown /proc/131/cwd (readlink: Permission denied) 2024-03-07T07:43:30.565 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/u 131 root rtd unknown /proc/131/root (readlink: Permission denied) 2024-03-07T07:43:30.565 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/u 131 root txt unknown /proc/131/exe (readlink: Permission denied) 2024-03-07T07:43:30.566 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/u 131 root NOFD /proc/131/fd (opendir: Permission denied) 2024-03-07T07:43:30.566 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/0 278 root cwd unknown /proc/278/cwd (readlink: Permission denied) 2024-03-07T07:43:30.566 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/0 278 root rtd unknown /proc/278/root (readlink: Permission denied) 2024-03-07T07:43:30.566 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/0 278 root txt unknown /proc/278/exe (readlink: Permission denied) 2024-03-07T07:43:30.566 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/0 278 root NOFD /proc/278/fd (opendir: Permission denied) 2024-03-07T07:43:30.566 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/5 280 root cwd unknown /proc/280/cwd (readlink: Permission denied) 2024-03-07T07:43:30.566 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/5 280 root rtd unknown /proc/280/root (readlink: Permission denied) 2024-03-07T07:43:30.566 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/5 280 root txt unknown /proc/280/exe (readlink: Permission denied) 2024-03-07T07:43:30.566 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/5 280 root NOFD /proc/280/fd (opendir: Permission denied) 2024-03-07T07:43:30.566 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/4 281 root cwd unknown /proc/281/cwd (readlink: Permission denied) 2024-03-07T07:43:30.566 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/4 281 root rtd unknown /proc/281/root (readlink: Permission denied) 2024-03-07T07:43:30.566 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/4 281 root txt unknown /proc/281/exe (readlink: Permission denied) 2024-03-07T07:43:30.566 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/4 281 root NOFD /proc/281/fd (opendir: Permission denied) 2024-03-07T07:43:30.566 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/7 284 root cwd unknown /proc/284/cwd (readlink: Permission denied) 2024-03-07T07:43:30.567 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/7 284 root rtd unknown /proc/284/root (readlink: Permission denied) 2024-03-07T07:43:30.567 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/7 284 root txt unknown /proc/284/exe (readlink: Permission denied) 2024-03-07T07:43:30.567 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/7 284 root NOFD /proc/284/fd (opendir: Permission denied) 2024-03-07T07:43:30.567 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/0 293 root cwd unknown /proc/293/cwd (readlink: Permission denied) 2024-03-07T07:43:30.567 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/0 293 root rtd unknown /proc/293/root (readlink: Permission denied) 2024-03-07T07:43:30.567 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/0 293 root txt unknown /proc/293/exe (readlink: Permission denied) 2024-03-07T07:43:30.567 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/0 293 root NOFD /proc/293/fd (opendir: Permission denied) 2024-03-07T07:43:30.567 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/6 327 root cwd unknown /proc/327/cwd (readlink: Permission denied) 2024-03-07T07:43:30.567 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/6 327 root rtd unknown /proc/327/root (readlink: Permission denied) 2024-03-07T07:43:30.567 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/6 327 root txt unknown /proc/327/exe (readlink: Permission denied) 2024-03-07T07:43:30.567 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/6 327 root NOFD /proc/327/fd (opendir: Permission denied) 2024-03-07T07:43:30.567 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/6 360 root cwd unknown /proc/360/cwd (readlink: Permission denied) 2024-03-07T07:43:30.567 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/6 360 root rtd unknown /proc/360/root (readlink: Permission denied) 2024-03-07T07:43:30.567 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/6 360 root txt unknown /proc/360/exe (readlink: Permission denied) 2024-03-07T07:43:30.568 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/6 360 root NOFD /proc/360/fd (opendir: Permission denied) 2024-03-07T07:43:30.568 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/6 361 root cwd unknown /proc/361/cwd (readlink: Permission denied) 2024-03-07T07:43:30.568 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/6 361 root rtd unknown /proc/361/root (readlink: Permission denied) 2024-03-07T07:43:30.568 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/6 361 root txt unknown /proc/361/exe (readlink: Permission denied) 2024-03-07T07:43:30.568 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/6 361 root NOFD /proc/361/fd (opendir: Permission denied) 2024-03-07T07:43:30.568 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/3 364 root cwd unknown /proc/364/cwd (readlink: Permission denied) 2024-03-07T07:43:30.568 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/3 364 root rtd unknown /proc/364/root (readlink: Permission denied) 2024-03-07T07:43:30.568 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/3 364 root txt unknown /proc/364/exe (readlink: Permission denied) 2024-03-07T07:43:30.568 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/3 364 root NOFD /proc/364/fd (opendir: Permission denied) 2024-03-07T07:43:30.568 INFO:tasks.workunit.client.0.smithi160.stdout:ata_sff 430 root cwd unknown /proc/430/cwd (readlink: Permission denied) 2024-03-07T07:43:30.568 INFO:tasks.workunit.client.0.smithi160.stdout:ata_sff 430 root rtd unknown /proc/430/root (readlink: Permission denied) 2024-03-07T07:43:30.568 INFO:tasks.workunit.client.0.smithi160.stdout:ata_sff 430 root txt unknown /proc/430/exe (readlink: Permission denied) 2024-03-07T07:43:30.568 INFO:tasks.workunit.client.0.smithi160.stdout:ata_sff 430 root NOFD /proc/430/fd (opendir: Permission denied) 2024-03-07T07:43:30.569 INFO:tasks.workunit.client.0.smithi160.stdout:nvme-wq 432 root cwd unknown /proc/432/cwd (readlink: Permission denied) 2024-03-07T07:43:30.569 INFO:tasks.workunit.client.0.smithi160.stdout:nvme-wq 432 root rtd unknown /proc/432/root (readlink: Permission denied) 2024-03-07T07:43:30.569 INFO:tasks.workunit.client.0.smithi160.stdout:nvme-wq 432 root txt unknown /proc/432/exe (readlink: Permission denied) 2024-03-07T07:43:30.569 INFO:tasks.workunit.client.0.smithi160.stdout:nvme-wq 432 root NOFD /proc/432/fd (opendir: Permission denied) 2024-03-07T07:43:30.569 INFO:tasks.workunit.client.0.smithi160.stdout:nvme-rese 434 root cwd unknown /proc/434/cwd (readlink: Permission denied) 2024-03-07T07:43:30.569 INFO:tasks.workunit.client.0.smithi160.stdout:nvme-rese 434 root rtd unknown /proc/434/root (readlink: Permission denied) 2024-03-07T07:43:30.569 INFO:tasks.workunit.client.0.smithi160.stdout:nvme-rese 434 root txt unknown /proc/434/exe (readlink: Permission denied) 2024-03-07T07:43:30.569 INFO:tasks.workunit.client.0.smithi160.stdout:nvme-rese 434 root NOFD /proc/434/fd (opendir: Permission denied) 2024-03-07T07:43:30.569 INFO:tasks.workunit.client.0.smithi160.stdout:nvme-dele 440 root cwd unknown /proc/440/cwd (readlink: Permission denied) 2024-03-07T07:43:30.569 INFO:tasks.workunit.client.0.smithi160.stdout:nvme-dele 440 root rtd unknown /proc/440/root (readlink: Permission denied) 2024-03-07T07:43:30.569 INFO:tasks.workunit.client.0.smithi160.stdout:nvme-dele 440 root txt unknown /proc/440/exe (readlink: Permission denied) 2024-03-07T07:43:30.569 INFO:tasks.workunit.client.0.smithi160.stdout:nvme-dele 440 root NOFD /proc/440/fd (opendir: Permission denied) 2024-03-07T07:43:30.569 INFO:tasks.workunit.client.0.smithi160.stdout:nvme-auth 441 root cwd unknown /proc/441/cwd (readlink: Permission denied) 2024-03-07T07:43:30.569 INFO:tasks.workunit.client.0.smithi160.stdout:nvme-auth 441 root rtd unknown /proc/441/root (readlink: Permission denied) 2024-03-07T07:43:30.570 INFO:tasks.workunit.client.0.smithi160.stdout:nvme-auth 441 root txt unknown /proc/441/exe (readlink: Permission denied) 2024-03-07T07:43:30.570 INFO:tasks.workunit.client.0.smithi160.stdout:nvme-auth 441 root NOFD /proc/441/fd (opendir: Permission denied) 2024-03-07T07:43:30.570 INFO:tasks.workunit.client.0.smithi160.stdout:ixgbe 443 root cwd unknown /proc/443/cwd (readlink: Permission denied) 2024-03-07T07:43:30.570 INFO:tasks.workunit.client.0.smithi160.stdout:ixgbe 443 root rtd unknown /proc/443/root (readlink: Permission denied) 2024-03-07T07:43:30.570 INFO:tasks.workunit.client.0.smithi160.stdout:ixgbe 443 root txt unknown /proc/443/exe (readlink: Permission denied) 2024-03-07T07:43:30.570 INFO:tasks.workunit.client.0.smithi160.stdout:ixgbe 443 root NOFD /proc/443/fd (opendir: Permission denied) 2024-03-07T07:43:30.570 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_eh_0 444 root cwd unknown /proc/444/cwd (readlink: Permission denied) 2024-03-07T07:43:30.570 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_eh_0 444 root rtd unknown /proc/444/root (readlink: Permission denied) 2024-03-07T07:43:30.570 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_eh_0 444 root txt unknown /proc/444/exe (readlink: Permission denied) 2024-03-07T07:43:30.570 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_eh_0 444 root NOFD /proc/444/fd (opendir: Permission denied) 2024-03-07T07:43:30.570 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_tmf_ 445 root cwd unknown /proc/445/cwd (readlink: Permission denied) 2024-03-07T07:43:30.570 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_tmf_ 445 root rtd unknown /proc/445/root (readlink: Permission denied) 2024-03-07T07:43:30.570 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_tmf_ 445 root txt unknown /proc/445/exe (readlink: Permission denied) 2024-03-07T07:43:30.570 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_tmf_ 445 root NOFD /proc/445/fd (opendir: Permission denied) 2024-03-07T07:43:30.571 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_eh_1 446 root cwd unknown /proc/446/cwd (readlink: Permission denied) 2024-03-07T07:43:30.571 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_eh_1 446 root rtd unknown /proc/446/root (readlink: Permission denied) 2024-03-07T07:43:30.571 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_eh_1 446 root txt unknown /proc/446/exe (readlink: Permission denied) 2024-03-07T07:43:30.571 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_eh_1 446 root NOFD /proc/446/fd (opendir: Permission denied) 2024-03-07T07:43:30.571 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_tmf_ 447 root cwd unknown /proc/447/cwd (readlink: Permission denied) 2024-03-07T07:43:30.571 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_tmf_ 447 root rtd unknown /proc/447/root (readlink: Permission denied) 2024-03-07T07:43:30.571 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_tmf_ 447 root txt unknown /proc/447/exe (readlink: Permission denied) 2024-03-07T07:43:30.571 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_tmf_ 447 root NOFD /proc/447/fd (opendir: Permission denied) 2024-03-07T07:43:30.571 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_eh_2 448 root cwd unknown /proc/448/cwd (readlink: Permission denied) 2024-03-07T07:43:30.571 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_eh_2 448 root rtd unknown /proc/448/root (readlink: Permission denied) 2024-03-07T07:43:30.571 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_eh_2 448 root txt unknown /proc/448/exe (readlink: Permission denied) 2024-03-07T07:43:30.571 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_eh_2 448 root NOFD /proc/448/fd (opendir: Permission denied) 2024-03-07T07:43:30.571 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_tmf_ 449 root cwd unknown /proc/449/cwd (readlink: Permission denied) 2024-03-07T07:43:30.571 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_tmf_ 449 root rtd unknown /proc/449/root (readlink: Permission denied) 2024-03-07T07:43:30.572 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_tmf_ 449 root txt unknown /proc/449/exe (readlink: Permission denied) 2024-03-07T07:43:30.572 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_tmf_ 449 root NOFD /proc/449/fd (opendir: Permission denied) 2024-03-07T07:43:30.572 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_eh_3 450 root cwd unknown /proc/450/cwd (readlink: Permission denied) 2024-03-07T07:43:30.572 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_eh_3 450 root rtd unknown /proc/450/root (readlink: Permission denied) 2024-03-07T07:43:30.572 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_eh_3 450 root txt unknown /proc/450/exe (readlink: Permission denied) 2024-03-07T07:43:30.572 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_eh_3 450 root NOFD /proc/450/fd (opendir: Permission denied) 2024-03-07T07:43:30.572 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_tmf_ 451 root cwd unknown /proc/451/cwd (readlink: Permission denied) 2024-03-07T07:43:30.572 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_tmf_ 451 root rtd unknown /proc/451/root (readlink: Permission denied) 2024-03-07T07:43:30.572 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_tmf_ 451 root txt unknown /proc/451/exe (readlink: Permission denied) 2024-03-07T07:43:30.572 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_tmf_ 451 root NOFD /proc/451/fd (opendir: Permission denied) 2024-03-07T07:43:30.572 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_eh_4 453 root cwd unknown /proc/453/cwd (readlink: Permission denied) 2024-03-07T07:43:30.572 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_eh_4 453 root rtd unknown /proc/453/root (readlink: Permission denied) 2024-03-07T07:43:30.572 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_eh_4 453 root txt unknown /proc/453/exe (readlink: Permission denied) 2024-03-07T07:43:30.572 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_eh_4 453 root NOFD /proc/453/fd (opendir: Permission denied) 2024-03-07T07:43:30.573 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_tmf_ 454 root cwd unknown /proc/454/cwd (readlink: Permission denied) 2024-03-07T07:43:30.573 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_tmf_ 454 root rtd unknown /proc/454/root (readlink: Permission denied) 2024-03-07T07:43:30.573 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_tmf_ 454 root txt unknown /proc/454/exe (readlink: Permission denied) 2024-03-07T07:43:30.573 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_tmf_ 454 root NOFD /proc/454/fd (opendir: Permission denied) 2024-03-07T07:43:30.573 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_eh_5 455 root cwd unknown /proc/455/cwd (readlink: Permission denied) 2024-03-07T07:43:30.573 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_eh_5 455 root rtd unknown /proc/455/root (readlink: Permission denied) 2024-03-07T07:43:30.573 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_eh_5 455 root txt unknown /proc/455/exe (readlink: Permission denied) 2024-03-07T07:43:30.573 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_eh_5 455 root NOFD /proc/455/fd (opendir: Permission denied) 2024-03-07T07:43:30.573 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_tmf_ 456 root cwd unknown /proc/456/cwd (readlink: Permission denied) 2024-03-07T07:43:30.573 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_tmf_ 456 root rtd unknown /proc/456/root (readlink: Permission denied) 2024-03-07T07:43:30.573 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_tmf_ 456 root txt unknown /proc/456/exe (readlink: Permission denied) 2024-03-07T07:43:30.573 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_tmf_ 456 root NOFD /proc/456/fd (opendir: Permission denied) 2024-03-07T07:43:30.573 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_eh_6 457 root cwd unknown /proc/457/cwd (readlink: Permission denied) 2024-03-07T07:43:30.573 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_eh_6 457 root rtd unknown /proc/457/root (readlink: Permission denied) 2024-03-07T07:43:30.574 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_eh_6 457 root txt unknown /proc/457/exe (readlink: Permission denied) 2024-03-07T07:43:30.574 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_eh_6 457 root NOFD /proc/457/fd (opendir: Permission denied) 2024-03-07T07:43:30.574 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_tmf_ 458 root cwd unknown /proc/458/cwd (readlink: Permission denied) 2024-03-07T07:43:30.574 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_tmf_ 458 root rtd unknown /proc/458/root (readlink: Permission denied) 2024-03-07T07:43:30.574 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_tmf_ 458 root txt unknown /proc/458/exe (readlink: Permission denied) 2024-03-07T07:43:30.574 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_tmf_ 458 root NOFD /proc/458/fd (opendir: Permission denied) 2024-03-07T07:43:30.574 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_eh_7 459 root cwd unknown /proc/459/cwd (readlink: Permission denied) 2024-03-07T07:43:30.574 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_eh_7 459 root rtd unknown /proc/459/root (readlink: Permission denied) 2024-03-07T07:43:30.574 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_eh_7 459 root txt unknown /proc/459/exe (readlink: Permission denied) 2024-03-07T07:43:30.574 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_eh_7 459 root NOFD /proc/459/fd (opendir: Permission denied) 2024-03-07T07:43:30.574 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_tmf_ 460 root cwd unknown /proc/460/cwd (readlink: Permission denied) 2024-03-07T07:43:30.574 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_tmf_ 460 root rtd unknown /proc/460/root (readlink: Permission denied) 2024-03-07T07:43:30.574 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_tmf_ 460 root txt unknown /proc/460/exe (readlink: Permission denied) 2024-03-07T07:43:30.574 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_tmf_ 460 root NOFD /proc/460/fd (opendir: Permission denied) 2024-03-07T07:43:30.575 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_eh_8 461 root cwd unknown /proc/461/cwd (readlink: Permission denied) 2024-03-07T07:43:30.575 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_eh_8 461 root rtd unknown /proc/461/root (readlink: Permission denied) 2024-03-07T07:43:30.575 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_eh_8 461 root txt unknown /proc/461/exe (readlink: Permission denied) 2024-03-07T07:43:30.575 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_eh_8 461 root NOFD /proc/461/fd (opendir: Permission denied) 2024-03-07T07:43:30.575 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_tmf_ 462 root cwd unknown /proc/462/cwd (readlink: Permission denied) 2024-03-07T07:43:30.575 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_tmf_ 462 root rtd unknown /proc/462/root (readlink: Permission denied) 2024-03-07T07:43:30.575 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_tmf_ 462 root txt unknown /proc/462/exe (readlink: Permission denied) 2024-03-07T07:43:30.575 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_tmf_ 462 root NOFD /proc/462/fd (opendir: Permission denied) 2024-03-07T07:43:30.575 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_eh_9 463 root cwd unknown /proc/463/cwd (readlink: Permission denied) 2024-03-07T07:43:30.575 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_eh_9 463 root rtd unknown /proc/463/root (readlink: Permission denied) 2024-03-07T07:43:30.575 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_eh_9 463 root txt unknown /proc/463/exe (readlink: Permission denied) 2024-03-07T07:43:30.575 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_eh_9 463 root NOFD /proc/463/fd (opendir: Permission denied) 2024-03-07T07:43:30.575 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_tmf_ 464 root cwd unknown /proc/464/cwd (readlink: Permission denied) 2024-03-07T07:43:30.575 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_tmf_ 464 root rtd unknown /proc/464/root (readlink: Permission denied) 2024-03-07T07:43:30.575 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_tmf_ 464 root txt unknown /proc/464/exe (readlink: Permission denied) 2024-03-07T07:43:30.576 INFO:tasks.workunit.client.0.smithi160.stdout:scsi_tmf_ 464 root NOFD /proc/464/fd (opendir: Permission denied) 2024-03-07T07:43:30.576 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/u 465 root cwd unknown /proc/465/cwd (readlink: Permission denied) 2024-03-07T07:43:30.576 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/u 465 root rtd unknown /proc/465/root (readlink: Permission denied) 2024-03-07T07:43:30.576 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/u 465 root txt unknown /proc/465/exe (readlink: Permission denied) 2024-03-07T07:43:30.576 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/u 465 root NOFD /proc/465/fd (opendir: Permission denied) 2024-03-07T07:43:30.576 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/u 466 root cwd unknown /proc/466/cwd (readlink: Permission denied) 2024-03-07T07:43:30.576 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/u 466 root rtd unknown /proc/466/root (readlink: Permission denied) 2024-03-07T07:43:30.576 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/u 466 root txt unknown /proc/466/exe (readlink: Permission denied) 2024-03-07T07:43:30.576 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/u 466 root NOFD /proc/466/fd (opendir: Permission denied) 2024-03-07T07:43:30.576 INFO:tasks.workunit.client.0.smithi160.stdout:jbd2/sda1 500 root cwd unknown /proc/500/cwd (readlink: Permission denied) 2024-03-07T07:43:30.576 INFO:tasks.workunit.client.0.smithi160.stdout:jbd2/sda1 500 root rtd unknown /proc/500/root (readlink: Permission denied) 2024-03-07T07:43:30.576 INFO:tasks.workunit.client.0.smithi160.stdout:jbd2/sda1 500 root txt unknown /proc/500/exe (readlink: Permission denied) 2024-03-07T07:43:30.576 INFO:tasks.workunit.client.0.smithi160.stdout:jbd2/sda1 500 root NOFD /proc/500/fd (opendir: Permission denied) 2024-03-07T07:43:30.577 INFO:tasks.workunit.client.0.smithi160.stdout:ext4-rsv- 501 root cwd unknown /proc/501/cwd (readlink: Permission denied) 2024-03-07T07:43:30.577 INFO:tasks.workunit.client.0.smithi160.stdout:ext4-rsv- 501 root rtd unknown /proc/501/root (readlink: Permission denied) 2024-03-07T07:43:30.577 INFO:tasks.workunit.client.0.smithi160.stdout:ext4-rsv- 501 root txt unknown /proc/501/exe (readlink: Permission denied) 2024-03-07T07:43:30.577 INFO:tasks.workunit.client.0.smithi160.stdout:ext4-rsv- 501 root NOFD /proc/501/fd (opendir: Permission denied) 2024-03-07T07:43:30.577 INFO:tasks.workunit.client.0.smithi160.stdout:systemd-j 577 root cwd unknown /proc/577/cwd (readlink: Permission denied) 2024-03-07T07:43:30.577 INFO:tasks.workunit.client.0.smithi160.stdout:systemd-j 577 root rtd unknown /proc/577/root (readlink: Permission denied) 2024-03-07T07:43:30.577 INFO:tasks.workunit.client.0.smithi160.stdout:systemd-j 577 root txt unknown /proc/577/exe (readlink: Permission denied) 2024-03-07T07:43:30.577 INFO:tasks.workunit.client.0.smithi160.stdout:systemd-j 577 root NOFD /proc/577/fd (opendir: Permission denied) 2024-03-07T07:43:30.577 INFO:tasks.workunit.client.0.smithi160.stdout:ext4lazyi 583 root cwd unknown /proc/583/cwd (readlink: Permission denied) 2024-03-07T07:43:30.577 INFO:tasks.workunit.client.0.smithi160.stdout:ext4lazyi 583 root rtd unknown /proc/583/root (readlink: Permission denied) 2024-03-07T07:43:30.577 INFO:tasks.workunit.client.0.smithi160.stdout:ext4lazyi 583 root txt unknown /proc/583/exe (readlink: Permission denied) 2024-03-07T07:43:30.577 INFO:tasks.workunit.client.0.smithi160.stdout:ext4lazyi 583 root NOFD /proc/583/fd (opendir: Permission denied) 2024-03-07T07:43:30.577 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/5 589 root cwd unknown /proc/589/cwd (readlink: Permission denied) 2024-03-07T07:43:30.577 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/5 589 root rtd unknown /proc/589/root (readlink: Permission denied) 2024-03-07T07:43:30.578 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/5 589 root txt unknown /proc/589/exe (readlink: Permission denied) 2024-03-07T07:43:30.578 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/5 589 root NOFD /proc/589/fd (opendir: Permission denied) 2024-03-07T07:43:30.578 INFO:tasks.workunit.client.0.smithi160.stdout:systemd-u 593 root cwd unknown /proc/593/cwd (readlink: Permission denied) 2024-03-07T07:43:30.578 INFO:tasks.workunit.client.0.smithi160.stdout:systemd-u 593 root rtd unknown /proc/593/root (readlink: Permission denied) 2024-03-07T07:43:30.578 INFO:tasks.workunit.client.0.smithi160.stdout:systemd-u 593 root txt unknown /proc/593/exe (readlink: Permission denied) 2024-03-07T07:43:30.578 INFO:tasks.workunit.client.0.smithi160.stdout:systemd-u 593 root NOFD /proc/593/fd (opendir: Permission denied) 2024-03-07T07:43:30.578 INFO:tasks.workunit.client.0.smithi160.stdout:ipmi-msgh 642 root cwd unknown /proc/642/cwd (readlink: Permission denied) 2024-03-07T07:43:30.578 INFO:tasks.workunit.client.0.smithi160.stdout:ipmi-msgh 642 root rtd unknown /proc/642/root (readlink: Permission denied) 2024-03-07T07:43:30.578 INFO:tasks.workunit.client.0.smithi160.stdout:ipmi-msgh 642 root txt unknown /proc/642/exe (readlink: Permission denied) 2024-03-07T07:43:30.578 INFO:tasks.workunit.client.0.smithi160.stdout:ipmi-msgh 642 root NOFD /proc/642/fd (opendir: Permission denied) 2024-03-07T07:43:30.578 INFO:tasks.workunit.client.0.smithi160.stdout:kipmi0 656 root cwd unknown /proc/656/cwd (readlink: Permission denied) 2024-03-07T07:43:30.578 INFO:tasks.workunit.client.0.smithi160.stdout:kipmi0 656 root rtd unknown /proc/656/root (readlink: Permission denied) 2024-03-07T07:43:30.578 INFO:tasks.workunit.client.0.smithi160.stdout:kipmi0 656 root txt unknown /proc/656/exe (readlink: Permission denied) 2024-03-07T07:43:30.578 INFO:tasks.workunit.client.0.smithi160.stdout:kipmi0 656 root NOFD /proc/656/fd (opendir: Permission denied) 2024-03-07T07:43:30.578 INFO:tasks.workunit.client.0.smithi160.stdout:rpcbind 669 rpc cwd unknown /proc/669/cwd (readlink: Permission denied) 2024-03-07T07:43:30.579 INFO:tasks.workunit.client.0.smithi160.stdout:rpcbind 669 rpc rtd unknown /proc/669/root (readlink: Permission denied) 2024-03-07T07:43:30.579 INFO:tasks.workunit.client.0.smithi160.stdout:rpcbind 669 rpc txt unknown /proc/669/exe (readlink: Permission denied) 2024-03-07T07:43:30.579 INFO:tasks.workunit.client.0.smithi160.stdout:rpcbind 669 rpc NOFD /proc/669/fd (opendir: Permission denied) 2024-03-07T07:43:30.579 INFO:tasks.workunit.client.0.smithi160.stdout:rpciod 672 root cwd unknown /proc/672/cwd (readlink: Permission denied) 2024-03-07T07:43:30.579 INFO:tasks.workunit.client.0.smithi160.stdout:rpciod 672 root rtd unknown /proc/672/root (readlink: Permission denied) 2024-03-07T07:43:30.579 INFO:tasks.workunit.client.0.smithi160.stdout:rpciod 672 root txt unknown /proc/672/exe (readlink: Permission denied) 2024-03-07T07:43:30.579 INFO:tasks.workunit.client.0.smithi160.stdout:rpciod 672 root NOFD /proc/672/fd (opendir: Permission denied) 2024-03-07T07:43:30.579 INFO:tasks.workunit.client.0.smithi160.stdout:xprtiod 673 root cwd unknown /proc/673/cwd (readlink: Permission denied) 2024-03-07T07:43:30.579 INFO:tasks.workunit.client.0.smithi160.stdout:xprtiod 673 root rtd unknown /proc/673/root (readlink: Permission denied) 2024-03-07T07:43:30.579 INFO:tasks.workunit.client.0.smithi160.stdout:xprtiod 673 root txt unknown /proc/673/exe (readlink: Permission denied) 2024-03-07T07:43:30.579 INFO:tasks.workunit.client.0.smithi160.stdout:xprtiod 673 root NOFD /proc/673/fd (opendir: Permission denied) 2024-03-07T07:43:30.579 INFO:tasks.workunit.client.0.smithi160.stdout:auditd 674 root cwd unknown /proc/674/cwd (readlink: Permission denied) 2024-03-07T07:43:30.579 INFO:tasks.workunit.client.0.smithi160.stdout:auditd 674 root rtd unknown /proc/674/root (readlink: Permission denied) 2024-03-07T07:43:30.579 INFO:tasks.workunit.client.0.smithi160.stdout:auditd 674 root txt unknown /proc/674/exe (readlink: Permission denied) 2024-03-07T07:43:30.579 INFO:tasks.workunit.client.0.smithi160.stdout:auditd 674 root NOFD /proc/674/fd (opendir: Permission denied) 2024-03-07T07:43:30.580 INFO:tasks.workunit.client.0.smithi160.stdout:auditd 674 675 auditd root cwd unknown /proc/674/task/675/cwd (readlink: Permission denied) 2024-03-07T07:43:30.580 INFO:tasks.workunit.client.0.smithi160.stdout:auditd 674 675 auditd root rtd unknown /proc/674/task/675/root (readlink: Permission denied) 2024-03-07T07:43:30.580 INFO:tasks.workunit.client.0.smithi160.stdout:auditd 674 675 auditd root txt unknown /proc/674/task/675/exe (readlink: Permission denied) 2024-03-07T07:43:30.580 INFO:tasks.workunit.client.0.smithi160.stdout:auditd 674 675 auditd root NOFD /proc/674/task/675/fd (opendir: Permission denied) 2024-03-07T07:43:30.580 INFO:tasks.workunit.client.0.smithi160.stdout:auditd 674 677 auditd root cwd unknown /proc/674/task/677/cwd (readlink: Permission denied) 2024-03-07T07:43:30.580 INFO:tasks.workunit.client.0.smithi160.stdout:auditd 674 677 auditd root rtd unknown /proc/674/task/677/root (readlink: Permission denied) 2024-03-07T07:43:30.580 INFO:tasks.workunit.client.0.smithi160.stdout:auditd 674 677 auditd root txt unknown /proc/674/task/677/exe (readlink: Permission denied) 2024-03-07T07:43:30.580 INFO:tasks.workunit.client.0.smithi160.stdout:auditd 674 677 auditd root NOFD /proc/674/task/677/fd (opendir: Permission denied) 2024-03-07T07:43:30.580 INFO:tasks.workunit.client.0.smithi160.stdout:sedispatc 676 root cwd unknown /proc/676/cwd (readlink: Permission denied) 2024-03-07T07:43:30.580 INFO:tasks.workunit.client.0.smithi160.stdout:sedispatc 676 root rtd unknown /proc/676/root (readlink: Permission denied) 2024-03-07T07:43:30.580 INFO:tasks.workunit.client.0.smithi160.stdout:sedispatc 676 root txt unknown /proc/676/exe (readlink: Permission denied) 2024-03-07T07:43:30.580 INFO:tasks.workunit.client.0.smithi160.stdout:sedispatc 676 root NOFD /proc/676/fd (opendir: Permission denied) 2024-03-07T07:43:30.580 INFO:tasks.workunit.client.0.smithi160.stdout:dbus-brok 713 dbus cwd unknown /proc/713/cwd (readlink: Permission denied) 2024-03-07T07:43:30.581 INFO:tasks.workunit.client.0.smithi160.stdout:dbus-brok 713 dbus rtd unknown /proc/713/root (readlink: Permission denied) 2024-03-07T07:43:30.581 INFO:tasks.workunit.client.0.smithi160.stdout:dbus-brok 713 dbus txt unknown /proc/713/exe (readlink: Permission denied) 2024-03-07T07:43:30.581 INFO:tasks.workunit.client.0.smithi160.stdout:dbus-brok 713 dbus NOFD /proc/713/fd (opendir: Permission denied) 2024-03-07T07:43:30.581 INFO:tasks.workunit.client.0.smithi160.stdout:dbus-brok 714 dbus cwd unknown /proc/714/cwd (readlink: Permission denied) 2024-03-07T07:43:30.581 INFO:tasks.workunit.client.0.smithi160.stdout:dbus-brok 714 dbus rtd unknown /proc/714/root (readlink: Permission denied) 2024-03-07T07:43:30.581 INFO:tasks.workunit.client.0.smithi160.stdout:dbus-brok 714 dbus txt unknown /proc/714/exe (readlink: Permission denied) 2024-03-07T07:43:30.581 INFO:tasks.workunit.client.0.smithi160.stdout:dbus-brok 714 dbus NOFD /proc/714/fd (opendir: Permission denied) 2024-03-07T07:43:30.581 INFO:tasks.workunit.client.0.smithi160.stdout:NetworkMa 715 root cwd unknown /proc/715/cwd (readlink: Permission denied) 2024-03-07T07:43:30.581 INFO:tasks.workunit.client.0.smithi160.stdout:NetworkMa 715 root rtd unknown /proc/715/root (readlink: Permission denied) 2024-03-07T07:43:30.581 INFO:tasks.workunit.client.0.smithi160.stdout:NetworkMa 715 root txt unknown /proc/715/exe (readlink: Permission denied) 2024-03-07T07:43:30.581 INFO:tasks.workunit.client.0.smithi160.stdout:NetworkMa 715 root NOFD /proc/715/fd (opendir: Permission denied) 2024-03-07T07:43:30.581 INFO:tasks.workunit.client.0.smithi160.stdout:NetworkMa 715 743 gmain root cwd unknown /proc/715/task/743/cwd (readlink: Permission denied) 2024-03-07T07:43:30.581 INFO:tasks.workunit.client.0.smithi160.stdout:NetworkMa 715 743 gmain root rtd unknown /proc/715/task/743/root (readlink: Permission denied) 2024-03-07T07:43:30.581 INFO:tasks.workunit.client.0.smithi160.stdout:NetworkMa 715 743 gmain root txt unknown /proc/715/task/743/exe (readlink: Permission denied) 2024-03-07T07:43:30.582 INFO:tasks.workunit.client.0.smithi160.stdout:NetworkMa 715 743 gmain root NOFD /proc/715/task/743/fd (opendir: Permission denied) 2024-03-07T07:43:30.582 INFO:tasks.workunit.client.0.smithi160.stdout:NetworkMa 715 746 gdbus root cwd unknown /proc/715/task/746/cwd (readlink: Permission denied) 2024-03-07T07:43:30.582 INFO:tasks.workunit.client.0.smithi160.stdout:NetworkMa 715 746 gdbus root rtd unknown /proc/715/task/746/root (readlink: Permission denied) 2024-03-07T07:43:30.582 INFO:tasks.workunit.client.0.smithi160.stdout:NetworkMa 715 746 gdbus root txt unknown /proc/715/task/746/exe (readlink: Permission denied) 2024-03-07T07:43:30.582 INFO:tasks.workunit.client.0.smithi160.stdout:NetworkMa 715 746 gdbus root NOFD /proc/715/task/746/fd (opendir: Permission denied) 2024-03-07T07:43:30.582 INFO:tasks.workunit.client.0.smithi160.stdout:irqbalanc 718 root cwd unknown /proc/718/cwd (readlink: Permission denied) 2024-03-07T07:43:30.582 INFO:tasks.workunit.client.0.smithi160.stdout:irqbalanc 718 root rtd unknown /proc/718/root (readlink: Permission denied) 2024-03-07T07:43:30.582 INFO:tasks.workunit.client.0.smithi160.stdout:irqbalanc 718 root txt unknown /proc/718/exe (readlink: Permission denied) 2024-03-07T07:43:30.582 INFO:tasks.workunit.client.0.smithi160.stdout:irqbalanc 718 root NOFD /proc/718/fd (opendir: Permission denied) 2024-03-07T07:43:30.582 INFO:tasks.workunit.client.0.smithi160.stdout:irqbalanc 718 725 gmain root cwd unknown /proc/718/task/725/cwd (readlink: Permission denied) 2024-03-07T07:43:30.582 INFO:tasks.workunit.client.0.smithi160.stdout:irqbalanc 718 725 gmain root rtd unknown /proc/718/task/725/root (readlink: Permission denied) 2024-03-07T07:43:30.582 INFO:tasks.workunit.client.0.smithi160.stdout:irqbalanc 718 725 gmain root txt unknown /proc/718/task/725/exe (readlink: Permission denied) 2024-03-07T07:43:30.582 INFO:tasks.workunit.client.0.smithi160.stdout:irqbalanc 718 725 gmain root NOFD /proc/718/task/725/fd (opendir: Permission denied) 2024-03-07T07:43:30.582 INFO:tasks.workunit.client.0.smithi160.stdout:lsmd 719 libstoragemgmt cwd unknown /proc/719/cwd (readlink: Permission denied) 2024-03-07T07:43:30.582 INFO:tasks.workunit.client.0.smithi160.stdout:lsmd 719 libstoragemgmt rtd unknown /proc/719/root (readlink: Permission denied) 2024-03-07T07:43:30.583 INFO:tasks.workunit.client.0.smithi160.stdout:lsmd 719 libstoragemgmt txt unknown /proc/719/exe (readlink: Permission denied) 2024-03-07T07:43:30.583 INFO:tasks.workunit.client.0.smithi160.stdout:lsmd 719 libstoragemgmt NOFD /proc/719/fd (opendir: Permission denied) 2024-03-07T07:43:30.583 INFO:tasks.workunit.client.0.smithi160.stdout:mcelog 720 root cwd unknown /proc/720/cwd (readlink: Permission denied) 2024-03-07T07:43:30.583 INFO:tasks.workunit.client.0.smithi160.stdout:mcelog 720 root rtd unknown /proc/720/root (readlink: Permission denied) 2024-03-07T07:43:30.583 INFO:tasks.workunit.client.0.smithi160.stdout:mcelog 720 root txt unknown /proc/720/exe (readlink: Permission denied) 2024-03-07T07:43:30.583 INFO:tasks.workunit.client.0.smithi160.stdout:mcelog 720 root NOFD /proc/720/fd (opendir: Permission denied) 2024-03-07T07:43:30.583 INFO:tasks.workunit.client.0.smithi160.stdout:polkitd 721 polkitd cwd unknown /proc/721/cwd (readlink: Permission denied) 2024-03-07T07:43:30.583 INFO:tasks.workunit.client.0.smithi160.stdout:polkitd 721 polkitd rtd unknown /proc/721/root (readlink: Permission denied) 2024-03-07T07:43:30.583 INFO:tasks.workunit.client.0.smithi160.stdout:polkitd 721 polkitd txt unknown /proc/721/exe (readlink: Permission denied) 2024-03-07T07:43:30.583 INFO:tasks.workunit.client.0.smithi160.stdout:polkitd 721 polkitd NOFD /proc/721/fd (opendir: Permission denied) 2024-03-07T07:43:30.583 INFO:tasks.workunit.client.0.smithi160.stdout:polkitd 721 733 gmain polkitd cwd unknown /proc/721/task/733/cwd (readlink: Permission denied) 2024-03-07T07:43:30.583 INFO:tasks.workunit.client.0.smithi160.stdout:polkitd 721 733 gmain polkitd rtd unknown /proc/721/task/733/root (readlink: Permission denied) 2024-03-07T07:43:30.583 INFO:tasks.workunit.client.0.smithi160.stdout:polkitd 721 733 gmain polkitd txt unknown /proc/721/task/733/exe (readlink: Permission denied) 2024-03-07T07:43:30.583 INFO:tasks.workunit.client.0.smithi160.stdout:polkitd 721 733 gmain polkitd NOFD /proc/721/task/733/fd (opendir: Permission denied) 2024-03-07T07:43:30.584 INFO:tasks.workunit.client.0.smithi160.stdout:polkitd 721 734 gdbus polkitd cwd unknown /proc/721/task/734/cwd (readlink: Permission denied) 2024-03-07T07:43:30.584 INFO:tasks.workunit.client.0.smithi160.stdout:polkitd 721 734 gdbus polkitd rtd unknown /proc/721/task/734/root (readlink: Permission denied) 2024-03-07T07:43:30.584 INFO:tasks.workunit.client.0.smithi160.stdout:polkitd 721 734 gdbus polkitd txt unknown /proc/721/task/734/exe (readlink: Permission denied) 2024-03-07T07:43:30.584 INFO:tasks.workunit.client.0.smithi160.stdout:polkitd 721 734 gdbus polkitd NOFD /proc/721/task/734/fd (opendir: Permission denied) 2024-03-07T07:43:30.584 INFO:tasks.workunit.client.0.smithi160.stdout:polkitd 721 735 JS\x20Hel polkitd cwd unknown /proc/721/task/735/cwd (readlink: Permission denied) 2024-03-07T07:43:30.584 INFO:tasks.workunit.client.0.smithi160.stdout:polkitd 721 735 JS\x20Hel polkitd rtd unknown /proc/721/task/735/root (readlink: Permission denied) 2024-03-07T07:43:30.584 INFO:tasks.workunit.client.0.smithi160.stdout:polkitd 721 735 JS\x20Hel polkitd txt unknown /proc/721/task/735/exe (readlink: Permission denied) 2024-03-07T07:43:30.584 INFO:tasks.workunit.client.0.smithi160.stdout:polkitd 721 735 JS\x20Hel polkitd NOFD /proc/721/task/735/fd (opendir: Permission denied) 2024-03-07T07:43:30.584 INFO:tasks.workunit.client.0.smithi160.stdout:polkitd 721 736 JS\x20Hel polkitd cwd unknown /proc/721/task/736/cwd (readlink: Permission denied) 2024-03-07T07:43:30.584 INFO:tasks.workunit.client.0.smithi160.stdout:polkitd 721 736 JS\x20Hel polkitd rtd unknown /proc/721/task/736/root (readlink: Permission denied) 2024-03-07T07:43:30.584 INFO:tasks.workunit.client.0.smithi160.stdout:polkitd 721 736 JS\x20Hel polkitd txt unknown /proc/721/task/736/exe (readlink: Permission denied) 2024-03-07T07:43:30.584 INFO:tasks.workunit.client.0.smithi160.stdout:polkitd 721 736 JS\x20Hel polkitd NOFD /proc/721/task/736/fd (opendir: Permission denied) 2024-03-07T07:43:30.584 INFO:tasks.workunit.client.0.smithi160.stdout:polkitd 721 737 JS\x20Hel polkitd cwd unknown /proc/721/task/737/cwd (readlink: Permission denied) 2024-03-07T07:43:30.584 INFO:tasks.workunit.client.0.smithi160.stdout:polkitd 721 737 JS\x20Hel polkitd rtd unknown /proc/721/task/737/root (readlink: Permission denied) 2024-03-07T07:43:30.585 INFO:tasks.workunit.client.0.smithi160.stdout:polkitd 721 737 JS\x20Hel polkitd txt unknown /proc/721/task/737/exe (readlink: Permission denied) 2024-03-07T07:43:30.585 INFO:tasks.workunit.client.0.smithi160.stdout:polkitd 721 737 JS\x20Hel polkitd NOFD /proc/721/task/737/fd (opendir: Permission denied) 2024-03-07T07:43:30.585 INFO:tasks.workunit.client.0.smithi160.stdout:polkitd 721 738 JS\x20Hel polkitd cwd unknown /proc/721/task/738/cwd (readlink: Permission denied) 2024-03-07T07:43:30.585 INFO:tasks.workunit.client.0.smithi160.stdout:polkitd 721 738 JS\x20Hel polkitd rtd unknown /proc/721/task/738/root (readlink: Permission denied) 2024-03-07T07:43:30.585 INFO:tasks.workunit.client.0.smithi160.stdout:polkitd 721 738 JS\x20Hel polkitd txt unknown /proc/721/task/738/exe (readlink: Permission denied) 2024-03-07T07:43:30.585 INFO:tasks.workunit.client.0.smithi160.stdout:polkitd 721 738 JS\x20Hel polkitd NOFD /proc/721/task/738/fd (opendir: Permission denied) 2024-03-07T07:43:30.585 INFO:tasks.workunit.client.0.smithi160.stdout:polkitd 721 739 JS\x20Hel polkitd cwd unknown /proc/721/task/739/cwd (readlink: Permission denied) 2024-03-07T07:43:30.585 INFO:tasks.workunit.client.0.smithi160.stdout:polkitd 721 739 JS\x20Hel polkitd rtd unknown /proc/721/task/739/root (readlink: Permission denied) 2024-03-07T07:43:30.585 INFO:tasks.workunit.client.0.smithi160.stdout:polkitd 721 739 JS\x20Hel polkitd txt unknown /proc/721/task/739/exe (readlink: Permission denied) 2024-03-07T07:43:30.585 INFO:tasks.workunit.client.0.smithi160.stdout:polkitd 721 739 JS\x20Hel polkitd NOFD /proc/721/task/739/fd (opendir: Permission denied) 2024-03-07T07:43:30.585 INFO:tasks.workunit.client.0.smithi160.stdout:polkitd 721 740 JS\x20Hel polkitd cwd unknown /proc/721/task/740/cwd (readlink: Permission denied) 2024-03-07T07:43:30.585 INFO:tasks.workunit.client.0.smithi160.stdout:polkitd 721 740 JS\x20Hel polkitd rtd unknown /proc/721/task/740/root (readlink: Permission denied) 2024-03-07T07:43:30.585 INFO:tasks.workunit.client.0.smithi160.stdout:polkitd 721 740 JS\x20Hel polkitd txt unknown /proc/721/task/740/exe (readlink: Permission denied) 2024-03-07T07:43:30.585 INFO:tasks.workunit.client.0.smithi160.stdout:polkitd 721 740 JS\x20Hel polkitd NOFD /proc/721/task/740/fd (opendir: Permission denied) 2024-03-07T07:43:30.586 INFO:tasks.workunit.client.0.smithi160.stdout:polkitd 721 741 JS\x20Hel polkitd cwd unknown /proc/721/task/741/cwd (readlink: Permission denied) 2024-03-07T07:43:30.586 INFO:tasks.workunit.client.0.smithi160.stdout:polkitd 721 741 JS\x20Hel polkitd rtd unknown /proc/721/task/741/root (readlink: Permission denied) 2024-03-07T07:43:30.586 INFO:tasks.workunit.client.0.smithi160.stdout:polkitd 721 741 JS\x20Hel polkitd txt unknown /proc/721/task/741/exe (readlink: Permission denied) 2024-03-07T07:43:30.586 INFO:tasks.workunit.client.0.smithi160.stdout:polkitd 721 741 JS\x20Hel polkitd NOFD /proc/721/task/741/fd (opendir: Permission denied) 2024-03-07T07:43:30.586 INFO:tasks.workunit.client.0.smithi160.stdout:polkitd 721 742 JS\x20Hel polkitd cwd unknown /proc/721/task/742/cwd (readlink: Permission denied) 2024-03-07T07:43:30.586 INFO:tasks.workunit.client.0.smithi160.stdout:polkitd 721 742 JS\x20Hel polkitd rtd unknown /proc/721/task/742/root (readlink: Permission denied) 2024-03-07T07:43:30.586 INFO:tasks.workunit.client.0.smithi160.stdout:polkitd 721 742 JS\x20Hel polkitd txt unknown /proc/721/task/742/exe (readlink: Permission denied) 2024-03-07T07:43:30.586 INFO:tasks.workunit.client.0.smithi160.stdout:polkitd 721 742 JS\x20Hel polkitd NOFD /proc/721/task/742/fd (opendir: Permission denied) 2024-03-07T07:43:30.586 INFO:tasks.workunit.client.0.smithi160.stdout:polkitd 721 744 polkitd polkitd cwd unknown /proc/721/task/744/cwd (readlink: Permission denied) 2024-03-07T07:43:30.586 INFO:tasks.workunit.client.0.smithi160.stdout:polkitd 721 744 polkitd polkitd rtd unknown /proc/721/task/744/root (readlink: Permission denied) 2024-03-07T07:43:30.586 INFO:tasks.workunit.client.0.smithi160.stdout:polkitd 721 744 polkitd polkitd txt unknown /proc/721/task/744/exe (readlink: Permission denied) 2024-03-07T07:43:30.586 INFO:tasks.workunit.client.0.smithi160.stdout:polkitd 721 744 polkitd polkitd NOFD /proc/721/task/744/fd (opendir: Permission denied) 2024-03-07T07:43:30.586 INFO:tasks.workunit.client.0.smithi160.stdout:systemd-l 724 root cwd unknown /proc/724/cwd (readlink: Permission denied) 2024-03-07T07:43:30.586 INFO:tasks.workunit.client.0.smithi160.stdout:systemd-l 724 root rtd unknown /proc/724/root (readlink: Permission denied) 2024-03-07T07:43:30.586 INFO:tasks.workunit.client.0.smithi160.stdout:systemd-l 724 root txt unknown /proc/724/exe (readlink: Permission denied) 2024-03-07T07:43:30.587 INFO:tasks.workunit.client.0.smithi160.stdout:systemd-l 724 root NOFD /proc/724/fd (opendir: Permission denied) 2024-03-07T07:43:30.587 INFO:tasks.workunit.client.0.smithi160.stdout:sshd 757 root cwd unknown /proc/757/cwd (readlink: Permission denied) 2024-03-07T07:43:30.587 INFO:tasks.workunit.client.0.smithi160.stdout:sshd 757 root rtd unknown /proc/757/root (readlink: Permission denied) 2024-03-07T07:43:30.587 INFO:tasks.workunit.client.0.smithi160.stdout:sshd 757 root txt unknown /proc/757/exe (readlink: Permission denied) 2024-03-07T07:43:30.587 INFO:tasks.workunit.client.0.smithi160.stdout:sshd 757 root NOFD /proc/757/fd (opendir: Permission denied) 2024-03-07T07:43:30.587 INFO:tasks.workunit.client.0.smithi160.stdout:tuned 758 root cwd unknown /proc/758/cwd (readlink: Permission denied) 2024-03-07T07:43:30.587 INFO:tasks.workunit.client.0.smithi160.stdout:tuned 758 root rtd unknown /proc/758/root (readlink: Permission denied) 2024-03-07T07:43:30.587 INFO:tasks.workunit.client.0.smithi160.stdout:tuned 758 root txt unknown /proc/758/exe (readlink: Permission denied) 2024-03-07T07:43:30.587 INFO:tasks.workunit.client.0.smithi160.stdout:tuned 758 root NOFD /proc/758/fd (opendir: Permission denied) 2024-03-07T07:43:30.587 INFO:tasks.workunit.client.0.smithi160.stdout:tuned 758 863 tuned root cwd unknown /proc/758/task/863/cwd (readlink: Permission denied) 2024-03-07T07:43:30.587 INFO:tasks.workunit.client.0.smithi160.stdout:tuned 758 863 tuned root rtd unknown /proc/758/task/863/root (readlink: Permission denied) 2024-03-07T07:43:30.587 INFO:tasks.workunit.client.0.smithi160.stdout:tuned 758 863 tuned root txt unknown /proc/758/task/863/exe (readlink: Permission denied) 2024-03-07T07:43:30.587 INFO:tasks.workunit.client.0.smithi160.stdout:tuned 758 863 tuned root NOFD /proc/758/task/863/fd (opendir: Permission denied) 2024-03-07T07:43:30.588 INFO:tasks.workunit.client.0.smithi160.stdout:gssproxy 782 root cwd unknown /proc/782/cwd (readlink: Permission denied) 2024-03-07T07:43:30.588 INFO:tasks.workunit.client.0.smithi160.stdout:gssproxy 782 root rtd unknown /proc/782/root (readlink: Permission denied) 2024-03-07T07:43:30.588 INFO:tasks.workunit.client.0.smithi160.stdout:gssproxy 782 root txt unknown /proc/782/exe (readlink: Permission denied) 2024-03-07T07:43:30.588 INFO:tasks.workunit.client.0.smithi160.stdout:gssproxy 782 root NOFD /proc/782/fd (opendir: Permission denied) 2024-03-07T07:43:30.588 INFO:tasks.workunit.client.0.smithi160.stdout:gssproxy 782 783 gssproxy root cwd unknown /proc/782/task/783/cwd (readlink: Permission denied) 2024-03-07T07:43:30.588 INFO:tasks.workunit.client.0.smithi160.stdout:gssproxy 782 783 gssproxy root rtd unknown /proc/782/task/783/root (readlink: Permission denied) 2024-03-07T07:43:30.588 INFO:tasks.workunit.client.0.smithi160.stdout:gssproxy 782 783 gssproxy root txt unknown /proc/782/task/783/exe (readlink: Permission denied) 2024-03-07T07:43:30.588 INFO:tasks.workunit.client.0.smithi160.stdout:gssproxy 782 783 gssproxy root NOFD /proc/782/task/783/fd (opendir: Permission denied) 2024-03-07T07:43:30.588 INFO:tasks.workunit.client.0.smithi160.stdout:gssproxy 782 784 gssproxy root cwd unknown /proc/782/task/784/cwd (readlink: Permission denied) 2024-03-07T07:43:30.588 INFO:tasks.workunit.client.0.smithi160.stdout:gssproxy 782 784 gssproxy root rtd unknown /proc/782/task/784/root (readlink: Permission denied) 2024-03-07T07:43:30.588 INFO:tasks.workunit.client.0.smithi160.stdout:gssproxy 782 784 gssproxy root txt unknown /proc/782/task/784/exe (readlink: Permission denied) 2024-03-07T07:43:30.588 INFO:tasks.workunit.client.0.smithi160.stdout:gssproxy 782 784 gssproxy root NOFD /proc/782/task/784/fd (opendir: Permission denied) 2024-03-07T07:43:30.588 INFO:tasks.workunit.client.0.smithi160.stdout:gssproxy 782 785 gssproxy root cwd unknown /proc/782/task/785/cwd (readlink: Permission denied) 2024-03-07T07:43:30.588 INFO:tasks.workunit.client.0.smithi160.stdout:gssproxy 782 785 gssproxy root rtd unknown /proc/782/task/785/root (readlink: Permission denied) 2024-03-07T07:43:30.589 INFO:tasks.workunit.client.0.smithi160.stdout:gssproxy 782 785 gssproxy root txt unknown /proc/782/task/785/exe (readlink: Permission denied) 2024-03-07T07:43:30.589 INFO:tasks.workunit.client.0.smithi160.stdout:gssproxy 782 785 gssproxy root NOFD /proc/782/task/785/fd (opendir: Permission denied) 2024-03-07T07:43:30.589 INFO:tasks.workunit.client.0.smithi160.stdout:gssproxy 782 786 gssproxy root cwd unknown /proc/782/task/786/cwd (readlink: Permission denied) 2024-03-07T07:43:30.589 INFO:tasks.workunit.client.0.smithi160.stdout:gssproxy 782 786 gssproxy root rtd unknown /proc/782/task/786/root (readlink: Permission denied) 2024-03-07T07:43:30.589 INFO:tasks.workunit.client.0.smithi160.stdout:gssproxy 782 786 gssproxy root txt unknown /proc/782/task/786/exe (readlink: Permission denied) 2024-03-07T07:43:30.589 INFO:tasks.workunit.client.0.smithi160.stdout:gssproxy 782 786 gssproxy root NOFD /proc/782/task/786/fd (opendir: Permission denied) 2024-03-07T07:43:30.589 INFO:tasks.workunit.client.0.smithi160.stdout:gssproxy 782 787 gssproxy root cwd unknown /proc/782/task/787/cwd (readlink: Permission denied) 2024-03-07T07:43:30.589 INFO:tasks.workunit.client.0.smithi160.stdout:gssproxy 782 787 gssproxy root rtd unknown /proc/782/task/787/root (readlink: Permission denied) 2024-03-07T07:43:30.589 INFO:tasks.workunit.client.0.smithi160.stdout:gssproxy 782 787 gssproxy root txt unknown /proc/782/task/787/exe (readlink: Permission denied) 2024-03-07T07:43:30.589 INFO:tasks.workunit.client.0.smithi160.stdout:gssproxy 782 787 gssproxy root NOFD /proc/782/task/787/fd (opendir: Permission denied) 2024-03-07T07:43:30.589 INFO:tasks.workunit.client.0.smithi160.stdout:atd 789 root cwd unknown /proc/789/cwd (readlink: Permission denied) 2024-03-07T07:43:30.589 INFO:tasks.workunit.client.0.smithi160.stdout:atd 789 root rtd unknown /proc/789/root (readlink: Permission denied) 2024-03-07T07:43:30.589 INFO:tasks.workunit.client.0.smithi160.stdout:atd 789 root txt unknown /proc/789/exe (readlink: Permission denied) 2024-03-07T07:43:30.589 INFO:tasks.workunit.client.0.smithi160.stdout:atd 789 root NOFD /proc/789/fd (opendir: Permission denied) 2024-03-07T07:43:30.590 INFO:tasks.workunit.client.0.smithi160.stdout:crond 790 root cwd unknown /proc/790/cwd (readlink: Permission denied) 2024-03-07T07:43:30.590 INFO:tasks.workunit.client.0.smithi160.stdout:crond 790 root rtd unknown /proc/790/root (readlink: Permission denied) 2024-03-07T07:43:30.590 INFO:tasks.workunit.client.0.smithi160.stdout:crond 790 root txt unknown /proc/790/exe (readlink: Permission denied) 2024-03-07T07:43:30.590 INFO:tasks.workunit.client.0.smithi160.stdout:crond 790 root NOFD /proc/790/fd (opendir: Permission denied) 2024-03-07T07:43:30.590 INFO:tasks.workunit.client.0.smithi160.stdout:nrpe 884 nrpe cwd unknown /proc/884/cwd (readlink: Permission denied) 2024-03-07T07:43:30.590 INFO:tasks.workunit.client.0.smithi160.stdout:nrpe 884 nrpe rtd unknown /proc/884/root (readlink: Permission denied) 2024-03-07T07:43:30.590 INFO:tasks.workunit.client.0.smithi160.stdout:nrpe 884 nrpe txt unknown /proc/884/exe (readlink: Permission denied) 2024-03-07T07:43:30.590 INFO:tasks.workunit.client.0.smithi160.stdout:nrpe 884 nrpe NOFD /proc/884/fd (opendir: Permission denied) 2024-03-07T07:43:30.590 INFO:tasks.workunit.client.0.smithi160.stdout:agetty 1331 root cwd unknown /proc/1331/cwd (readlink: Permission denied) 2024-03-07T07:43:30.590 INFO:tasks.workunit.client.0.smithi160.stdout:agetty 1331 root rtd unknown /proc/1331/root (readlink: Permission denied) 2024-03-07T07:43:30.590 INFO:tasks.workunit.client.0.smithi160.stdout:agetty 1331 root txt unknown /proc/1331/exe (readlink: Permission denied) 2024-03-07T07:43:30.590 INFO:tasks.workunit.client.0.smithi160.stdout:agetty 1331 root NOFD /proc/1331/fd (opendir: Permission denied) 2024-03-07T07:43:30.590 INFO:tasks.workunit.client.0.smithi160.stdout:agetty 1334 root cwd unknown /proc/1334/cwd (readlink: Permission denied) 2024-03-07T07:43:30.590 INFO:tasks.workunit.client.0.smithi160.stdout:agetty 1334 root rtd unknown /proc/1334/root (readlink: Permission denied) 2024-03-07T07:43:30.590 INFO:tasks.workunit.client.0.smithi160.stdout:agetty 1334 root txt unknown /proc/1334/exe (readlink: Permission denied) 2024-03-07T07:43:30.591 INFO:tasks.workunit.client.0.smithi160.stdout:agetty 1334 root NOFD /proc/1334/fd (opendir: Permission denied) 2024-03-07T07:43:30.591 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu cwd DIR 8,1 4096 2 / 2024-03-07T07:43:30.591 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu rtd DIR 8,1 4096 2 / 2024-03-07T07:43:30.591 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu txt REG 8,1 102120 20353 /usr/lib/systemd/systemd 2024-03-07T07:43:30.591 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu DEL REG 8,1 1167 /etc/selinux/targeted/contexts/files/file_contexts.bin 2024-03-07T07:43:30.591 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2024-03-07T07:43:30.591 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2024-03-07T07:43:30.591 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2024-03-07T07:43:30.591 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu DEL REG 8,1 10007 /usr/lib64/libcrypto.so.3.0.7 2024-03-07T07:43:30.591 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu DEL REG 8,1 1697 /etc/selinux/targeted/contexts/files/file_contexts.homedirs.bin 2024-03-07T07:43:30.591 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2024-03-07T07:43:30.591 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2024-03-07T07:43:30.592 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu mem REG 8,1 1293840 9789 /usr/lib64/libp11-kit.so.0.3.0 2024-03-07T07:43:30.592 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2024-03-07T07:43:30.592 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2024-03-07T07:43:30.592 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2024-03-07T07:43:30.592 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2024-03-07T07:43:30.592 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2024-03-07T07:43:30.592 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2024-03-07T07:43:30.592 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu DEL REG 8,1 10126 /usr/lib64/libblkid.so.1.1.0 2024-03-07T07:43:30.592 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu mem REG 8,1 3645472 49253 /usr/lib64/systemd/libsystemd-shared-252.so 2024-03-07T07:43:30.592 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu mem REG 8,1 28552 9804 /usr/lib64/libattr.so.1.1.2501 2024-03-07T07:43:30.592 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2024-03-07T07:43:30.592 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu mem REG 8,1 40424 10956 /usr/lib64/libeconf.so.0.4.1 2024-03-07T07:43:30.592 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu DEL REG 8,1 10156 /usr/lib64/libmount.so.1.1.0 2024-03-07T07:43:30.592 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2024-03-07T07:43:30.593 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu mem REG 8,1 2174088 49251 /usr/lib64/systemd/libsystemd-core-252.so 2024-03-07T07:43:30.593 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2024-03-07T07:43:30.593 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu DEL REG 8,1 9806 /usr/lib64/libacl.so.1.1.2301 2024-03-07T07:43:30.593 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu mem REG 8,1 107424 10180 /usr/lib64/libkmod.so.2.3.6 2024-03-07T07:43:30.593 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2024-03-07T07:43:30.593 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu mem REG 8,1 69472 10965 /usr/lib64/libpam.so.0.85.1 2024-03-07T07:43:30.593 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu mem REG 8,1 127680 9972 /usr/lib64/libseccomp.so.2.5.2 2024-03-07T07:43:30.593 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2024-03-07T07:43:30.593 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2024-03-07T07:43:30.593 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu mem REG 0,19 0 19 /sys/fs/selinux/status 2024-03-07T07:43:30.593 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu 0r CHR 1,3 0t0 4 /dev/null 2024-03-07T07:43:30.593 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu 1u unix 0x0000000000000000 0t0 22070 type=STREAM (CONNECTED) 2024-03-07T07:43:30.593 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu 2u unix 0x0000000000000000 0t0 22070 type=STREAM (CONNECTED) 2024-03-07T07:43:30.594 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu 3u unix 0x0000000000000000 0t0 22098 type=DGRAM (CONNECTED) 2024-03-07T07:43:30.594 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu 4u a_inode 0,14 0 2078 [eventpoll:5,6,8,9,11,12,14,15,16,17,19,20,21] 2024-03-07T07:43:30.594 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu 5u a_inode 0,14 0 2078 [signalfd] 2024-03-07T07:43:30.594 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu 6r a_inode 0,14 0 2078 inotify 2024-03-07T07:43:30.594 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu 7r DIR 0,25 0 3393 /sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service 2024-03-07T07:43:30.594 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu 8u a_inode 0,14 0 2078 [timerfd] 2024-03-07T07:43:30.594 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu 9u a_inode 0,14 0 2078 [eventpoll:10,13] 2024-03-07T07:43:30.594 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu 10r REG 0,20 0 22087 /proc/4843/mountinfo 2024-03-07T07:43:30.594 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu 11r a_inode 0,14 0 2078 inotify 2024-03-07T07:43:30.594 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu 12u unix 0x0000000000000000 0t0 22120 /run/user/1000/bus type=STREAM (LISTEN) 2024-03-07T07:43:30.594 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu 13r a_inode 0,14 0 2078 inotify 2024-03-07T07:43:30.594 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu 14r REG 0,20 0 4026532076 /proc/swaps 2024-03-07T07:43:30.594 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu 15u netlink 0t0 22102 KOBJECT_UEVENT 2024-03-07T07:43:30.594 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu 16u unix 0x0000000000000000 0t0 22107 /run/user/1000/systemd/notify type=DGRAM (UNCONNECTED) 2024-03-07T07:43:30.595 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu 17u unix 0x0000000000000000 0t0 22108 type=DGRAM (CONNECTED) 2024-03-07T07:43:30.595 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu 18u unix 0x0000000000000000 0t0 22109 type=DGRAM (CONNECTED) 2024-03-07T07:43:30.595 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu 19u unix 0x0000000000000000 0t0 22110 /run/user/1000/systemd/private type=STREAM (LISTEN) 2024-03-07T07:43:30.595 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu 20u unix 0x0000000000000000 0t0 22112 type=STREAM (CONNECTED) 2024-03-07T07:43:30.595 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 4843 ubuntu 21u a_inode 0,14 0 2078 [timerfd] 2024-03-07T07:43:30.595 INFO:tasks.workunit.client.0.smithi160.stdout:(sd-pam) 4845 ubuntu cwd unknown /proc/4845/cwd (readlink: Permission denied) 2024-03-07T07:43:30.595 INFO:tasks.workunit.client.0.smithi160.stdout:(sd-pam) 4845 ubuntu rtd unknown /proc/4845/root (readlink: Permission denied) 2024-03-07T07:43:30.595 INFO:tasks.workunit.client.0.smithi160.stdout:(sd-pam) 4845 ubuntu txt unknown /proc/4845/exe (readlink: Permission denied) 2024-03-07T07:43:30.595 INFO:tasks.workunit.client.0.smithi160.stdout:(sd-pam) 4845 ubuntu NOFD /proc/4845/fd (opendir: Permission denied) 2024-03-07T07:43:30.595 INFO:tasks.workunit.client.0.smithi160.stdout:rsyslogd 5088 root cwd unknown /proc/5088/cwd (readlink: Permission denied) 2024-03-07T07:43:30.595 INFO:tasks.workunit.client.0.smithi160.stdout:rsyslogd 5088 root rtd unknown /proc/5088/root (readlink: Permission denied) 2024-03-07T07:43:30.595 INFO:tasks.workunit.client.0.smithi160.stdout:rsyslogd 5088 root txt unknown /proc/5088/exe (readlink: Permission denied) 2024-03-07T07:43:30.595 INFO:tasks.workunit.client.0.smithi160.stdout:rsyslogd 5088 root NOFD /proc/5088/fd (opendir: Permission denied) 2024-03-07T07:43:30.595 INFO:tasks.workunit.client.0.smithi160.stdout:rsyslogd 5088 5090 in:imjour root cwd unknown /proc/5088/task/5090/cwd (readlink: Permission denied) 2024-03-07T07:43:30.596 INFO:tasks.workunit.client.0.smithi160.stdout:rsyslogd 5088 5090 in:imjour root rtd unknown /proc/5088/task/5090/root (readlink: Permission denied) 2024-03-07T07:43:30.596 INFO:tasks.workunit.client.0.smithi160.stdout:rsyslogd 5088 5090 in:imjour root txt unknown /proc/5088/task/5090/exe (readlink: Permission denied) 2024-03-07T07:43:30.596 INFO:tasks.workunit.client.0.smithi160.stdout:rsyslogd 5088 5090 in:imjour root NOFD /proc/5088/task/5090/fd (opendir: Permission denied) 2024-03-07T07:43:30.596 INFO:tasks.workunit.client.0.smithi160.stdout:rsyslogd 5088 5091 rs:main root cwd unknown /proc/5088/task/5091/cwd (readlink: Permission denied) 2024-03-07T07:43:30.596 INFO:tasks.workunit.client.0.smithi160.stdout:rsyslogd 5088 5091 rs:main root rtd unknown /proc/5088/task/5091/root (readlink: Permission denied) 2024-03-07T07:43:30.596 INFO:tasks.workunit.client.0.smithi160.stdout:rsyslogd 5088 5091 rs:main root txt unknown /proc/5088/task/5091/exe (readlink: Permission denied) 2024-03-07T07:43:30.596 INFO:tasks.workunit.client.0.smithi160.stdout:rsyslogd 5088 5091 rs:main root NOFD /proc/5088/task/5091/fd (opendir: Permission denied) 2024-03-07T07:43:30.596 INFO:tasks.workunit.client.0.smithi160.stdout:tls-strp 15778 root cwd unknown /proc/15778/cwd (readlink: Permission denied) 2024-03-07T07:43:30.596 INFO:tasks.workunit.client.0.smithi160.stdout:tls-strp 15778 root rtd unknown /proc/15778/root (readlink: Permission denied) 2024-03-07T07:43:30.596 INFO:tasks.workunit.client.0.smithi160.stdout:tls-strp 15778 root txt unknown /proc/15778/exe (readlink: Permission denied) 2024-03-07T07:43:30.596 INFO:tasks.workunit.client.0.smithi160.stdout:tls-strp 15778 root NOFD /proc/15778/fd (opendir: Permission denied) 2024-03-07T07:43:30.596 INFO:tasks.workunit.client.0.smithi160.stdout:smartd 15826 root cwd unknown /proc/15826/cwd (readlink: Permission denied) 2024-03-07T07:43:30.596 INFO:tasks.workunit.client.0.smithi160.stdout:smartd 15826 root rtd unknown /proc/15826/root (readlink: Permission denied) 2024-03-07T07:43:30.596 INFO:tasks.workunit.client.0.smithi160.stdout:smartd 15826 root txt unknown /proc/15826/exe (readlink: Permission denied) 2024-03-07T07:43:30.597 INFO:tasks.workunit.client.0.smithi160.stdout:smartd 15826 root NOFD /proc/15826/fd (opendir: Permission denied) 2024-03-07T07:43:30.597 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/2 19074 root cwd unknown /proc/19074/cwd (readlink: Permission denied) 2024-03-07T07:43:30.597 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/2 19074 root rtd unknown /proc/19074/root (readlink: Permission denied) 2024-03-07T07:43:30.597 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/2 19074 root txt unknown /proc/19074/exe (readlink: Permission denied) 2024-03-07T07:43:30.597 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/2 19074 root NOFD /proc/19074/fd (opendir: Permission denied) 2024-03-07T07:43:30.597 INFO:tasks.workunit.client.0.smithi160.stdout:kdmflush/ 21188 root cwd unknown /proc/21188/cwd (readlink: Permission denied) 2024-03-07T07:43:30.597 INFO:tasks.workunit.client.0.smithi160.stdout:kdmflush/ 21188 root rtd unknown /proc/21188/root (readlink: Permission denied) 2024-03-07T07:43:30.597 INFO:tasks.workunit.client.0.smithi160.stdout:kdmflush/ 21188 root txt unknown /proc/21188/exe (readlink: Permission denied) 2024-03-07T07:43:30.597 INFO:tasks.workunit.client.0.smithi160.stdout:kdmflush/ 21188 root NOFD /proc/21188/fd (opendir: Permission denied) 2024-03-07T07:43:30.597 INFO:tasks.workunit.client.0.smithi160.stdout:kdmflush/ 21217 root cwd unknown /proc/21217/cwd (readlink: Permission denied) 2024-03-07T07:43:30.597 INFO:tasks.workunit.client.0.smithi160.stdout:kdmflush/ 21217 root rtd unknown /proc/21217/root (readlink: Permission denied) 2024-03-07T07:43:30.597 INFO:tasks.workunit.client.0.smithi160.stdout:kdmflush/ 21217 root txt unknown /proc/21217/exe (readlink: Permission denied) 2024-03-07T07:43:30.597 INFO:tasks.workunit.client.0.smithi160.stdout:kdmflush/ 21217 root NOFD /proc/21217/fd (opendir: Permission denied) 2024-03-07T07:43:30.598 INFO:tasks.workunit.client.0.smithi160.stdout:kdmflush/ 21246 root cwd unknown /proc/21246/cwd (readlink: Permission denied) 2024-03-07T07:43:30.598 INFO:tasks.workunit.client.0.smithi160.stdout:kdmflush/ 21246 root rtd unknown /proc/21246/root (readlink: Permission denied) 2024-03-07T07:43:30.598 INFO:tasks.workunit.client.0.smithi160.stdout:kdmflush/ 21246 root txt unknown /proc/21246/exe (readlink: Permission denied) 2024-03-07T07:43:30.598 INFO:tasks.workunit.client.0.smithi160.stdout:kdmflush/ 21246 root NOFD /proc/21246/fd (opendir: Permission denied) 2024-03-07T07:43:30.598 INFO:tasks.workunit.client.0.smithi160.stdout:kdmflush/ 21275 root cwd unknown /proc/21275/cwd (readlink: Permission denied) 2024-03-07T07:43:30.598 INFO:tasks.workunit.client.0.smithi160.stdout:kdmflush/ 21275 root rtd unknown /proc/21275/root (readlink: Permission denied) 2024-03-07T07:43:30.598 INFO:tasks.workunit.client.0.smithi160.stdout:kdmflush/ 21275 root txt unknown /proc/21275/exe (readlink: Permission denied) 2024-03-07T07:43:30.598 INFO:tasks.workunit.client.0.smithi160.stdout:kdmflush/ 21275 root NOFD /proc/21275/fd (opendir: Permission denied) 2024-03-07T07:43:30.598 INFO:tasks.workunit.client.0.smithi160.stdout:kdmflush/ 21304 root cwd unknown /proc/21304/cwd (readlink: Permission denied) 2024-03-07T07:43:30.598 INFO:tasks.workunit.client.0.smithi160.stdout:kdmflush/ 21304 root rtd unknown /proc/21304/root (readlink: Permission denied) 2024-03-07T07:43:30.598 INFO:tasks.workunit.client.0.smithi160.stdout:kdmflush/ 21304 root txt unknown /proc/21304/exe (readlink: Permission denied) 2024-03-07T07:43:30.598 INFO:tasks.workunit.client.0.smithi160.stdout:kdmflush/ 21304 root NOFD /proc/21304/fd (opendir: Permission denied) 2024-03-07T07:43:30.598 INFO:tasks.workunit.client.0.smithi160.stdout:xfsalloc 21468 root cwd unknown /proc/21468/cwd (readlink: Permission denied) 2024-03-07T07:43:30.598 INFO:tasks.workunit.client.0.smithi160.stdout:xfsalloc 21468 root rtd unknown /proc/21468/root (readlink: Permission denied) 2024-03-07T07:43:30.599 INFO:tasks.workunit.client.0.smithi160.stdout:xfsalloc 21468 root txt unknown /proc/21468/exe (readlink: Permission denied) 2024-03-07T07:43:30.599 INFO:tasks.workunit.client.0.smithi160.stdout:xfsalloc 21468 root NOFD /proc/21468/fd (opendir: Permission denied) 2024-03-07T07:43:30.599 INFO:tasks.workunit.client.0.smithi160.stdout:xfs_mru_c 21469 root cwd unknown /proc/21469/cwd (readlink: Permission denied) 2024-03-07T07:43:30.599 INFO:tasks.workunit.client.0.smithi160.stdout:xfs_mru_c 21469 root rtd unknown /proc/21469/root (readlink: Permission denied) 2024-03-07T07:43:30.599 INFO:tasks.workunit.client.0.smithi160.stdout:xfs_mru_c 21469 root txt unknown /proc/21469/exe (readlink: Permission denied) 2024-03-07T07:43:30.599 INFO:tasks.workunit.client.0.smithi160.stdout:xfs_mru_c 21469 root NOFD /proc/21469/fd (opendir: Permission denied) 2024-03-07T07:43:30.599 INFO:tasks.workunit.client.0.smithi160.stdout:xfs-buf/d 21470 root cwd unknown /proc/21470/cwd (readlink: Permission denied) 2024-03-07T07:43:30.599 INFO:tasks.workunit.client.0.smithi160.stdout:xfs-buf/d 21470 root rtd unknown /proc/21470/root (readlink: Permission denied) 2024-03-07T07:43:30.599 INFO:tasks.workunit.client.0.smithi160.stdout:xfs-buf/d 21470 root txt unknown /proc/21470/exe (readlink: Permission denied) 2024-03-07T07:43:30.599 INFO:tasks.workunit.client.0.smithi160.stdout:xfs-buf/d 21470 root NOFD /proc/21470/fd (opendir: Permission denied) 2024-03-07T07:43:30.599 INFO:tasks.workunit.client.0.smithi160.stdout:xfs-conv/ 21471 root cwd unknown /proc/21471/cwd (readlink: Permission denied) 2024-03-07T07:43:30.599 INFO:tasks.workunit.client.0.smithi160.stdout:xfs-conv/ 21471 root rtd unknown /proc/21471/root (readlink: Permission denied) 2024-03-07T07:43:30.599 INFO:tasks.workunit.client.0.smithi160.stdout:xfs-conv/ 21471 root txt unknown /proc/21471/exe (readlink: Permission denied) 2024-03-07T07:43:30.599 INFO:tasks.workunit.client.0.smithi160.stdout:xfs-conv/ 21471 root NOFD /proc/21471/fd (opendir: Permission denied) 2024-03-07T07:43:30.600 INFO:tasks.workunit.client.0.smithi160.stdout:xfs-recla 21472 root cwd unknown /proc/21472/cwd (readlink: Permission denied) 2024-03-07T07:43:30.600 INFO:tasks.workunit.client.0.smithi160.stdout:xfs-recla 21472 root rtd unknown /proc/21472/root (readlink: Permission denied) 2024-03-07T07:43:30.600 INFO:tasks.workunit.client.0.smithi160.stdout:xfs-recla 21472 root txt unknown /proc/21472/exe (readlink: Permission denied) 2024-03-07T07:43:30.600 INFO:tasks.workunit.client.0.smithi160.stdout:xfs-recla 21472 root NOFD /proc/21472/fd (opendir: Permission denied) 2024-03-07T07:43:30.600 INFO:tasks.workunit.client.0.smithi160.stdout:xfs-block 21473 root cwd unknown /proc/21473/cwd (readlink: Permission denied) 2024-03-07T07:43:30.600 INFO:tasks.workunit.client.0.smithi160.stdout:xfs-block 21473 root rtd unknown /proc/21473/root (readlink: Permission denied) 2024-03-07T07:43:30.600 INFO:tasks.workunit.client.0.smithi160.stdout:xfs-block 21473 root txt unknown /proc/21473/exe (readlink: Permission denied) 2024-03-07T07:43:30.600 INFO:tasks.workunit.client.0.smithi160.stdout:xfs-block 21473 root NOFD /proc/21473/fd (opendir: Permission denied) 2024-03-07T07:43:30.600 INFO:tasks.workunit.client.0.smithi160.stdout:xfs-inode 21474 root cwd unknown /proc/21474/cwd (readlink: Permission denied) 2024-03-07T07:43:30.600 INFO:tasks.workunit.client.0.smithi160.stdout:xfs-inode 21474 root rtd unknown /proc/21474/root (readlink: Permission denied) 2024-03-07T07:43:30.600 INFO:tasks.workunit.client.0.smithi160.stdout:xfs-inode 21474 root txt unknown /proc/21474/exe (readlink: Permission denied) 2024-03-07T07:43:30.600 INFO:tasks.workunit.client.0.smithi160.stdout:xfs-inode 21474 root NOFD /proc/21474/fd (opendir: Permission denied) 2024-03-07T07:43:30.600 INFO:tasks.workunit.client.0.smithi160.stdout:xfs-log/d 21475 root cwd unknown /proc/21475/cwd (readlink: Permission denied) 2024-03-07T07:43:30.601 INFO:tasks.workunit.client.0.smithi160.stdout:xfs-log/d 21475 root rtd unknown /proc/21475/root (readlink: Permission denied) 2024-03-07T07:43:30.601 INFO:tasks.workunit.client.0.smithi160.stdout:xfs-log/d 21475 root txt unknown /proc/21475/exe (readlink: Permission denied) 2024-03-07T07:43:30.601 INFO:tasks.workunit.client.0.smithi160.stdout:xfs-log/d 21475 root NOFD /proc/21475/fd (opendir: Permission denied) 2024-03-07T07:43:30.601 INFO:tasks.workunit.client.0.smithi160.stdout:xfs-cil/d 21476 root cwd unknown /proc/21476/cwd (readlink: Permission denied) 2024-03-07T07:43:30.601 INFO:tasks.workunit.client.0.smithi160.stdout:xfs-cil/d 21476 root rtd unknown /proc/21476/root (readlink: Permission denied) 2024-03-07T07:43:30.601 INFO:tasks.workunit.client.0.smithi160.stdout:xfs-cil/d 21476 root txt unknown /proc/21476/exe (readlink: Permission denied) 2024-03-07T07:43:30.601 INFO:tasks.workunit.client.0.smithi160.stdout:xfs-cil/d 21476 root NOFD /proc/21476/fd (opendir: Permission denied) 2024-03-07T07:43:30.601 INFO:tasks.workunit.client.0.smithi160.stdout:xfsaild/d 21477 root cwd unknown /proc/21477/cwd (readlink: Permission denied) 2024-03-07T07:43:30.601 INFO:tasks.workunit.client.0.smithi160.stdout:xfsaild/d 21477 root rtd unknown /proc/21477/root (readlink: Permission denied) 2024-03-07T07:43:30.601 INFO:tasks.workunit.client.0.smithi160.stdout:xfsaild/d 21477 root txt unknown /proc/21477/exe (readlink: Permission denied) 2024-03-07T07:43:30.601 INFO:tasks.workunit.client.0.smithi160.stdout:xfsaild/d 21477 root NOFD /proc/21477/fd (opendir: Permission denied) 2024-03-07T07:43:30.601 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/4 23255 root cwd unknown /proc/23255/cwd (readlink: Permission denied) 2024-03-07T07:43:30.601 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/4 23255 root rtd unknown /proc/23255/root (readlink: Permission denied) 2024-03-07T07:43:30.601 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/4 23255 root txt unknown /proc/23255/exe (readlink: Permission denied) 2024-03-07T07:43:30.602 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/4 23255 root NOFD /proc/23255/fd (opendir: Permission denied) 2024-03-07T07:43:30.602 INFO:tasks.workunit.client.0.smithi160.stdout:sshd 23412 root cwd unknown /proc/23412/cwd (readlink: Permission denied) 2024-03-07T07:43:30.602 INFO:tasks.workunit.client.0.smithi160.stdout:sshd 23412 root rtd unknown /proc/23412/root (readlink: Permission denied) 2024-03-07T07:43:30.602 INFO:tasks.workunit.client.0.smithi160.stdout:sshd 23412 root txt unknown /proc/23412/exe (readlink: Permission denied) 2024-03-07T07:43:30.602 INFO:tasks.workunit.client.0.smithi160.stdout:sshd 23412 root NOFD /proc/23412/fd (opendir: Permission denied) 2024-03-07T07:43:30.602 INFO:tasks.workunit.client.0.smithi160.stdout:sshd 23415 ubuntu cwd unknown /proc/23415/cwd (readlink: Permission denied) 2024-03-07T07:43:30.602 INFO:tasks.workunit.client.0.smithi160.stdout:sshd 23415 ubuntu rtd unknown /proc/23415/root (readlink: Permission denied) 2024-03-07T07:43:30.602 INFO:tasks.workunit.client.0.smithi160.stdout:sshd 23415 ubuntu txt unknown /proc/23415/exe (readlink: Permission denied) 2024-03-07T07:43:30.602 INFO:tasks.workunit.client.0.smithi160.stdout:sshd 23415 ubuntu NOFD /proc/23415/fd (opendir: Permission denied) 2024-03-07T07:43:30.602 INFO:tasks.workunit.client.0.smithi160.stdout:chronyd 23475 chrony cwd unknown /proc/23475/cwd (readlink: Permission denied) 2024-03-07T07:43:30.602 INFO:tasks.workunit.client.0.smithi160.stdout:chronyd 23475 chrony rtd unknown /proc/23475/root (readlink: Permission denied) 2024-03-07T07:43:30.602 INFO:tasks.workunit.client.0.smithi160.stdout:chronyd 23475 chrony txt unknown /proc/23475/exe (readlink: Permission denied) 2024-03-07T07:43:30.602 INFO:tasks.workunit.client.0.smithi160.stdout:chronyd 23475 chrony NOFD /proc/23475/fd (opendir: Permission denied) 2024-03-07T07:43:30.602 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-cras 23821 ceph cwd unknown /proc/23821/cwd (readlink: Permission denied) 2024-03-07T07:43:30.603 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-cras 23821 ceph rtd unknown /proc/23821/root (readlink: Permission denied) 2024-03-07T07:43:30.603 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-cras 23821 ceph txt unknown /proc/23821/exe (readlink: Permission denied) 2024-03-07T07:43:30.603 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-cras 23821 ceph NOFD /proc/23821/fd (opendir: Permission denied) 2024-03-07T07:43:30.603 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/1 25817 root cwd unknown /proc/25817/cwd (readlink: Permission denied) 2024-03-07T07:43:30.603 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/1 25817 root rtd unknown /proc/25817/root (readlink: Permission denied) 2024-03-07T07:43:30.603 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/1 25817 root txt unknown /proc/25817/exe (readlink: Permission denied) 2024-03-07T07:43:30.603 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/1 25817 root NOFD /proc/25817/fd (opendir: Permission denied) 2024-03-07T07:43:30.603 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/5 30575 root cwd unknown /proc/30575/cwd (readlink: Permission denied) 2024-03-07T07:43:30.603 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/5 30575 root rtd unknown /proc/30575/root (readlink: Permission denied) 2024-03-07T07:43:30.603 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/5 30575 root txt unknown /proc/30575/exe (readlink: Permission denied) 2024-03-07T07:43:30.603 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/5 30575 root NOFD /proc/30575/fd (opendir: Permission denied) 2024-03-07T07:43:30.603 INFO:tasks.workunit.client.0.smithi160.stdout:sudo 30926 root cwd unknown /proc/30926/cwd (readlink: Permission denied) 2024-03-07T07:43:30.603 INFO:tasks.workunit.client.0.smithi160.stdout:sudo 30926 root rtd unknown /proc/30926/root (readlink: Permission denied) 2024-03-07T07:43:30.603 INFO:tasks.workunit.client.0.smithi160.stdout:sudo 30926 root txt unknown /proc/30926/exe (readlink: Permission denied) 2024-03-07T07:43:30.604 INFO:tasks.workunit.client.0.smithi160.stdout:sudo 30926 root NOFD /proc/30926/fd (opendir: Permission denied) 2024-03-07T07:43:30.604 INFO:tasks.workunit.client.0.smithi160.stdout:journalct 30943 root cwd unknown /proc/30943/cwd (readlink: Permission denied) 2024-03-07T07:43:30.604 INFO:tasks.workunit.client.0.smithi160.stdout:journalct 30943 root rtd unknown /proc/30943/root (readlink: Permission denied) 2024-03-07T07:43:30.604 INFO:tasks.workunit.client.0.smithi160.stdout:journalct 30943 root txt unknown /proc/30943/exe (readlink: Permission denied) 2024-03-07T07:43:30.604 INFO:tasks.workunit.client.0.smithi160.stdout:journalct 30943 root NOFD /proc/30943/fd (opendir: Permission denied) 2024-03-07T07:43:30.604 INFO:tasks.workunit.client.0.smithi160.stdout:sudo 30944 root cwd unknown /proc/30944/cwd (readlink: Permission denied) 2024-03-07T07:43:30.604 INFO:tasks.workunit.client.0.smithi160.stdout:sudo 30944 root rtd unknown /proc/30944/root (readlink: Permission denied) 2024-03-07T07:43:30.604 INFO:tasks.workunit.client.0.smithi160.stdout:sudo 30944 root txt unknown /proc/30944/exe (readlink: Permission denied) 2024-03-07T07:43:30.604 INFO:tasks.workunit.client.0.smithi160.stdout:sudo 30944 root NOFD /proc/30944/fd (opendir: Permission denied) 2024-03-07T07:43:30.604 INFO:tasks.workunit.client.0.smithi160.stdout:journalct 30961 root cwd unknown /proc/30961/cwd (readlink: Permission denied) 2024-03-07T07:43:30.604 INFO:tasks.workunit.client.0.smithi160.stdout:journalct 30961 root rtd unknown /proc/30961/root (readlink: Permission denied) 2024-03-07T07:43:30.604 INFO:tasks.workunit.client.0.smithi160.stdout:journalct 30961 root txt unknown /proc/30961/exe (readlink: Permission denied) 2024-03-07T07:43:30.604 INFO:tasks.workunit.client.0.smithi160.stdout:journalct 30961 root NOFD /proc/30961/fd (opendir: Permission denied) 2024-03-07T07:43:30.605 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/6 31907 root cwd unknown /proc/31907/cwd (readlink: Permission denied) 2024-03-07T07:43:30.605 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/6 31907 root rtd unknown /proc/31907/root (readlink: Permission denied) 2024-03-07T07:43:30.605 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/6 31907 root txt unknown /proc/31907/exe (readlink: Permission denied) 2024-03-07T07:43:30.605 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/6 31907 root NOFD /proc/31907/fd (opendir: Permission denied) 2024-03-07T07:43:30.605 INFO:tasks.workunit.client.0.smithi160.stdout:conmon 31959 root cwd unknown /proc/31959/cwd (readlink: Permission denied) 2024-03-07T07:43:30.605 INFO:tasks.workunit.client.0.smithi160.stdout:conmon 31959 root rtd unknown /proc/31959/root (readlink: Permission denied) 2024-03-07T07:43:30.605 INFO:tasks.workunit.client.0.smithi160.stdout:conmon 31959 root txt unknown /proc/31959/exe (readlink: Permission denied) 2024-03-07T07:43:30.605 INFO:tasks.workunit.client.0.smithi160.stdout:conmon 31959 root NOFD /proc/31959/fd (opendir: Permission denied) 2024-03-07T07:43:30.605 INFO:tasks.workunit.client.0.smithi160.stdout:podman-in 31961 root cwd unknown /proc/31961/cwd (readlink: Permission denied) 2024-03-07T07:43:30.605 INFO:tasks.workunit.client.0.smithi160.stdout:podman-in 31961 root rtd unknown /proc/31961/root (readlink: Permission denied) 2024-03-07T07:43:30.605 INFO:tasks.workunit.client.0.smithi160.stdout:podman-in 31961 root txt unknown /proc/31961/exe (readlink: Permission denied) 2024-03-07T07:43:30.605 INFO:tasks.workunit.client.0.smithi160.stdout:podman-in 31961 root NOFD /proc/31961/fd (opendir: Permission denied) 2024-03-07T07:43:30.605 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 ceph cwd unknown /proc/31963/cwd (readlink: Permission denied) 2024-03-07T07:43:30.605 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 ceph rtd unknown /proc/31963/root (readlink: Permission denied) 2024-03-07T07:43:30.605 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 ceph txt unknown /proc/31963/exe (readlink: Permission denied) 2024-03-07T07:43:30.606 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 ceph NOFD /proc/31963/fd (opendir: Permission denied) 2024-03-07T07:43:30.606 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31964 log ceph cwd unknown /proc/31963/task/31964/cwd (readlink: Permission denied) 2024-03-07T07:43:30.606 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31964 log ceph rtd unknown /proc/31963/task/31964/root (readlink: Permission denied) 2024-03-07T07:43:30.606 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31964 log ceph txt unknown /proc/31963/task/31964/exe (readlink: Permission denied) 2024-03-07T07:43:30.606 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31964 log ceph NOFD /proc/31963/task/31964/fd (opendir: Permission denied) 2024-03-07T07:43:30.606 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31965 service ceph cwd unknown /proc/31963/task/31965/cwd (readlink: Permission denied) 2024-03-07T07:43:30.606 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31965 service ceph rtd unknown /proc/31963/task/31965/root (readlink: Permission denied) 2024-03-07T07:43:30.606 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31965 service ceph txt unknown /proc/31963/task/31965/exe (readlink: Permission denied) 2024-03-07T07:43:30.606 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31965 service ceph NOFD /proc/31963/task/31965/fd (opendir: Permission denied) 2024-03-07T07:43:30.606 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31966 admin_soc ceph cwd unknown /proc/31963/task/31966/cwd (readlink: Permission denied) 2024-03-07T07:43:30.606 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31966 admin_soc ceph rtd unknown /proc/31963/task/31966/root (readlink: Permission denied) 2024-03-07T07:43:30.606 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31966 admin_soc ceph txt unknown /proc/31963/task/31966/exe (readlink: Permission denied) 2024-03-07T07:43:30.606 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31966 admin_soc ceph NOFD /proc/31963/task/31966/fd (opendir: Permission denied) 2024-03-07T07:43:30.606 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31967 signal_ha ceph cwd unknown /proc/31963/task/31967/cwd (readlink: Permission denied) 2024-03-07T07:43:30.607 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31967 signal_ha ceph rtd unknown /proc/31963/task/31967/root (readlink: Permission denied) 2024-03-07T07:43:30.607 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31967 signal_ha ceph txt unknown /proc/31963/task/31967/exe (readlink: Permission denied) 2024-03-07T07:43:30.607 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31967 signal_ha ceph NOFD /proc/31963/task/31967/fd (opendir: Permission denied) 2024-03-07T07:43:30.607 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31968 rocksdb:l ceph cwd unknown /proc/31963/task/31968/cwd (readlink: Permission denied) 2024-03-07T07:43:30.607 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31968 rocksdb:l ceph rtd unknown /proc/31963/task/31968/root (readlink: Permission denied) 2024-03-07T07:43:30.607 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31968 rocksdb:l ceph txt unknown /proc/31963/task/31968/exe (readlink: Permission denied) 2024-03-07T07:43:30.607 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31968 rocksdb:l ceph NOFD /proc/31963/task/31968/fd (opendir: Permission denied) 2024-03-07T07:43:30.607 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31969 rocksdb:h ceph cwd unknown /proc/31963/task/31969/cwd (readlink: Permission denied) 2024-03-07T07:43:30.607 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31969 rocksdb:h ceph rtd unknown /proc/31963/task/31969/root (readlink: Permission denied) 2024-03-07T07:43:30.607 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31969 rocksdb:h ceph txt unknown /proc/31963/task/31969/exe (readlink: Permission denied) 2024-03-07T07:43:30.607 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31969 rocksdb:h ceph NOFD /proc/31963/task/31969/fd (opendir: Permission denied) 2024-03-07T07:43:30.607 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31985 ceph-mon ceph cwd unknown /proc/31963/task/31985/cwd (readlink: Permission denied) 2024-03-07T07:43:30.607 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31985 ceph-mon ceph rtd unknown /proc/31963/task/31985/root (readlink: Permission denied) 2024-03-07T07:43:30.608 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31985 ceph-mon ceph txt unknown /proc/31963/task/31985/exe (readlink: Permission denied) 2024-03-07T07:43:30.608 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31985 ceph-mon ceph NOFD /proc/31963/task/31985/fd (opendir: Permission denied) 2024-03-07T07:43:30.608 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31986 fn_monsto ceph cwd unknown /proc/31963/task/31986/cwd (readlink: Permission denied) 2024-03-07T07:43:30.608 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31986 fn_monsto ceph rtd unknown /proc/31963/task/31986/root (readlink: Permission denied) 2024-03-07T07:43:30.608 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31986 fn_monsto ceph txt unknown /proc/31963/task/31986/exe (readlink: Permission denied) 2024-03-07T07:43:30.608 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31986 fn_monsto ceph NOFD /proc/31963/task/31986/fd (opendir: Permission denied) 2024-03-07T07:43:30.608 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31987 msgr-work ceph cwd unknown /proc/31963/task/31987/cwd (readlink: Permission denied) 2024-03-07T07:43:30.608 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31987 msgr-work ceph rtd unknown /proc/31963/task/31987/root (readlink: Permission denied) 2024-03-07T07:43:30.608 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31987 msgr-work ceph txt unknown /proc/31963/task/31987/exe (readlink: Permission denied) 2024-03-07T07:43:30.608 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31987 msgr-work ceph NOFD /proc/31963/task/31987/fd (opendir: Permission denied) 2024-03-07T07:43:30.608 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31988 msgr-work ceph cwd unknown /proc/31963/task/31988/cwd (readlink: Permission denied) 2024-03-07T07:43:30.608 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31988 msgr-work ceph rtd unknown /proc/31963/task/31988/root (readlink: Permission denied) 2024-03-07T07:43:30.608 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31988 msgr-work ceph txt unknown /proc/31963/task/31988/exe (readlink: Permission denied) 2024-03-07T07:43:30.608 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31988 msgr-work ceph NOFD /proc/31963/task/31988/fd (opendir: Permission denied) 2024-03-07T07:43:30.609 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31989 msgr-work ceph cwd unknown /proc/31963/task/31989/cwd (readlink: Permission denied) 2024-03-07T07:43:30.609 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31989 msgr-work ceph rtd unknown /proc/31963/task/31989/root (readlink: Permission denied) 2024-03-07T07:43:30.609 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31989 msgr-work ceph txt unknown /proc/31963/task/31989/exe (readlink: Permission denied) 2024-03-07T07:43:30.609 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31989 msgr-work ceph NOFD /proc/31963/task/31989/fd (opendir: Permission denied) 2024-03-07T07:43:30.609 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31990 OpHistory ceph cwd unknown /proc/31963/task/31990/cwd (readlink: Permission denied) 2024-03-07T07:43:30.609 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31990 OpHistory ceph rtd unknown /proc/31963/task/31990/root (readlink: Permission denied) 2024-03-07T07:43:30.609 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31990 OpHistory ceph txt unknown /proc/31963/task/31990/exe (readlink: Permission denied) 2024-03-07T07:43:30.609 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31990 OpHistory ceph NOFD /proc/31963/task/31990/fd (opendir: Permission denied) 2024-03-07T07:43:30.609 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31991 fin ceph cwd unknown /proc/31963/task/31991/cwd (readlink: Permission denied) 2024-03-07T07:43:30.609 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31991 fin ceph rtd unknown /proc/31963/task/31991/root (readlink: Permission denied) 2024-03-07T07:43:30.609 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31991 fin ceph txt unknown /proc/31963/task/31991/exe (readlink: Permission denied) 2024-03-07T07:43:30.609 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31991 fin ceph NOFD /proc/31963/task/31991/fd (opendir: Permission denied) 2024-03-07T07:43:30.609 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31992 safe_time ceph cwd unknown /proc/31963/task/31992/cwd (readlink: Permission denied) 2024-03-07T07:43:30.609 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31992 safe_time ceph rtd unknown /proc/31963/task/31992/root (readlink: Permission denied) 2024-03-07T07:43:30.610 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31992 safe_time ceph txt unknown /proc/31963/task/31992/exe (readlink: Permission denied) 2024-03-07T07:43:30.610 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31992 safe_time ceph NOFD /proc/31963/task/31992/fd (opendir: Permission denied) 2024-03-07T07:43:30.610 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31993 cpu_tp ceph cwd unknown /proc/31963/task/31993/cwd (readlink: Permission denied) 2024-03-07T07:43:30.610 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31993 cpu_tp ceph rtd unknown /proc/31963/task/31993/root (readlink: Permission denied) 2024-03-07T07:43:30.610 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31993 cpu_tp ceph txt unknown /proc/31963/task/31993/exe (readlink: Permission denied) 2024-03-07T07:43:30.610 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31993 cpu_tp ceph NOFD /proc/31963/task/31993/fd (opendir: Permission denied) 2024-03-07T07:43:30.610 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31994 cpu_tp ceph cwd unknown /proc/31963/task/31994/cwd (readlink: Permission denied) 2024-03-07T07:43:30.610 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31994 cpu_tp ceph rtd unknown /proc/31963/task/31994/root (readlink: Permission denied) 2024-03-07T07:43:30.610 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31994 cpu_tp ceph txt unknown /proc/31963/task/31994/exe (readlink: Permission denied) 2024-03-07T07:43:30.610 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31994 cpu_tp ceph NOFD /proc/31963/task/31994/fd (opendir: Permission denied) 2024-03-07T07:43:30.610 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31995 cpu_tp ceph cwd unknown /proc/31963/task/31995/cwd (readlink: Permission denied) 2024-03-07T07:43:30.610 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31995 cpu_tp ceph rtd unknown /proc/31963/task/31995/root (readlink: Permission denied) 2024-03-07T07:43:30.610 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31995 cpu_tp ceph txt unknown /proc/31963/task/31995/exe (readlink: Permission denied) 2024-03-07T07:43:30.610 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31995 cpu_tp ceph NOFD /proc/31963/task/31995/fd (opendir: Permission denied) 2024-03-07T07:43:30.610 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31996 cpu_tp ceph cwd unknown /proc/31963/task/31996/cwd (readlink: Permission denied) 2024-03-07T07:43:30.611 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31996 cpu_tp ceph rtd unknown /proc/31963/task/31996/root (readlink: Permission denied) 2024-03-07T07:43:30.611 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31996 cpu_tp ceph txt unknown /proc/31963/task/31996/exe (readlink: Permission denied) 2024-03-07T07:43:30.611 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31996 cpu_tp ceph NOFD /proc/31963/task/31996/fd (opendir: Permission denied) 2024-03-07T07:43:30.611 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31997 ms_dispat ceph cwd unknown /proc/31963/task/31997/cwd (readlink: Permission denied) 2024-03-07T07:43:30.611 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31997 ms_dispat ceph rtd unknown /proc/31963/task/31997/root (readlink: Permission denied) 2024-03-07T07:43:30.611 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31997 ms_dispat ceph txt unknown /proc/31963/task/31997/exe (readlink: Permission denied) 2024-03-07T07:43:30.611 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31997 ms_dispat ceph NOFD /proc/31963/task/31997/fd (opendir: Permission denied) 2024-03-07T07:43:30.611 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31998 ms_local ceph cwd unknown /proc/31963/task/31998/cwd (readlink: Permission denied) 2024-03-07T07:43:30.611 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31998 ms_local ceph rtd unknown /proc/31963/task/31998/root (readlink: Permission denied) 2024-03-07T07:43:30.611 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31998 ms_local ceph txt unknown /proc/31963/task/31998/exe (readlink: Permission denied) 2024-03-07T07:43:30.611 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31998 ms_local ceph NOFD /proc/31963/task/31998/fd (opendir: Permission denied) 2024-03-07T07:43:30.611 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31999 safe_time ceph cwd unknown /proc/31963/task/31999/cwd (readlink: Permission denied) 2024-03-07T07:43:30.611 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31999 safe_time ceph rtd unknown /proc/31963/task/31999/root (readlink: Permission denied) 2024-03-07T07:43:30.612 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31999 safe_time ceph txt unknown /proc/31963/task/31999/exe (readlink: Permission denied) 2024-03-07T07:43:30.612 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 31999 safe_time ceph NOFD /proc/31963/task/31999/fd (opendir: Permission denied) 2024-03-07T07:43:30.612 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 32000 ms_dispat ceph cwd unknown /proc/31963/task/32000/cwd (readlink: Permission denied) 2024-03-07T07:43:30.612 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 32000 ms_dispat ceph rtd unknown /proc/31963/task/32000/root (readlink: Permission denied) 2024-03-07T07:43:30.612 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 32000 ms_dispat ceph txt unknown /proc/31963/task/32000/exe (readlink: Permission denied) 2024-03-07T07:43:30.612 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 32000 ms_dispat ceph NOFD /proc/31963/task/32000/fd (opendir: Permission denied) 2024-03-07T07:43:30.612 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 32001 ms_local ceph cwd unknown /proc/31963/task/32001/cwd (readlink: Permission denied) 2024-03-07T07:43:30.612 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 32001 ms_local ceph rtd unknown /proc/31963/task/32001/root (readlink: Permission denied) 2024-03-07T07:43:30.612 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 32001 ms_local ceph txt unknown /proc/31963/task/32001/exe (readlink: Permission denied) 2024-03-07T07:43:30.612 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 32001 ms_local ceph NOFD /proc/31963/task/32001/fd (opendir: Permission denied) 2024-03-07T07:43:30.612 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 71234 rstore_co ceph cwd unknown /proc/31963/task/71234/cwd (readlink: Permission denied) 2024-03-07T07:43:30.612 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 71234 rstore_co ceph rtd unknown /proc/31963/task/71234/root (readlink: Permission denied) 2024-03-07T07:43:30.612 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 71234 rstore_co ceph txt unknown /proc/31963/task/71234/exe (readlink: Permission denied) 2024-03-07T07:43:30.613 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mon 31963 71234 rstore_co ceph NOFD /proc/31963/task/71234/fd (opendir: Permission denied) 2024-03-07T07:43:30.613 INFO:tasks.workunit.client.0.smithi160.stdout:conmon 32196 root cwd unknown /proc/32196/cwd (readlink: Permission denied) 2024-03-07T07:43:30.613 INFO:tasks.workunit.client.0.smithi160.stdout:conmon 32196 root rtd unknown /proc/32196/root (readlink: Permission denied) 2024-03-07T07:43:30.613 INFO:tasks.workunit.client.0.smithi160.stdout:conmon 32196 root txt unknown /proc/32196/exe (readlink: Permission denied) 2024-03-07T07:43:30.613 INFO:tasks.workunit.client.0.smithi160.stdout:conmon 32196 root NOFD /proc/32196/fd (opendir: Permission denied) 2024-03-07T07:43:30.613 INFO:tasks.workunit.client.0.smithi160.stdout:podman-in 32198 root cwd unknown /proc/32198/cwd (readlink: Permission denied) 2024-03-07T07:43:30.613 INFO:tasks.workunit.client.0.smithi160.stdout:podman-in 32198 root rtd unknown /proc/32198/root (readlink: Permission denied) 2024-03-07T07:43:30.613 INFO:tasks.workunit.client.0.smithi160.stdout:podman-in 32198 root txt unknown /proc/32198/exe (readlink: Permission denied) 2024-03-07T07:43:30.613 INFO:tasks.workunit.client.0.smithi160.stdout:podman-in 32198 root NOFD /proc/32198/fd (opendir: Permission denied) 2024-03-07T07:43:30.613 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 ceph cwd unknown /proc/32200/cwd (readlink: Permission denied) 2024-03-07T07:43:30.613 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 ceph rtd unknown /proc/32200/root (readlink: Permission denied) 2024-03-07T07:43:30.613 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 ceph txt unknown /proc/32200/exe (readlink: Permission denied) 2024-03-07T07:43:30.613 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 ceph NOFD /proc/32200/fd (opendir: Permission denied) 2024-03-07T07:43:30.613 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34748 log ceph cwd unknown /proc/32200/task/34748/cwd (readlink: Permission denied) 2024-03-07T07:43:30.614 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34748 log ceph rtd unknown /proc/32200/task/34748/root (readlink: Permission denied) 2024-03-07T07:43:30.614 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34748 log ceph txt unknown /proc/32200/task/34748/exe (readlink: Permission denied) 2024-03-07T07:43:30.614 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34748 log ceph NOFD /proc/32200/task/34748/fd (opendir: Permission denied) 2024-03-07T07:43:30.614 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34751 msgr-work ceph cwd unknown /proc/32200/task/34751/cwd (readlink: Permission denied) 2024-03-07T07:43:30.614 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34751 msgr-work ceph rtd unknown /proc/32200/task/34751/root (readlink: Permission denied) 2024-03-07T07:43:30.614 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34751 msgr-work ceph txt unknown /proc/32200/task/34751/exe (readlink: Permission denied) 2024-03-07T07:43:30.614 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34751 msgr-work ceph NOFD /proc/32200/task/34751/fd (opendir: Permission denied) 2024-03-07T07:43:30.614 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34752 msgr-work ceph cwd unknown /proc/32200/task/34752/cwd (readlink: Permission denied) 2024-03-07T07:43:30.614 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34752 msgr-work ceph rtd unknown /proc/32200/task/34752/root (readlink: Permission denied) 2024-03-07T07:43:30.614 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34752 msgr-work ceph txt unknown /proc/32200/task/34752/exe (readlink: Permission denied) 2024-03-07T07:43:30.614 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34752 msgr-work ceph NOFD /proc/32200/task/34752/fd (opendir: Permission denied) 2024-03-07T07:43:30.614 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34753 msgr-work ceph cwd unknown /proc/32200/task/34753/cwd (readlink: Permission denied) 2024-03-07T07:43:30.614 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34753 msgr-work ceph rtd unknown /proc/32200/task/34753/root (readlink: Permission denied) 2024-03-07T07:43:30.614 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34753 msgr-work ceph txt unknown /proc/32200/task/34753/exe (readlink: Permission denied) 2024-03-07T07:43:30.614 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34753 msgr-work ceph NOFD /proc/32200/task/34753/fd (opendir: Permission denied) 2024-03-07T07:43:30.615 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34757 service ceph cwd unknown /proc/32200/task/34757/cwd (readlink: Permission denied) 2024-03-07T07:43:30.615 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34757 service ceph rtd unknown /proc/32200/task/34757/root (readlink: Permission denied) 2024-03-07T07:43:30.615 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34757 service ceph txt unknown /proc/32200/task/34757/exe (readlink: Permission denied) 2024-03-07T07:43:30.615 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34757 service ceph NOFD /proc/32200/task/34757/fd (opendir: Permission denied) 2024-03-07T07:43:30.615 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34758 admin_soc ceph cwd unknown /proc/32200/task/34758/cwd (readlink: Permission denied) 2024-03-07T07:43:30.615 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34758 admin_soc ceph rtd unknown /proc/32200/task/34758/root (readlink: Permission denied) 2024-03-07T07:43:30.615 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34758 admin_soc ceph txt unknown /proc/32200/task/34758/exe (readlink: Permission denied) 2024-03-07T07:43:30.615 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34758 admin_soc ceph NOFD /proc/32200/task/34758/fd (opendir: Permission denied) 2024-03-07T07:43:30.615 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34759 ceph_time ceph cwd unknown /proc/32200/task/34759/cwd (readlink: Permission denied) 2024-03-07T07:43:30.615 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34759 ceph_time ceph rtd unknown /proc/32200/task/34759/root (readlink: Permission denied) 2024-03-07T07:43:30.615 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34759 ceph_time ceph txt unknown /proc/32200/task/34759/exe (readlink: Permission denied) 2024-03-07T07:43:30.615 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34759 ceph_time ceph NOFD /proc/32200/task/34759/fd (opendir: Permission denied) 2024-03-07T07:43:30.615 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34760 fn_anonym ceph cwd unknown /proc/32200/task/34760/cwd (readlink: Permission denied) 2024-03-07T07:43:30.616 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34760 fn_anonym ceph rtd unknown /proc/32200/task/34760/root (readlink: Permission denied) 2024-03-07T07:43:30.616 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34760 fn_anonym ceph txt unknown /proc/32200/task/34760/exe (readlink: Permission denied) 2024-03-07T07:43:30.616 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34760 fn_anonym ceph NOFD /proc/32200/task/34760/fd (opendir: Permission denied) 2024-03-07T07:43:30.616 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34761 signal_ha ceph cwd unknown /proc/32200/task/34761/cwd (readlink: Permission denied) 2024-03-07T07:43:30.616 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34761 signal_ha ceph rtd unknown /proc/32200/task/34761/root (readlink: Permission denied) 2024-03-07T07:43:30.616 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34761 signal_ha ceph txt unknown /proc/32200/task/34761/exe (readlink: Permission denied) 2024-03-07T07:43:30.616 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34761 signal_ha ceph NOFD /proc/32200/task/34761/fd (opendir: Permission denied) 2024-03-07T07:43:30.616 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34762 mgrsb-fin ceph cwd unknown /proc/32200/task/34762/cwd (readlink: Permission denied) 2024-03-07T07:43:30.616 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34762 mgrsb-fin ceph rtd unknown /proc/32200/task/34762/root (readlink: Permission denied) 2024-03-07T07:43:30.616 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34762 mgrsb-fin ceph txt unknown /proc/32200/task/34762/exe (readlink: Permission denied) 2024-03-07T07:43:30.616 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34762 mgrsb-fin ceph NOFD /proc/32200/task/34762/fd (opendir: Permission denied) 2024-03-07T07:43:30.616 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34763 ms_dispat ceph cwd unknown /proc/32200/task/34763/cwd (readlink: Permission denied) 2024-03-07T07:43:30.616 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34763 ms_dispat ceph rtd unknown /proc/32200/task/34763/root (readlink: Permission denied) 2024-03-07T07:43:30.616 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34763 ms_dispat ceph txt unknown /proc/32200/task/34763/exe (readlink: Permission denied) 2024-03-07T07:43:30.617 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34763 ms_dispat ceph NOFD /proc/32200/task/34763/fd (opendir: Permission denied) 2024-03-07T07:43:30.617 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34764 ms_local ceph cwd unknown /proc/32200/task/34764/cwd (readlink: Permission denied) 2024-03-07T07:43:30.617 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34764 ms_local ceph rtd unknown /proc/32200/task/34764/root (readlink: Permission denied) 2024-03-07T07:43:30.617 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34764 ms_local ceph txt unknown /proc/32200/task/34764/exe (readlink: Permission denied) 2024-03-07T07:43:30.617 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34764 ms_local ceph NOFD /proc/32200/task/34764/fd (opendir: Permission denied) 2024-03-07T07:43:30.617 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34765 io_contex ceph cwd unknown /proc/32200/task/34765/cwd (readlink: Permission denied) 2024-03-07T07:43:30.617 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34765 io_contex ceph rtd unknown /proc/32200/task/34765/root (readlink: Permission denied) 2024-03-07T07:43:30.617 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34765 io_contex ceph txt unknown /proc/32200/task/34765/exe (readlink: Permission denied) 2024-03-07T07:43:30.617 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34765 io_contex ceph NOFD /proc/32200/task/34765/fd (opendir: Permission denied) 2024-03-07T07:43:30.617 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34766 io_contex ceph cwd unknown /proc/32200/task/34766/cwd (readlink: Permission denied) 2024-03-07T07:43:30.617 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34766 io_contex ceph rtd unknown /proc/32200/task/34766/root (readlink: Permission denied) 2024-03-07T07:43:30.617 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34766 io_contex ceph txt unknown /proc/32200/task/34766/exe (readlink: Permission denied) 2024-03-07T07:43:30.617 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34766 io_contex ceph NOFD /proc/32200/task/34766/fd (opendir: Permission denied) 2024-03-07T07:43:30.617 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34767 safe_time ceph cwd unknown /proc/32200/task/34767/cwd (readlink: Permission denied) 2024-03-07T07:43:30.618 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34767 safe_time ceph rtd unknown /proc/32200/task/34767/root (readlink: Permission denied) 2024-03-07T07:43:30.618 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34767 safe_time ceph txt unknown /proc/32200/task/34767/exe (readlink: Permission denied) 2024-03-07T07:43:30.618 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34767 safe_time ceph NOFD /proc/32200/task/34767/fd (opendir: Permission denied) 2024-03-07T07:43:30.618 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34768 safe_time ceph cwd unknown /proc/32200/task/34768/cwd (readlink: Permission denied) 2024-03-07T07:43:30.618 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34768 safe_time ceph rtd unknown /proc/32200/task/34768/root (readlink: Permission denied) 2024-03-07T07:43:30.618 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34768 safe_time ceph txt unknown /proc/32200/task/34768/exe (readlink: Permission denied) 2024-03-07T07:43:30.618 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34768 safe_time ceph NOFD /proc/32200/task/34768/fd (opendir: Permission denied) 2024-03-07T07:43:30.618 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34769 safe_time ceph cwd unknown /proc/32200/task/34769/cwd (readlink: Permission denied) 2024-03-07T07:43:30.618 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34769 safe_time ceph rtd unknown /proc/32200/task/34769/root (readlink: Permission denied) 2024-03-07T07:43:30.618 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34769 safe_time ceph txt unknown /proc/32200/task/34769/exe (readlink: Permission denied) 2024-03-07T07:43:30.618 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34769 safe_time ceph NOFD /proc/32200/task/34769/fd (opendir: Permission denied) 2024-03-07T07:43:30.618 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34770 fn_anonym ceph cwd unknown /proc/32200/task/34770/cwd (readlink: Permission denied) 2024-03-07T07:43:30.618 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34770 fn_anonym ceph rtd unknown /proc/32200/task/34770/root (readlink: Permission denied) 2024-03-07T07:43:30.618 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34770 fn_anonym ceph txt unknown /proc/32200/task/34770/exe (readlink: Permission denied) 2024-03-07T07:43:30.619 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34770 fn_anonym ceph NOFD /proc/32200/task/34770/fd (opendir: Permission denied) 2024-03-07T07:43:30.619 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34771 flusher ceph cwd unknown /proc/32200/task/34771/cwd (readlink: Permission denied) 2024-03-07T07:43:30.619 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34771 flusher ceph rtd unknown /proc/32200/task/34771/root (readlink: Permission denied) 2024-03-07T07:43:30.619 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34771 flusher ceph txt unknown /proc/32200/task/34771/exe (readlink: Permission denied) 2024-03-07T07:43:30.619 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34771 flusher ceph NOFD /proc/32200/task/34771/fd (opendir: Permission denied) 2024-03-07T07:43:30.619 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34772 safe_time ceph cwd unknown /proc/32200/task/34772/cwd (readlink: Permission denied) 2024-03-07T07:43:30.619 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34772 safe_time ceph rtd unknown /proc/32200/task/34772/root (readlink: Permission denied) 2024-03-07T07:43:30.619 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34772 safe_time ceph txt unknown /proc/32200/task/34772/exe (readlink: Permission denied) 2024-03-07T07:43:30.619 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34772 safe_time ceph NOFD /proc/32200/task/34772/fd (opendir: Permission denied) 2024-03-07T07:43:30.619 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34938 mgr-fin ceph cwd unknown /proc/32200/task/34938/cwd (readlink: Permission denied) 2024-03-07T07:43:30.619 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34938 mgr-fin ceph rtd unknown /proc/32200/task/34938/root (readlink: Permission denied) 2024-03-07T07:43:30.619 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34938 mgr-fin ceph txt unknown /proc/32200/task/34938/exe (readlink: Permission denied) 2024-03-07T07:43:30.619 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34938 mgr-fin ceph NOFD /proc/32200/task/34938/fd (opendir: Permission denied) 2024-03-07T07:43:30.619 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34939 ms_dispat ceph cwd unknown /proc/32200/task/34939/cwd (readlink: Permission denied) 2024-03-07T07:43:30.620 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34939 ms_dispat ceph rtd unknown /proc/32200/task/34939/root (readlink: Permission denied) 2024-03-07T07:43:30.620 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34939 ms_dispat ceph txt unknown /proc/32200/task/34939/exe (readlink: Permission denied) 2024-03-07T07:43:30.620 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34939 ms_dispat ceph NOFD /proc/32200/task/34939/fd (opendir: Permission denied) 2024-03-07T07:43:30.620 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34940 ms_local ceph cwd unknown /proc/32200/task/34940/cwd (readlink: Permission denied) 2024-03-07T07:43:30.620 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34940 ms_local ceph rtd unknown /proc/32200/task/34940/root (readlink: Permission denied) 2024-03-07T07:43:30.620 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34940 ms_local ceph txt unknown /proc/32200/task/34940/exe (readlink: Permission denied) 2024-03-07T07:43:30.620 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34940 ms_local ceph NOFD /proc/32200/task/34940/fd (opendir: Permission denied) 2024-03-07T07:43:30.620 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34941 safe_time ceph cwd unknown /proc/32200/task/34941/cwd (readlink: Permission denied) 2024-03-07T07:43:30.620 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34941 safe_time ceph rtd unknown /proc/32200/task/34941/root (readlink: Permission denied) 2024-03-07T07:43:30.620 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34941 safe_time ceph txt unknown /proc/32200/task/34941/exe (readlink: Permission denied) 2024-03-07T07:43:30.620 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34941 safe_time ceph NOFD /proc/32200/task/34941/fd (opendir: Permission denied) 2024-03-07T07:43:30.620 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34942 cmdfin ceph cwd unknown /proc/32200/task/34942/cwd (readlink: Permission denied) 2024-03-07T07:43:30.620 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34942 cmdfin ceph rtd unknown /proc/32200/task/34942/root (readlink: Permission denied) 2024-03-07T07:43:30.620 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34942 cmdfin ceph txt unknown /proc/32200/task/34942/exe (readlink: Permission denied) 2024-03-07T07:43:30.621 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34942 cmdfin ceph NOFD /proc/32200/task/34942/fd (opendir: Permission denied) 2024-03-07T07:43:30.621 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34946 io_contex ceph cwd unknown /proc/32200/task/34946/cwd (readlink: Permission denied) 2024-03-07T07:43:30.621 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34946 io_contex ceph rtd unknown /proc/32200/task/34946/root (readlink: Permission denied) 2024-03-07T07:43:30.621 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34946 io_contex ceph txt unknown /proc/32200/task/34946/exe (readlink: Permission denied) 2024-03-07T07:43:30.621 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34946 io_contex ceph NOFD /proc/32200/task/34946/fd (opendir: Permission denied) 2024-03-07T07:43:30.621 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34947 io_contex ceph cwd unknown /proc/32200/task/34947/cwd (readlink: Permission denied) 2024-03-07T07:43:30.621 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34947 io_contex ceph rtd unknown /proc/32200/task/34947/root (readlink: Permission denied) 2024-03-07T07:43:30.621 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34947 io_contex ceph txt unknown /proc/32200/task/34947/exe (readlink: Permission denied) 2024-03-07T07:43:30.621 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34947 io_contex ceph NOFD /proc/32200/task/34947/fd (opendir: Permission denied) 2024-03-07T07:43:30.621 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34948 ceph_time ceph cwd unknown /proc/32200/task/34948/cwd (readlink: Permission denied) 2024-03-07T07:43:30.621 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34948 ceph_time ceph rtd unknown /proc/32200/task/34948/root (readlink: Permission denied) 2024-03-07T07:43:30.621 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34948 ceph_time ceph txt unknown /proc/32200/task/34948/exe (readlink: Permission denied) 2024-03-07T07:43:30.621 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34948 ceph_time ceph NOFD /proc/32200/task/34948/fd (opendir: Permission denied) 2024-03-07T07:43:30.621 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34949 ms_dispat ceph cwd unknown /proc/32200/task/34949/cwd (readlink: Permission denied) 2024-03-07T07:43:30.622 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34949 ms_dispat ceph rtd unknown /proc/32200/task/34949/root (readlink: Permission denied) 2024-03-07T07:43:30.622 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34949 ms_dispat ceph txt unknown /proc/32200/task/34949/exe (readlink: Permission denied) 2024-03-07T07:43:30.622 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34949 ms_dispat ceph NOFD /proc/32200/task/34949/fd (opendir: Permission denied) 2024-03-07T07:43:30.622 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34950 ms_local ceph cwd unknown /proc/32200/task/34950/cwd (readlink: Permission denied) 2024-03-07T07:43:30.622 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34950 ms_local ceph rtd unknown /proc/32200/task/34950/root (readlink: Permission denied) 2024-03-07T07:43:30.622 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34950 ms_local ceph txt unknown /proc/32200/task/34950/exe (readlink: Permission denied) 2024-03-07T07:43:30.622 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34950 ms_local ceph NOFD /proc/32200/task/34950/fd (opendir: Permission denied) 2024-03-07T07:43:30.622 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34951 safe_time ceph cwd unknown /proc/32200/task/34951/cwd (readlink: Permission denied) 2024-03-07T07:43:30.622 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34951 safe_time ceph rtd unknown /proc/32200/task/34951/root (readlink: Permission denied) 2024-03-07T07:43:30.622 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34951 safe_time ceph txt unknown /proc/32200/task/34951/exe (readlink: Permission denied) 2024-03-07T07:43:30.622 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34951 safe_time ceph NOFD /proc/32200/task/34951/fd (opendir: Permission denied) 2024-03-07T07:43:30.622 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34952 safe_time ceph cwd unknown /proc/32200/task/34952/cwd (readlink: Permission denied) 2024-03-07T07:43:30.622 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34952 safe_time ceph rtd unknown /proc/32200/task/34952/root (readlink: Permission denied) 2024-03-07T07:43:30.622 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34952 safe_time ceph txt unknown /proc/32200/task/34952/exe (readlink: Permission denied) 2024-03-07T07:43:30.623 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34952 safe_time ceph NOFD /proc/32200/task/34952/fd (opendir: Permission denied) 2024-03-07T07:43:30.623 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34953 balancer ceph cwd unknown /proc/32200/task/34953/cwd (readlink: Permission denied) 2024-03-07T07:43:30.623 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34953 balancer ceph rtd unknown /proc/32200/task/34953/root (readlink: Permission denied) 2024-03-07T07:43:30.623 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34953 balancer ceph txt unknown /proc/32200/task/34953/exe (readlink: Permission denied) 2024-03-07T07:43:30.623 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34953 balancer ceph NOFD /proc/32200/task/34953/fd (opendir: Permission denied) 2024-03-07T07:43:30.623 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34954 m-fin-bal ceph cwd unknown /proc/32200/task/34954/cwd (readlink: Permission denied) 2024-03-07T07:43:30.623 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34954 m-fin-bal ceph rtd unknown /proc/32200/task/34954/root (readlink: Permission denied) 2024-03-07T07:43:30.623 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34954 m-fin-bal ceph txt unknown /proc/32200/task/34954/exe (readlink: Permission denied) 2024-03-07T07:43:30.623 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34954 m-fin-bal ceph NOFD /proc/32200/task/34954/fd (opendir: Permission denied) 2024-03-07T07:43:30.623 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34955 mgr-fin ceph cwd unknown /proc/32200/task/34955/cwd (readlink: Permission denied) 2024-03-07T07:43:30.623 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34955 mgr-fin ceph rtd unknown /proc/32200/task/34955/root (readlink: Permission denied) 2024-03-07T07:43:30.623 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34955 mgr-fin ceph txt unknown /proc/32200/task/34955/exe (readlink: Permission denied) 2024-03-07T07:43:30.623 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34955 mgr-fin ceph NOFD /proc/32200/task/34955/fd (opendir: Permission denied) 2024-03-07T07:43:30.623 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34956 mgr-fin ceph cwd unknown /proc/32200/task/34956/cwd (readlink: Permission denied) 2024-03-07T07:43:30.624 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34956 mgr-fin ceph rtd unknown /proc/32200/task/34956/root (readlink: Permission denied) 2024-03-07T07:43:30.624 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34956 mgr-fin ceph txt unknown /proc/32200/task/34956/exe (readlink: Permission denied) 2024-03-07T07:43:30.624 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34956 mgr-fin ceph NOFD /proc/32200/task/34956/fd (opendir: Permission denied) 2024-03-07T07:43:30.624 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34957 mgr-fin ceph cwd unknown /proc/32200/task/34957/cwd (readlink: Permission denied) 2024-03-07T07:43:30.624 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34957 mgr-fin ceph rtd unknown /proc/32200/task/34957/root (readlink: Permission denied) 2024-03-07T07:43:30.624 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34957 mgr-fin ceph txt unknown /proc/32200/task/34957/exe (readlink: Permission denied) 2024-03-07T07:43:30.624 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34957 mgr-fin ceph NOFD /proc/32200/task/34957/fd (opendir: Permission denied) 2024-03-07T07:43:30.624 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34958 mgr-fin ceph cwd unknown /proc/32200/task/34958/cwd (readlink: Permission denied) 2024-03-07T07:43:30.624 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34958 mgr-fin ceph rtd unknown /proc/32200/task/34958/root (readlink: Permission denied) 2024-03-07T07:43:30.624 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34958 mgr-fin ceph txt unknown /proc/32200/task/34958/exe (readlink: Permission denied) 2024-03-07T07:43:30.624 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34958 mgr-fin ceph NOFD /proc/32200/task/34958/fd (opendir: Permission denied) 2024-03-07T07:43:30.624 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34959 mgr-fin ceph cwd unknown /proc/32200/task/34959/cwd (readlink: Permission denied) 2024-03-07T07:43:30.624 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34959 mgr-fin ceph rtd unknown /proc/32200/task/34959/root (readlink: Permission denied) 2024-03-07T07:43:30.625 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34959 mgr-fin ceph txt unknown /proc/32200/task/34959/exe (readlink: Permission denied) 2024-03-07T07:43:30.625 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34959 mgr-fin ceph NOFD /proc/32200/task/34959/fd (opendir: Permission denied) 2024-03-07T07:43:30.625 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34960 mgr-fin ceph cwd unknown /proc/32200/task/34960/cwd (readlink: Permission denied) 2024-03-07T07:43:30.625 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34960 mgr-fin ceph rtd unknown /proc/32200/task/34960/root (readlink: Permission denied) 2024-03-07T07:43:30.625 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34960 mgr-fin ceph txt unknown /proc/32200/task/34960/exe (readlink: Permission denied) 2024-03-07T07:43:30.625 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34960 mgr-fin ceph NOFD /proc/32200/task/34960/fd (opendir: Permission denied) 2024-03-07T07:43:30.625 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34961 mgr-fin ceph cwd unknown /proc/32200/task/34961/cwd (readlink: Permission denied) 2024-03-07T07:43:30.625 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34961 mgr-fin ceph rtd unknown /proc/32200/task/34961/root (readlink: Permission denied) 2024-03-07T07:43:30.625 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34961 mgr-fin ceph txt unknown /proc/32200/task/34961/exe (readlink: Permission denied) 2024-03-07T07:43:30.625 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34961 mgr-fin ceph NOFD /proc/32200/task/34961/fd (opendir: Permission denied) 2024-03-07T07:43:30.625 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34962 mgr-fin ceph cwd unknown /proc/32200/task/34962/cwd (readlink: Permission denied) 2024-03-07T07:43:30.625 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34962 mgr-fin ceph rtd unknown /proc/32200/task/34962/root (readlink: Permission denied) 2024-03-07T07:43:30.625 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34962 mgr-fin ceph txt unknown /proc/32200/task/34962/exe (readlink: Permission denied) 2024-03-07T07:43:30.625 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34962 mgr-fin ceph NOFD /proc/32200/task/34962/fd (opendir: Permission denied) 2024-03-07T07:43:30.626 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34963 mgr-fin ceph cwd unknown /proc/32200/task/34963/cwd (readlink: Permission denied) 2024-03-07T07:43:30.626 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34963 mgr-fin ceph rtd unknown /proc/32200/task/34963/root (readlink: Permission denied) 2024-03-07T07:43:30.626 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34963 mgr-fin ceph txt unknown /proc/32200/task/34963/exe (readlink: Permission denied) 2024-03-07T07:43:30.626 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34963 mgr-fin ceph NOFD /proc/32200/task/34963/fd (opendir: Permission denied) 2024-03-07T07:43:30.626 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34964 mgr-fin ceph cwd unknown /proc/32200/task/34964/cwd (readlink: Permission denied) 2024-03-07T07:43:30.626 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34964 mgr-fin ceph rtd unknown /proc/32200/task/34964/root (readlink: Permission denied) 2024-03-07T07:43:30.626 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34964 mgr-fin ceph txt unknown /proc/32200/task/34964/exe (readlink: Permission denied) 2024-03-07T07:43:30.626 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34964 mgr-fin ceph NOFD /proc/32200/task/34964/fd (opendir: Permission denied) 2024-03-07T07:43:30.626 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34965 mgr-fin ceph cwd unknown /proc/32200/task/34965/cwd (readlink: Permission denied) 2024-03-07T07:43:30.626 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34965 mgr-fin ceph rtd unknown /proc/32200/task/34965/root (readlink: Permission denied) 2024-03-07T07:43:30.626 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34965 mgr-fin ceph txt unknown /proc/32200/task/34965/exe (readlink: Permission denied) 2024-03-07T07:43:30.626 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34965 mgr-fin ceph NOFD /proc/32200/task/34965/fd (opendir: Permission denied) 2024-03-07T07:43:30.626 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34966 mgr-fin ceph cwd unknown /proc/32200/task/34966/cwd (readlink: Permission denied) 2024-03-07T07:43:30.626 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34966 mgr-fin ceph rtd unknown /proc/32200/task/34966/root (readlink: Permission denied) 2024-03-07T07:43:30.627 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34966 mgr-fin ceph txt unknown /proc/32200/task/34966/exe (readlink: Permission denied) 2024-03-07T07:43:30.627 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34966 mgr-fin ceph NOFD /proc/32200/task/34966/fd (opendir: Permission denied) 2024-03-07T07:43:30.627 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34967 mgr-fin ceph cwd unknown /proc/32200/task/34967/cwd (readlink: Permission denied) 2024-03-07T07:43:30.627 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34967 mgr-fin ceph rtd unknown /proc/32200/task/34967/root (readlink: Permission denied) 2024-03-07T07:43:30.627 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34967 mgr-fin ceph txt unknown /proc/32200/task/34967/exe (readlink: Permission denied) 2024-03-07T07:43:30.627 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34967 mgr-fin ceph NOFD /proc/32200/task/34967/fd (opendir: Permission denied) 2024-03-07T07:43:30.627 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34968 mgr-fin ceph cwd unknown /proc/32200/task/34968/cwd (readlink: Permission denied) 2024-03-07T07:43:30.627 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34968 mgr-fin ceph rtd unknown /proc/32200/task/34968/root (readlink: Permission denied) 2024-03-07T07:43:30.627 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34968 mgr-fin ceph txt unknown /proc/32200/task/34968/exe (readlink: Permission denied) 2024-03-07T07:43:30.627 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34968 mgr-fin ceph NOFD /proc/32200/task/34968/fd (opendir: Permission denied) 2024-03-07T07:43:30.627 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34969 mgr-fin ceph cwd unknown /proc/32200/task/34969/cwd (readlink: Permission denied) 2024-03-07T07:43:30.627 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34969 mgr-fin ceph rtd unknown /proc/32200/task/34969/root (readlink: Permission denied) 2024-03-07T07:43:30.627 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34969 mgr-fin ceph txt unknown /proc/32200/task/34969/exe (readlink: Permission denied) 2024-03-07T07:43:30.627 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34969 mgr-fin ceph NOFD /proc/32200/task/34969/fd (opendir: Permission denied) 2024-03-07T07:43:30.628 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34970 mgr-fin ceph cwd unknown /proc/32200/task/34970/cwd (readlink: Permission denied) 2024-03-07T07:43:30.628 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34970 mgr-fin ceph rtd unknown /proc/32200/task/34970/root (readlink: Permission denied) 2024-03-07T07:43:30.628 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34970 mgr-fin ceph txt unknown /proc/32200/task/34970/exe (readlink: Permission denied) 2024-03-07T07:43:30.628 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34970 mgr-fin ceph NOFD /proc/32200/task/34970/fd (opendir: Permission denied) 2024-03-07T07:43:30.628 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34971 cephadm ceph cwd unknown /proc/32200/task/34971/cwd (readlink: Permission denied) 2024-03-07T07:43:30.628 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34971 cephadm ceph rtd unknown /proc/32200/task/34971/root (readlink: Permission denied) 2024-03-07T07:43:30.628 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34971 cephadm ceph txt unknown /proc/32200/task/34971/exe (readlink: Permission denied) 2024-03-07T07:43:30.628 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34971 cephadm ceph NOFD /proc/32200/task/34971/fd (opendir: Permission denied) 2024-03-07T07:43:30.628 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34972 m-fin-cep ceph cwd unknown /proc/32200/task/34972/cwd (readlink: Permission denied) 2024-03-07T07:43:30.628 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34972 m-fin-cep ceph rtd unknown /proc/32200/task/34972/root (readlink: Permission denied) 2024-03-07T07:43:30.628 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34972 m-fin-cep ceph txt unknown /proc/32200/task/34972/exe (readlink: Permission denied) 2024-03-07T07:43:30.628 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34972 m-fin-cep ceph NOFD /proc/32200/task/34972/fd (opendir: Permission denied) 2024-03-07T07:43:30.628 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34973 mgr-fin ceph cwd unknown /proc/32200/task/34973/cwd (readlink: Permission denied) 2024-03-07T07:43:30.629 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34973 mgr-fin ceph rtd unknown /proc/32200/task/34973/root (readlink: Permission denied) 2024-03-07T07:43:30.629 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34973 mgr-fin ceph txt unknown /proc/32200/task/34973/exe (readlink: Permission denied) 2024-03-07T07:43:30.629 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34973 mgr-fin ceph NOFD /proc/32200/task/34973/fd (opendir: Permission denied) 2024-03-07T07:43:30.629 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34974 mgr-fin ceph cwd unknown /proc/32200/task/34974/cwd (readlink: Permission denied) 2024-03-07T07:43:30.629 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34974 mgr-fin ceph rtd unknown /proc/32200/task/34974/root (readlink: Permission denied) 2024-03-07T07:43:30.629 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34974 mgr-fin ceph txt unknown /proc/32200/task/34974/exe (readlink: Permission denied) 2024-03-07T07:43:30.629 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34974 mgr-fin ceph NOFD /proc/32200/task/34974/fd (opendir: Permission denied) 2024-03-07T07:43:30.629 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34975 cephadm ceph cwd unknown /proc/32200/task/34975/cwd (readlink: Permission denied) 2024-03-07T07:43:30.629 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34975 cephadm ceph rtd unknown /proc/32200/task/34975/root (readlink: Permission denied) 2024-03-07T07:43:30.629 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34975 cephadm ceph txt unknown /proc/32200/task/34975/exe (readlink: Permission denied) 2024-03-07T07:43:30.629 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34975 cephadm ceph NOFD /proc/32200/task/34975/fd (opendir: Permission denied) 2024-03-07T07:43:30.629 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34976 mgr-fin ceph cwd unknown /proc/32200/task/34976/cwd (readlink: Permission denied) 2024-03-07T07:43:30.629 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34976 mgr-fin ceph rtd unknown /proc/32200/task/34976/root (readlink: Permission denied) 2024-03-07T07:43:30.629 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34976 mgr-fin ceph txt unknown /proc/32200/task/34976/exe (readlink: Permission denied) 2024-03-07T07:43:30.630 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34976 mgr-fin ceph NOFD /proc/32200/task/34976/fd (opendir: Permission denied) 2024-03-07T07:43:30.630 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34977 crash ceph cwd unknown /proc/32200/task/34977/cwd (readlink: Permission denied) 2024-03-07T07:43:30.630 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34977 crash ceph rtd unknown /proc/32200/task/34977/root (readlink: Permission denied) 2024-03-07T07:43:30.630 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34977 crash ceph txt unknown /proc/32200/task/34977/exe (readlink: Permission denied) 2024-03-07T07:43:30.630 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34977 crash ceph NOFD /proc/32200/task/34977/fd (opendir: Permission denied) 2024-03-07T07:43:30.630 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34978 m-fin-cra ceph cwd unknown /proc/32200/task/34978/cwd (readlink: Permission denied) 2024-03-07T07:43:30.630 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34978 m-fin-cra ceph rtd unknown /proc/32200/task/34978/root (readlink: Permission denied) 2024-03-07T07:43:30.630 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34978 m-fin-cra ceph txt unknown /proc/32200/task/34978/exe (readlink: Permission denied) 2024-03-07T07:43:30.630 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34978 m-fin-cra ceph NOFD /proc/32200/task/34978/fd (opendir: Permission denied) 2024-03-07T07:43:30.630 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34979 mgr-fin ceph cwd unknown /proc/32200/task/34979/cwd (readlink: Permission denied) 2024-03-07T07:43:30.630 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34979 mgr-fin ceph rtd unknown /proc/32200/task/34979/root (readlink: Permission denied) 2024-03-07T07:43:30.630 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34979 mgr-fin ceph txt unknown /proc/32200/task/34979/exe (readlink: Permission denied) 2024-03-07T07:43:30.630 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34979 mgr-fin ceph NOFD /proc/32200/task/34979/fd (opendir: Permission denied) 2024-03-07T07:43:30.630 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34980 mgr-fin ceph cwd unknown /proc/32200/task/34980/cwd (readlink: Permission denied) 2024-03-07T07:43:30.631 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34980 mgr-fin ceph rtd unknown /proc/32200/task/34980/root (readlink: Permission denied) 2024-03-07T07:43:30.631 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34980 mgr-fin ceph txt unknown /proc/32200/task/34980/exe (readlink: Permission denied) 2024-03-07T07:43:30.631 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34980 mgr-fin ceph NOFD /proc/32200/task/34980/fd (opendir: Permission denied) 2024-03-07T07:43:30.631 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34981 dashboard ceph cwd unknown /proc/32200/task/34981/cwd (readlink: Permission denied) 2024-03-07T07:43:30.631 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34981 dashboard ceph rtd unknown /proc/32200/task/34981/root (readlink: Permission denied) 2024-03-07T07:43:30.631 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34981 dashboard ceph txt unknown /proc/32200/task/34981/exe (readlink: Permission denied) 2024-03-07T07:43:30.631 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34981 dashboard ceph NOFD /proc/32200/task/34981/fd (opendir: Permission denied) 2024-03-07T07:43:30.631 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34982 m-fin-das ceph cwd unknown /proc/32200/task/34982/cwd (readlink: Permission denied) 2024-03-07T07:43:30.631 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34982 m-fin-das ceph rtd unknown /proc/32200/task/34982/root (readlink: Permission denied) 2024-03-07T07:43:30.631 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34982 m-fin-das ceph txt unknown /proc/32200/task/34982/exe (readlink: Permission denied) 2024-03-07T07:43:30.631 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34982 m-fin-das ceph NOFD /proc/32200/task/34982/fd (opendir: Permission denied) 2024-03-07T07:43:30.631 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34983 mgr-fin ceph cwd unknown /proc/32200/task/34983/cwd (readlink: Permission denied) 2024-03-07T07:43:30.631 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34983 mgr-fin ceph rtd unknown /proc/32200/task/34983/root (readlink: Permission denied) 2024-03-07T07:43:30.631 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34983 mgr-fin ceph txt unknown /proc/32200/task/34983/exe (readlink: Permission denied) 2024-03-07T07:43:30.632 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34983 mgr-fin ceph NOFD /proc/32200/task/34983/fd (opendir: Permission denied) 2024-03-07T07:43:30.632 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34984 devicehea ceph cwd unknown /proc/32200/task/34984/cwd (readlink: Permission denied) 2024-03-07T07:43:30.632 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34984 devicehea ceph rtd unknown /proc/32200/task/34984/root (readlink: Permission denied) 2024-03-07T07:43:30.632 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34984 devicehea ceph txt unknown /proc/32200/task/34984/exe (readlink: Permission denied) 2024-03-07T07:43:30.632 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34984 devicehea ceph NOFD /proc/32200/task/34984/fd (opendir: Permission denied) 2024-03-07T07:43:30.632 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34985 m-fin-dev ceph cwd unknown /proc/32200/task/34985/cwd (readlink: Permission denied) 2024-03-07T07:43:30.632 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34985 m-fin-dev ceph rtd unknown /proc/32200/task/34985/root (readlink: Permission denied) 2024-03-07T07:43:30.632 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34985 m-fin-dev ceph txt unknown /proc/32200/task/34985/exe (readlink: Permission denied) 2024-03-07T07:43:30.632 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34985 m-fin-dev ceph NOFD /proc/32200/task/34985/fd (opendir: Permission denied) 2024-03-07T07:43:30.632 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34986 mgr-fin ceph cwd unknown /proc/32200/task/34986/cwd (readlink: Permission denied) 2024-03-07T07:43:30.632 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34986 mgr-fin ceph rtd unknown /proc/32200/task/34986/root (readlink: Permission denied) 2024-03-07T07:43:30.632 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34986 mgr-fin ceph txt unknown /proc/32200/task/34986/exe (readlink: Permission denied) 2024-03-07T07:43:30.632 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34986 mgr-fin ceph NOFD /proc/32200/task/34986/fd (opendir: Permission denied) 2024-03-07T07:43:30.632 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34988 m-fin-ios ceph cwd unknown /proc/32200/task/34988/cwd (readlink: Permission denied) 2024-03-07T07:43:30.633 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34988 m-fin-ios ceph rtd unknown /proc/32200/task/34988/root (readlink: Permission denied) 2024-03-07T07:43:30.633 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34988 m-fin-ios ceph txt unknown /proc/32200/task/34988/exe (readlink: Permission denied) 2024-03-07T07:43:30.633 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34988 m-fin-ios ceph NOFD /proc/32200/task/34988/fd (opendir: Permission denied) 2024-03-07T07:43:30.633 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34989 mgr-fin ceph cwd unknown /proc/32200/task/34989/cwd (readlink: Permission denied) 2024-03-07T07:43:30.633 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34989 mgr-fin ceph rtd unknown /proc/32200/task/34989/root (readlink: Permission denied) 2024-03-07T07:43:30.633 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34989 mgr-fin ceph txt unknown /proc/32200/task/34989/exe (readlink: Permission denied) 2024-03-07T07:43:30.633 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34989 mgr-fin ceph NOFD /proc/32200/task/34989/fd (opendir: Permission denied) 2024-03-07T07:43:30.633 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34990 mgr-fin ceph cwd unknown /proc/32200/task/34990/cwd (readlink: Permission denied) 2024-03-07T07:43:30.633 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34990 mgr-fin ceph rtd unknown /proc/32200/task/34990/root (readlink: Permission denied) 2024-03-07T07:43:30.633 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34990 mgr-fin ceph txt unknown /proc/32200/task/34990/exe (readlink: Permission denied) 2024-03-07T07:43:30.633 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34990 mgr-fin ceph NOFD /proc/32200/task/34990/fd (opendir: Permission denied) 2024-03-07T07:43:30.633 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34991 mgr-fin ceph cwd unknown /proc/32200/task/34991/cwd (readlink: Permission denied) 2024-03-07T07:43:30.633 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34991 mgr-fin ceph rtd unknown /proc/32200/task/34991/root (readlink: Permission denied) 2024-03-07T07:43:30.633 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34991 mgr-fin ceph txt unknown /proc/32200/task/34991/exe (readlink: Permission denied) 2024-03-07T07:43:30.634 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34991 mgr-fin ceph NOFD /proc/32200/task/34991/fd (opendir: Permission denied) 2024-03-07T07:43:30.634 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34993 m-fin-nfs ceph cwd unknown /proc/32200/task/34993/cwd (readlink: Permission denied) 2024-03-07T07:43:30.634 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34993 m-fin-nfs ceph rtd unknown /proc/32200/task/34993/root (readlink: Permission denied) 2024-03-07T07:43:30.634 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34993 m-fin-nfs ceph txt unknown /proc/32200/task/34993/exe (readlink: Permission denied) 2024-03-07T07:43:30.634 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34993 m-fin-nfs ceph NOFD /proc/32200/task/34993/fd (opendir: Permission denied) 2024-03-07T07:43:30.634 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34994 mgr-fin ceph cwd unknown /proc/32200/task/34994/cwd (readlink: Permission denied) 2024-03-07T07:43:30.634 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34994 mgr-fin ceph rtd unknown /proc/32200/task/34994/root (readlink: Permission denied) 2024-03-07T07:43:30.634 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34994 mgr-fin ceph txt unknown /proc/32200/task/34994/exe (readlink: Permission denied) 2024-03-07T07:43:30.634 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34994 mgr-fin ceph NOFD /proc/32200/task/34994/fd (opendir: Permission denied) 2024-03-07T07:43:30.634 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34995 mgr-fin ceph cwd unknown /proc/32200/task/34995/cwd (readlink: Permission denied) 2024-03-07T07:43:30.634 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34995 mgr-fin ceph rtd unknown /proc/32200/task/34995/root (readlink: Permission denied) 2024-03-07T07:43:30.634 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34995 mgr-fin ceph txt unknown /proc/32200/task/34995/exe (readlink: Permission denied) 2024-03-07T07:43:30.634 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34995 mgr-fin ceph NOFD /proc/32200/task/34995/fd (opendir: Permission denied) 2024-03-07T07:43:30.634 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34997 m-fin-orc ceph cwd unknown /proc/32200/task/34997/cwd (readlink: Permission denied) 2024-03-07T07:43:30.635 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34997 m-fin-orc ceph rtd unknown /proc/32200/task/34997/root (readlink: Permission denied) 2024-03-07T07:43:30.635 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34997 m-fin-orc ceph txt unknown /proc/32200/task/34997/exe (readlink: Permission denied) 2024-03-07T07:43:30.635 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34997 m-fin-orc ceph NOFD /proc/32200/task/34997/fd (opendir: Permission denied) 2024-03-07T07:43:30.635 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34998 pg_autosc ceph cwd unknown /proc/32200/task/34998/cwd (readlink: Permission denied) 2024-03-07T07:43:30.635 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34998 pg_autosc ceph rtd unknown /proc/32200/task/34998/root (readlink: Permission denied) 2024-03-07T07:43:30.635 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34998 pg_autosc ceph txt unknown /proc/32200/task/34998/exe (readlink: Permission denied) 2024-03-07T07:43:30.635 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34998 pg_autosc ceph NOFD /proc/32200/task/34998/fd (opendir: Permission denied) 2024-03-07T07:43:30.635 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34999 m-fin-pg_ ceph cwd unknown /proc/32200/task/34999/cwd (readlink: Permission denied) 2024-03-07T07:43:30.635 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34999 m-fin-pg_ ceph rtd unknown /proc/32200/task/34999/root (readlink: Permission denied) 2024-03-07T07:43:30.635 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34999 m-fin-pg_ ceph txt unknown /proc/32200/task/34999/exe (readlink: Permission denied) 2024-03-07T07:43:30.635 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 34999 m-fin-pg_ ceph NOFD /proc/32200/task/34999/fd (opendir: Permission denied) 2024-03-07T07:43:30.635 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35000 progress ceph cwd unknown /proc/32200/task/35000/cwd (readlink: Permission denied) 2024-03-07T07:43:30.635 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35000 progress ceph rtd unknown /proc/32200/task/35000/root (readlink: Permission denied) 2024-03-07T07:43:30.635 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35000 progress ceph txt unknown /proc/32200/task/35000/exe (readlink: Permission denied) 2024-03-07T07:43:30.636 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35000 progress ceph NOFD /proc/32200/task/35000/fd (opendir: Permission denied) 2024-03-07T07:43:30.636 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35001 m-fin-pro ceph cwd unknown /proc/32200/task/35001/cwd (readlink: Permission denied) 2024-03-07T07:43:30.636 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35001 m-fin-pro ceph rtd unknown /proc/32200/task/35001/root (readlink: Permission denied) 2024-03-07T07:43:30.636 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35001 m-fin-pro ceph txt unknown /proc/32200/task/35001/exe (readlink: Permission denied) 2024-03-07T07:43:30.636 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35001 m-fin-pro ceph NOFD /proc/32200/task/35001/fd (opendir: Permission denied) 2024-03-07T07:43:30.636 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35005 io_contex ceph cwd unknown /proc/32200/task/35005/cwd (readlink: Permission denied) 2024-03-07T07:43:30.636 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35005 io_contex ceph rtd unknown /proc/32200/task/35005/root (readlink: Permission denied) 2024-03-07T07:43:30.636 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35005 io_contex ceph txt unknown /proc/32200/task/35005/exe (readlink: Permission denied) 2024-03-07T07:43:30.636 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35005 io_contex ceph NOFD /proc/32200/task/35005/fd (opendir: Permission denied) 2024-03-07T07:43:30.636 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35006 io_contex ceph cwd unknown /proc/32200/task/35006/cwd (readlink: Permission denied) 2024-03-07T07:43:30.636 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35006 io_contex ceph rtd unknown /proc/32200/task/35006/root (readlink: Permission denied) 2024-03-07T07:43:30.636 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35006 io_contex ceph txt unknown /proc/32200/task/35006/exe (readlink: Permission denied) 2024-03-07T07:43:30.636 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35006 io_contex ceph NOFD /proc/32200/task/35006/fd (opendir: Permission denied) 2024-03-07T07:43:30.637 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35007 ceph_time ceph cwd unknown /proc/32200/task/35007/cwd (readlink: Permission denied) 2024-03-07T07:43:30.637 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35007 ceph_time ceph rtd unknown /proc/32200/task/35007/root (readlink: Permission denied) 2024-03-07T07:43:30.637 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35007 ceph_time ceph txt unknown /proc/32200/task/35007/exe (readlink: Permission denied) 2024-03-07T07:43:30.637 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35007 ceph_time ceph NOFD /proc/32200/task/35007/fd (opendir: Permission denied) 2024-03-07T07:43:30.637 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35008 ms_dispat ceph cwd unknown /proc/32200/task/35008/cwd (readlink: Permission denied) 2024-03-07T07:43:30.637 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35008 ms_dispat ceph rtd unknown /proc/32200/task/35008/root (readlink: Permission denied) 2024-03-07T07:43:30.637 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35008 ms_dispat ceph txt unknown /proc/32200/task/35008/exe (readlink: Permission denied) 2024-03-07T07:43:30.637 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35008 ms_dispat ceph NOFD /proc/32200/task/35008/fd (opendir: Permission denied) 2024-03-07T07:43:30.637 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35009 ms_local ceph cwd unknown /proc/32200/task/35009/cwd (readlink: Permission denied) 2024-03-07T07:43:30.637 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35009 ms_local ceph rtd unknown /proc/32200/task/35009/root (readlink: Permission denied) 2024-03-07T07:43:30.637 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35009 ms_local ceph txt unknown /proc/32200/task/35009/exe (readlink: Permission denied) 2024-03-07T07:43:30.637 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35009 ms_local ceph NOFD /proc/32200/task/35009/fd (opendir: Permission denied) 2024-03-07T07:43:30.637 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35010 safe_time ceph cwd unknown /proc/32200/task/35010/cwd (readlink: Permission denied) 2024-03-07T07:43:30.637 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35010 safe_time ceph rtd unknown /proc/32200/task/35010/root (readlink: Permission denied) 2024-03-07T07:43:30.638 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35010 safe_time ceph txt unknown /proc/32200/task/35010/exe (readlink: Permission denied) 2024-03-07T07:43:30.638 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35010 safe_time ceph NOFD /proc/32200/task/35010/fd (opendir: Permission denied) 2024-03-07T07:43:30.638 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35011 safe_time ceph cwd unknown /proc/32200/task/35011/cwd (readlink: Permission denied) 2024-03-07T07:43:30.638 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35011 safe_time ceph rtd unknown /proc/32200/task/35011/root (readlink: Permission denied) 2024-03-07T07:43:30.638 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35011 safe_time ceph txt unknown /proc/32200/task/35011/exe (readlink: Permission denied) 2024-03-07T07:43:30.638 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35011 safe_time ceph NOFD /proc/32200/task/35011/fd (opendir: Permission denied) 2024-03-07T07:43:30.638 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35012 mgr-fin ceph cwd unknown /proc/32200/task/35012/cwd (readlink: Permission denied) 2024-03-07T07:43:30.638 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35012 mgr-fin ceph rtd unknown /proc/32200/task/35012/root (readlink: Permission denied) 2024-03-07T07:43:30.638 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35012 mgr-fin ceph txt unknown /proc/32200/task/35012/exe (readlink: Permission denied) 2024-03-07T07:43:30.638 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35012 mgr-fin ceph NOFD /proc/32200/task/35012/fd (opendir: Permission denied) 2024-03-07T07:43:30.638 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35014 m-fin-rbd ceph cwd unknown /proc/32200/task/35014/cwd (readlink: Permission denied) 2024-03-07T07:43:30.638 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35014 m-fin-rbd ceph rtd unknown /proc/32200/task/35014/root (readlink: Permission denied) 2024-03-07T07:43:30.638 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35014 m-fin-rbd ceph txt unknown /proc/32200/task/35014/exe (readlink: Permission denied) 2024-03-07T07:43:30.638 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35014 m-fin-rbd ceph NOFD /proc/32200/task/35014/fd (opendir: Permission denied) 2024-03-07T07:43:30.639 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35015 restful ceph cwd unknown /proc/32200/task/35015/cwd (readlink: Permission denied) 2024-03-07T07:43:30.639 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35015 restful ceph rtd unknown /proc/32200/task/35015/root (readlink: Permission denied) 2024-03-07T07:43:30.639 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35015 restful ceph txt unknown /proc/32200/task/35015/exe (readlink: Permission denied) 2024-03-07T07:43:30.639 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35015 restful ceph NOFD /proc/32200/task/35015/fd (opendir: Permission denied) 2024-03-07T07:43:30.639 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35016 m-fin-res ceph cwd unknown /proc/32200/task/35016/cwd (readlink: Permission denied) 2024-03-07T07:43:30.639 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35016 m-fin-res ceph rtd unknown /proc/32200/task/35016/root (readlink: Permission denied) 2024-03-07T07:43:30.639 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35016 m-fin-res ceph txt unknown /proc/32200/task/35016/exe (readlink: Permission denied) 2024-03-07T07:43:30.639 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35016 m-fin-res ceph NOFD /proc/32200/task/35016/fd (opendir: Permission denied) 2024-03-07T07:43:30.639 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35017 mgr-fin ceph cwd unknown /proc/32200/task/35017/cwd (readlink: Permission denied) 2024-03-07T07:43:30.639 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35017 mgr-fin ceph rtd unknown /proc/32200/task/35017/root (readlink: Permission denied) 2024-03-07T07:43:30.639 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35017 mgr-fin ceph txt unknown /proc/32200/task/35017/exe (readlink: Permission denied) 2024-03-07T07:43:30.639 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35017 mgr-fin ceph NOFD /proc/32200/task/35017/fd (opendir: Permission denied) 2024-03-07T07:43:30.639 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35018 mgr-fin ceph cwd unknown /proc/32200/task/35018/cwd (readlink: Permission denied) 2024-03-07T07:43:30.639 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35018 mgr-fin ceph rtd unknown /proc/32200/task/35018/root (readlink: Permission denied) 2024-03-07T07:43:30.639 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35018 mgr-fin ceph txt unknown /proc/32200/task/35018/exe (readlink: Permission denied) 2024-03-07T07:43:30.640 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35018 mgr-fin ceph NOFD /proc/32200/task/35018/fd (opendir: Permission denied) 2024-03-07T07:43:30.640 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35019 mgr-fin ceph cwd unknown /proc/32200/task/35019/cwd (readlink: Permission denied) 2024-03-07T07:43:30.640 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35019 mgr-fin ceph rtd unknown /proc/32200/task/35019/root (readlink: Permission denied) 2024-03-07T07:43:30.640 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35019 mgr-fin ceph txt unknown /proc/32200/task/35019/exe (readlink: Permission denied) 2024-03-07T07:43:30.640 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35019 mgr-fin ceph NOFD /proc/32200/task/35019/fd (opendir: Permission denied) 2024-03-07T07:43:30.640 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35021 m-fin-sta ceph cwd unknown /proc/32200/task/35021/cwd (readlink: Permission denied) 2024-03-07T07:43:30.640 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35021 m-fin-sta ceph rtd unknown /proc/32200/task/35021/root (readlink: Permission denied) 2024-03-07T07:43:30.640 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35021 m-fin-sta ceph txt unknown /proc/32200/task/35021/exe (readlink: Permission denied) 2024-03-07T07:43:30.640 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35021 m-fin-sta ceph NOFD /proc/32200/task/35021/fd (opendir: Permission denied) 2024-03-07T07:43:30.640 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35022 mgr-fin ceph cwd unknown /proc/32200/task/35022/cwd (readlink: Permission denied) 2024-03-07T07:43:30.640 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35022 mgr-fin ceph rtd unknown /proc/32200/task/35022/root (readlink: Permission denied) 2024-03-07T07:43:30.640 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35022 mgr-fin ceph txt unknown /proc/32200/task/35022/exe (readlink: Permission denied) 2024-03-07T07:43:30.640 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35022 mgr-fin ceph NOFD /proc/32200/task/35022/fd (opendir: Permission denied) 2024-03-07T07:43:30.641 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35023 telemetry ceph cwd unknown /proc/32200/task/35023/cwd (readlink: Permission denied) 2024-03-07T07:43:30.641 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35023 telemetry ceph rtd unknown /proc/32200/task/35023/root (readlink: Permission denied) 2024-03-07T07:43:30.641 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35023 telemetry ceph txt unknown /proc/32200/task/35023/exe (readlink: Permission denied) 2024-03-07T07:43:30.641 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35023 telemetry ceph NOFD /proc/32200/task/35023/fd (opendir: Permission denied) 2024-03-07T07:43:30.641 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35024 m-fin-tel ceph cwd unknown /proc/32200/task/35024/cwd (readlink: Permission denied) 2024-03-07T07:43:30.641 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35024 m-fin-tel ceph rtd unknown /proc/32200/task/35024/root (readlink: Permission denied) 2024-03-07T07:43:30.641 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35024 m-fin-tel ceph txt unknown /proc/32200/task/35024/exe (readlink: Permission denied) 2024-03-07T07:43:30.641 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35024 m-fin-tel ceph NOFD /proc/32200/task/35024/fd (opendir: Permission denied) 2024-03-07T07:43:30.641 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35025 mgr-fin ceph cwd unknown /proc/32200/task/35025/cwd (readlink: Permission denied) 2024-03-07T07:43:30.641 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35025 mgr-fin ceph rtd unknown /proc/32200/task/35025/root (readlink: Permission denied) 2024-03-07T07:43:30.641 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35025 mgr-fin ceph txt unknown /proc/32200/task/35025/exe (readlink: Permission denied) 2024-03-07T07:43:30.641 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35025 mgr-fin ceph NOFD /proc/32200/task/35025/fd (opendir: Permission denied) 2024-03-07T07:43:30.641 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35030 io_contex ceph cwd unknown /proc/32200/task/35030/cwd (readlink: Permission denied) 2024-03-07T07:43:30.641 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35030 io_contex ceph rtd unknown /proc/32200/task/35030/root (readlink: Permission denied) 2024-03-07T07:43:30.641 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35030 io_contex ceph txt unknown /proc/32200/task/35030/exe (readlink: Permission denied) 2024-03-07T07:43:30.642 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35030 io_contex ceph NOFD /proc/32200/task/35030/fd (opendir: Permission denied) 2024-03-07T07:43:30.642 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35031 io_contex ceph cwd unknown /proc/32200/task/35031/cwd (readlink: Permission denied) 2024-03-07T07:43:30.642 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35031 io_contex ceph rtd unknown /proc/32200/task/35031/root (readlink: Permission denied) 2024-03-07T07:43:30.642 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35031 io_contex ceph txt unknown /proc/32200/task/35031/exe (readlink: Permission denied) 2024-03-07T07:43:30.642 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35031 io_contex ceph NOFD /proc/32200/task/35031/fd (opendir: Permission denied) 2024-03-07T07:43:30.642 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35032 ceph_time ceph cwd unknown /proc/32200/task/35032/cwd (readlink: Permission denied) 2024-03-07T07:43:30.642 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35032 ceph_time ceph rtd unknown /proc/32200/task/35032/root (readlink: Permission denied) 2024-03-07T07:43:30.642 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35032 ceph_time ceph txt unknown /proc/32200/task/35032/exe (readlink: Permission denied) 2024-03-07T07:43:30.642 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35032 ceph_time ceph NOFD /proc/32200/task/35032/fd (opendir: Permission denied) 2024-03-07T07:43:30.642 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35033 ms_dispat ceph cwd unknown /proc/32200/task/35033/cwd (readlink: Permission denied) 2024-03-07T07:43:30.642 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35033 ms_dispat ceph rtd unknown /proc/32200/task/35033/root (readlink: Permission denied) 2024-03-07T07:43:30.642 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35033 ms_dispat ceph txt unknown /proc/32200/task/35033/exe (readlink: Permission denied) 2024-03-07T07:43:30.642 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35033 ms_dispat ceph NOFD /proc/32200/task/35033/fd (opendir: Permission denied) 2024-03-07T07:43:30.642 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35034 ms_local ceph cwd unknown /proc/32200/task/35034/cwd (readlink: Permission denied) 2024-03-07T07:43:30.643 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35034 ms_local ceph rtd unknown /proc/32200/task/35034/root (readlink: Permission denied) 2024-03-07T07:43:30.643 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35034 ms_local ceph txt unknown /proc/32200/task/35034/exe (readlink: Permission denied) 2024-03-07T07:43:30.643 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35034 ms_local ceph NOFD /proc/32200/task/35034/fd (opendir: Permission denied) 2024-03-07T07:43:30.643 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35035 safe_time ceph cwd unknown /proc/32200/task/35035/cwd (readlink: Permission denied) 2024-03-07T07:43:30.643 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35035 safe_time ceph rtd unknown /proc/32200/task/35035/root (readlink: Permission denied) 2024-03-07T07:43:30.643 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35035 safe_time ceph txt unknown /proc/32200/task/35035/exe (readlink: Permission denied) 2024-03-07T07:43:30.643 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35035 safe_time ceph NOFD /proc/32200/task/35035/fd (opendir: Permission denied) 2024-03-07T07:43:30.643 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35036 safe_time ceph cwd unknown /proc/32200/task/35036/cwd (readlink: Permission denied) 2024-03-07T07:43:30.643 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35036 safe_time ceph rtd unknown /proc/32200/task/35036/root (readlink: Permission denied) 2024-03-07T07:43:30.643 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35036 safe_time ceph txt unknown /proc/32200/task/35036/exe (readlink: Permission denied) 2024-03-07T07:43:30.643 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35036 safe_time ceph NOFD /proc/32200/task/35036/fd (opendir: Permission denied) 2024-03-07T07:43:30.643 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35037 mgr-fin ceph cwd unknown /proc/32200/task/35037/cwd (readlink: Permission denied) 2024-03-07T07:43:30.643 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35037 mgr-fin ceph rtd unknown /proc/32200/task/35037/root (readlink: Permission denied) 2024-03-07T07:43:30.643 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35037 mgr-fin ceph txt unknown /proc/32200/task/35037/exe (readlink: Permission denied) 2024-03-07T07:43:30.643 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35037 mgr-fin ceph NOFD /proc/32200/task/35037/fd (opendir: Permission denied) 2024-03-07T07:43:30.644 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35038 mgr-fin ceph cwd unknown /proc/32200/task/35038/cwd (readlink: Permission denied) 2024-03-07T07:43:30.644 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35038 mgr-fin ceph rtd unknown /proc/32200/task/35038/root (readlink: Permission denied) 2024-03-07T07:43:30.644 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35038 mgr-fin ceph txt unknown /proc/32200/task/35038/exe (readlink: Permission denied) 2024-03-07T07:43:30.644 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35038 mgr-fin ceph NOFD /proc/32200/task/35038/fd (opendir: Permission denied) 2024-03-07T07:43:30.644 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35039 mgr-fin ceph cwd unknown /proc/32200/task/35039/cwd (readlink: Permission denied) 2024-03-07T07:43:30.644 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35039 mgr-fin ceph rtd unknown /proc/32200/task/35039/root (readlink: Permission denied) 2024-03-07T07:43:30.644 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35039 mgr-fin ceph txt unknown /proc/32200/task/35039/exe (readlink: Permission denied) 2024-03-07T07:43:30.644 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35039 mgr-fin ceph NOFD /proc/32200/task/35039/fd (opendir: Permission denied) 2024-03-07T07:43:30.644 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35040 mgr-fin ceph cwd unknown /proc/32200/task/35040/cwd (readlink: Permission denied) 2024-03-07T07:43:30.644 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35040 mgr-fin ceph rtd unknown /proc/32200/task/35040/root (readlink: Permission denied) 2024-03-07T07:43:30.644 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35040 mgr-fin ceph txt unknown /proc/32200/task/35040/exe (readlink: Permission denied) 2024-03-07T07:43:30.644 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35040 mgr-fin ceph NOFD /proc/32200/task/35040/fd (opendir: Permission denied) 2024-03-07T07:43:30.644 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35041 mgr-fin ceph cwd unknown /proc/32200/task/35041/cwd (readlink: Permission denied) 2024-03-07T07:43:30.645 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35041 mgr-fin ceph rtd unknown /proc/32200/task/35041/root (readlink: Permission denied) 2024-03-07T07:43:30.645 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35041 mgr-fin ceph txt unknown /proc/32200/task/35041/exe (readlink: Permission denied) 2024-03-07T07:43:30.645 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35041 mgr-fin ceph NOFD /proc/32200/task/35041/fd (opendir: Permission denied) 2024-03-07T07:43:30.645 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35042 mgr-fin ceph cwd unknown /proc/32200/task/35042/cwd (readlink: Permission denied) 2024-03-07T07:43:30.645 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35042 mgr-fin ceph rtd unknown /proc/32200/task/35042/root (readlink: Permission denied) 2024-03-07T07:43:30.645 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35042 mgr-fin ceph txt unknown /proc/32200/task/35042/exe (readlink: Permission denied) 2024-03-07T07:43:30.645 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35042 mgr-fin ceph NOFD /proc/32200/task/35042/fd (opendir: Permission denied) 2024-03-07T07:43:30.645 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35043 mgr-fin ceph cwd unknown /proc/32200/task/35043/cwd (readlink: Permission denied) 2024-03-07T07:43:30.645 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35043 mgr-fin ceph rtd unknown /proc/32200/task/35043/root (readlink: Permission denied) 2024-03-07T07:43:30.645 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35043 mgr-fin ceph txt unknown /proc/32200/task/35043/exe (readlink: Permission denied) 2024-03-07T07:43:30.645 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35043 mgr-fin ceph NOFD /proc/32200/task/35043/fd (opendir: Permission denied) 2024-03-07T07:43:30.645 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35044 mgr-fin ceph cwd unknown /proc/32200/task/35044/cwd (readlink: Permission denied) 2024-03-07T07:43:30.645 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35044 mgr-fin ceph rtd unknown /proc/32200/task/35044/root (readlink: Permission denied) 2024-03-07T07:43:30.645 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35044 mgr-fin ceph txt unknown /proc/32200/task/35044/exe (readlink: Permission denied) 2024-03-07T07:43:30.646 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35044 mgr-fin ceph NOFD /proc/32200/task/35044/fd (opendir: Permission denied) 2024-03-07T07:43:30.646 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35045 mgr-fin ceph cwd unknown /proc/32200/task/35045/cwd (readlink: Permission denied) 2024-03-07T07:43:30.646 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35045 mgr-fin ceph rtd unknown /proc/32200/task/35045/root (readlink: Permission denied) 2024-03-07T07:43:30.646 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35045 mgr-fin ceph txt unknown /proc/32200/task/35045/exe (readlink: Permission denied) 2024-03-07T07:43:30.646 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35045 mgr-fin ceph NOFD /proc/32200/task/35045/fd (opendir: Permission denied) 2024-03-07T07:43:30.646 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35046 mgr-fin ceph cwd unknown /proc/32200/task/35046/cwd (readlink: Permission denied) 2024-03-07T07:43:30.646 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35046 mgr-fin ceph rtd unknown /proc/32200/task/35046/root (readlink: Permission denied) 2024-03-07T07:43:30.646 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35046 mgr-fin ceph txt unknown /proc/32200/task/35046/exe (readlink: Permission denied) 2024-03-07T07:43:30.646 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35046 mgr-fin ceph NOFD /proc/32200/task/35046/fd (opendir: Permission denied) 2024-03-07T07:43:30.646 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35047 mgr-fin ceph cwd unknown /proc/32200/task/35047/cwd (readlink: Permission denied) 2024-03-07T07:43:30.646 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35047 mgr-fin ceph rtd unknown /proc/32200/task/35047/root (readlink: Permission denied) 2024-03-07T07:43:30.646 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35047 mgr-fin ceph txt unknown /proc/32200/task/35047/exe (readlink: Permission denied) 2024-03-07T07:43:30.646 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35047 mgr-fin ceph NOFD /proc/32200/task/35047/fd (opendir: Permission denied) 2024-03-07T07:43:30.646 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35048 mgr-fin ceph cwd unknown /proc/32200/task/35048/cwd (readlink: Permission denied) 2024-03-07T07:43:30.646 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35048 mgr-fin ceph rtd unknown /proc/32200/task/35048/root (readlink: Permission denied) 2024-03-07T07:43:30.647 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35048 mgr-fin ceph txt unknown /proc/32200/task/35048/exe (readlink: Permission denied) 2024-03-07T07:43:30.647 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35048 mgr-fin ceph NOFD /proc/32200/task/35048/fd (opendir: Permission denied) 2024-03-07T07:43:30.647 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35050 m-fin-vol ceph cwd unknown /proc/32200/task/35050/cwd (readlink: Permission denied) 2024-03-07T07:43:30.647 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35050 m-fin-vol ceph rtd unknown /proc/32200/task/35050/root (readlink: Permission denied) 2024-03-07T07:43:30.647 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35050 m-fin-vol ceph txt unknown /proc/32200/task/35050/exe (readlink: Permission denied) 2024-03-07T07:43:30.647 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35050 m-fin-vol ceph NOFD /proc/32200/task/35050/fd (opendir: Permission denied) 2024-03-07T07:43:30.647 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35121 mgr-fin ceph cwd unknown /proc/32200/task/35121/cwd (readlink: Permission denied) 2024-03-07T07:43:30.647 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35121 mgr-fin ceph rtd unknown /proc/32200/task/35121/root (readlink: Permission denied) 2024-03-07T07:43:30.647 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35121 mgr-fin ceph txt unknown /proc/32200/task/35121/exe (readlink: Permission denied) 2024-03-07T07:43:30.647 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35121 mgr-fin ceph NOFD /proc/32200/task/35121/fd (opendir: Permission denied) 2024-03-07T07:43:30.647 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35122 mgr-fin ceph cwd unknown /proc/32200/task/35122/cwd (readlink: Permission denied) 2024-03-07T07:43:30.647 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35122 mgr-fin ceph rtd unknown /proc/32200/task/35122/root (readlink: Permission denied) 2024-03-07T07:43:30.647 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35122 mgr-fin ceph txt unknown /proc/32200/task/35122/exe (readlink: Permission denied) 2024-03-07T07:43:30.647 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35122 mgr-fin ceph NOFD /proc/32200/task/35122/fd (opendir: Permission denied) 2024-03-07T07:43:30.648 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35123 mgr-fin ceph cwd unknown /proc/32200/task/35123/cwd (readlink: Permission denied) 2024-03-07T07:43:30.648 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35123 mgr-fin ceph rtd unknown /proc/32200/task/35123/root (readlink: Permission denied) 2024-03-07T07:43:30.648 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35123 mgr-fin ceph txt unknown /proc/32200/task/35123/exe (readlink: Permission denied) 2024-03-07T07:43:30.648 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35123 mgr-fin ceph NOFD /proc/32200/task/35123/fd (opendir: Permission denied) 2024-03-07T07:43:30.648 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35124 mgr-fin ceph cwd unknown /proc/32200/task/35124/cwd (readlink: Permission denied) 2024-03-07T07:43:30.648 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35124 mgr-fin ceph rtd unknown /proc/32200/task/35124/root (readlink: Permission denied) 2024-03-07T07:43:30.648 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35124 mgr-fin ceph txt unknown /proc/32200/task/35124/exe (readlink: Permission denied) 2024-03-07T07:43:30.648 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35124 mgr-fin ceph NOFD /proc/32200/task/35124/fd (opendir: Permission denied) 2024-03-07T07:43:30.648 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35125 mgr-fin ceph cwd unknown /proc/32200/task/35125/cwd (readlink: Permission denied) 2024-03-07T07:43:30.648 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35125 mgr-fin ceph rtd unknown /proc/32200/task/35125/root (readlink: Permission denied) 2024-03-07T07:43:30.648 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35125 mgr-fin ceph txt unknown /proc/32200/task/35125/exe (readlink: Permission denied) 2024-03-07T07:43:30.648 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35125 mgr-fin ceph NOFD /proc/32200/task/35125/fd (opendir: Permission denied) 2024-03-07T07:43:30.648 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35126 mgr-fin ceph cwd unknown /proc/32200/task/35126/cwd (readlink: Permission denied) 2024-03-07T07:43:30.649 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35126 mgr-fin ceph rtd unknown /proc/32200/task/35126/root (readlink: Permission denied) 2024-03-07T07:43:30.649 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35126 mgr-fin ceph txt unknown /proc/32200/task/35126/exe (readlink: Permission denied) 2024-03-07T07:43:30.649 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35126 mgr-fin ceph NOFD /proc/32200/task/35126/fd (opendir: Permission denied) 2024-03-07T07:43:30.649 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35127 mgr-fin ceph cwd unknown /proc/32200/task/35127/cwd (readlink: Permission denied) 2024-03-07T07:43:30.649 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35127 mgr-fin ceph rtd unknown /proc/32200/task/35127/root (readlink: Permission denied) 2024-03-07T07:43:30.649 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35127 mgr-fin ceph txt unknown /proc/32200/task/35127/exe (readlink: Permission denied) 2024-03-07T07:43:30.649 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35127 mgr-fin ceph NOFD /proc/32200/task/35127/fd (opendir: Permission denied) 2024-03-07T07:43:30.649 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35128 mgr-fin ceph cwd unknown /proc/32200/task/35128/cwd (readlink: Permission denied) 2024-03-07T07:43:30.649 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35128 mgr-fin ceph rtd unknown /proc/32200/task/35128/root (readlink: Permission denied) 2024-03-07T07:43:30.649 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35128 mgr-fin ceph txt unknown /proc/32200/task/35128/exe (readlink: Permission denied) 2024-03-07T07:43:30.649 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35128 mgr-fin ceph NOFD /proc/32200/task/35128/fd (opendir: Permission denied) 2024-03-07T07:43:30.649 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35129 mgr-fin ceph cwd unknown /proc/32200/task/35129/cwd (readlink: Permission denied) 2024-03-07T07:43:30.649 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35129 mgr-fin ceph rtd unknown /proc/32200/task/35129/root (readlink: Permission denied) 2024-03-07T07:43:30.649 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35129 mgr-fin ceph txt unknown /proc/32200/task/35129/exe (readlink: Permission denied) 2024-03-07T07:43:30.650 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35129 mgr-fin ceph NOFD /proc/32200/task/35129/fd (opendir: Permission denied) 2024-03-07T07:43:30.650 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35130 mgr-fin ceph cwd unknown /proc/32200/task/35130/cwd (readlink: Permission denied) 2024-03-07T07:43:30.650 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35130 mgr-fin ceph rtd unknown /proc/32200/task/35130/root (readlink: Permission denied) 2024-03-07T07:43:30.650 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35130 mgr-fin ceph txt unknown /proc/32200/task/35130/exe (readlink: Permission denied) 2024-03-07T07:43:30.650 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35130 mgr-fin ceph NOFD /proc/32200/task/35130/fd (opendir: Permission denied) 2024-03-07T07:43:30.650 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35131 mgr-fin ceph cwd unknown /proc/32200/task/35131/cwd (readlink: Permission denied) 2024-03-07T07:43:30.650 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35131 mgr-fin ceph rtd unknown /proc/32200/task/35131/root (readlink: Permission denied) 2024-03-07T07:43:30.650 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35131 mgr-fin ceph txt unknown /proc/32200/task/35131/exe (readlink: Permission denied) 2024-03-07T07:43:30.650 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35131 mgr-fin ceph NOFD /proc/32200/task/35131/fd (opendir: Permission denied) 2024-03-07T07:43:30.650 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35176 mgr-fin ceph cwd unknown /proc/32200/task/35176/cwd (readlink: Permission denied) 2024-03-07T07:43:30.650 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35176 mgr-fin ceph rtd unknown /proc/32200/task/35176/root (readlink: Permission denied) 2024-03-07T07:43:30.650 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35176 mgr-fin ceph txt unknown /proc/32200/task/35176/exe (readlink: Permission denied) 2024-03-07T07:43:30.650 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35176 mgr-fin ceph NOFD /proc/32200/task/35176/fd (opendir: Permission denied) 2024-03-07T07:43:30.650 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35177 mgr-fin ceph cwd unknown /proc/32200/task/35177/cwd (readlink: Permission denied) 2024-03-07T07:43:30.651 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35177 mgr-fin ceph rtd unknown /proc/32200/task/35177/root (readlink: Permission denied) 2024-03-07T07:43:30.651 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35177 mgr-fin ceph txt unknown /proc/32200/task/35177/exe (readlink: Permission denied) 2024-03-07T07:43:30.651 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35177 mgr-fin ceph NOFD /proc/32200/task/35177/fd (opendir: Permission denied) 2024-03-07T07:43:30.651 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35178 mgr-fin ceph cwd unknown /proc/32200/task/35178/cwd (readlink: Permission denied) 2024-03-07T07:43:30.651 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35178 mgr-fin ceph rtd unknown /proc/32200/task/35178/root (readlink: Permission denied) 2024-03-07T07:43:30.651 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35178 mgr-fin ceph txt unknown /proc/32200/task/35178/exe (readlink: Permission denied) 2024-03-07T07:43:30.651 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35178 mgr-fin ceph NOFD /proc/32200/task/35178/fd (opendir: Permission denied) 2024-03-07T07:43:30.651 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35179 mgr-fin ceph cwd unknown /proc/32200/task/35179/cwd (readlink: Permission denied) 2024-03-07T07:43:30.651 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35179 mgr-fin ceph rtd unknown /proc/32200/task/35179/root (readlink: Permission denied) 2024-03-07T07:43:30.651 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35179 mgr-fin ceph txt unknown /proc/32200/task/35179/exe (readlink: Permission denied) 2024-03-07T07:43:30.651 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35179 mgr-fin ceph NOFD /proc/32200/task/35179/fd (opendir: Permission denied) 2024-03-07T07:43:30.651 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35180 mgr-fin ceph cwd unknown /proc/32200/task/35180/cwd (readlink: Permission denied) 2024-03-07T07:43:30.651 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35180 mgr-fin ceph rtd unknown /proc/32200/task/35180/root (readlink: Permission denied) 2024-03-07T07:43:30.652 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35180 mgr-fin ceph txt unknown /proc/32200/task/35180/exe (readlink: Permission denied) 2024-03-07T07:43:30.652 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35180 mgr-fin ceph NOFD /proc/32200/task/35180/fd (opendir: Permission denied) 2024-03-07T07:43:30.652 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35182 mgr-fin ceph cwd unknown /proc/32200/task/35182/cwd (readlink: Permission denied) 2024-03-07T07:43:30.652 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35182 mgr-fin ceph rtd unknown /proc/32200/task/35182/root (readlink: Permission denied) 2024-03-07T07:43:30.652 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35182 mgr-fin ceph txt unknown /proc/32200/task/35182/exe (readlink: Permission denied) 2024-03-07T07:43:30.652 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35182 mgr-fin ceph NOFD /proc/32200/task/35182/fd (opendir: Permission denied) 2024-03-07T07:43:30.652 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35183 mgr-fin ceph cwd unknown /proc/32200/task/35183/cwd (readlink: Permission denied) 2024-03-07T07:43:30.652 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35183 mgr-fin ceph rtd unknown /proc/32200/task/35183/root (readlink: Permission denied) 2024-03-07T07:43:30.652 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35183 mgr-fin ceph txt unknown /proc/32200/task/35183/exe (readlink: Permission denied) 2024-03-07T07:43:30.652 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35183 mgr-fin ceph NOFD /proc/32200/task/35183/fd (opendir: Permission denied) 2024-03-07T07:43:30.652 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35184 mgr-fin ceph cwd unknown /proc/32200/task/35184/cwd (readlink: Permission denied) 2024-03-07T07:43:30.652 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35184 mgr-fin ceph rtd unknown /proc/32200/task/35184/root (readlink: Permission denied) 2024-03-07T07:43:30.652 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35184 mgr-fin ceph txt unknown /proc/32200/task/35184/exe (readlink: Permission denied) 2024-03-07T07:43:30.652 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35184 mgr-fin ceph NOFD /proc/32200/task/35184/fd (opendir: Permission denied) 2024-03-07T07:43:30.653 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35185 mgr-fin ceph cwd unknown /proc/32200/task/35185/cwd (readlink: Permission denied) 2024-03-07T07:43:30.653 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35185 mgr-fin ceph rtd unknown /proc/32200/task/35185/root (readlink: Permission denied) 2024-03-07T07:43:30.653 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35185 mgr-fin ceph txt unknown /proc/32200/task/35185/exe (readlink: Permission denied) 2024-03-07T07:43:30.653 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35185 mgr-fin ceph NOFD /proc/32200/task/35185/fd (opendir: Permission denied) 2024-03-07T07:43:30.653 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35186 mgr-fin ceph cwd unknown /proc/32200/task/35186/cwd (readlink: Permission denied) 2024-03-07T07:43:30.653 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35186 mgr-fin ceph rtd unknown /proc/32200/task/35186/root (readlink: Permission denied) 2024-03-07T07:43:30.653 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35186 mgr-fin ceph txt unknown /proc/32200/task/35186/exe (readlink: Permission denied) 2024-03-07T07:43:30.653 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35186 mgr-fin ceph NOFD /proc/32200/task/35186/fd (opendir: Permission denied) 2024-03-07T07:43:30.653 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35187 mgr-fin ceph cwd unknown /proc/32200/task/35187/cwd (readlink: Permission denied) 2024-03-07T07:43:30.653 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35187 mgr-fin ceph rtd unknown /proc/32200/task/35187/root (readlink: Permission denied) 2024-03-07T07:43:30.653 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35187 mgr-fin ceph txt unknown /proc/32200/task/35187/exe (readlink: Permission denied) 2024-03-07T07:43:30.653 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35187 mgr-fin ceph NOFD /proc/32200/task/35187/fd (opendir: Permission denied) 2024-03-07T07:43:30.653 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35721 dashboard ceph cwd unknown /proc/32200/task/35721/cwd (readlink: Permission denied) 2024-03-07T07:43:30.654 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35721 dashboard ceph rtd unknown /proc/32200/task/35721/root (readlink: Permission denied) 2024-03-07T07:43:30.654 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35721 dashboard ceph txt unknown /proc/32200/task/35721/exe (readlink: Permission denied) 2024-03-07T07:43:30.654 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35721 dashboard ceph NOFD /proc/32200/task/35721/fd (opendir: Permission denied) 2024-03-07T07:43:30.654 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35722 dashboard ceph cwd unknown /proc/32200/task/35722/cwd (readlink: Permission denied) 2024-03-07T07:43:30.654 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35722 dashboard ceph rtd unknown /proc/32200/task/35722/root (readlink: Permission denied) 2024-03-07T07:43:30.654 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35722 dashboard ceph txt unknown /proc/32200/task/35722/exe (readlink: Permission denied) 2024-03-07T07:43:30.654 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35722 dashboard ceph NOFD /proc/32200/task/35722/fd (opendir: Permission denied) 2024-03-07T07:43:30.654 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35723 dashboard ceph cwd unknown /proc/32200/task/35723/cwd (readlink: Permission denied) 2024-03-07T07:43:30.654 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35723 dashboard ceph rtd unknown /proc/32200/task/35723/root (readlink: Permission denied) 2024-03-07T07:43:30.654 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35723 dashboard ceph txt unknown /proc/32200/task/35723/exe (readlink: Permission denied) 2024-03-07T07:43:30.654 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35723 dashboard ceph NOFD /proc/32200/task/35723/fd (opendir: Permission denied) 2024-03-07T07:43:30.654 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35724 dashboard ceph cwd unknown /proc/32200/task/35724/cwd (readlink: Permission denied) 2024-03-07T07:43:30.654 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35724 dashboard ceph rtd unknown /proc/32200/task/35724/root (readlink: Permission denied) 2024-03-07T07:43:30.654 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35724 dashboard ceph txt unknown /proc/32200/task/35724/exe (readlink: Permission denied) 2024-03-07T07:43:30.655 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35724 dashboard ceph NOFD /proc/32200/task/35724/fd (opendir: Permission denied) 2024-03-07T07:43:30.655 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35725 dashboard ceph cwd unknown /proc/32200/task/35725/cwd (readlink: Permission denied) 2024-03-07T07:43:30.655 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35725 dashboard ceph rtd unknown /proc/32200/task/35725/root (readlink: Permission denied) 2024-03-07T07:43:30.655 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35725 dashboard ceph txt unknown /proc/32200/task/35725/exe (readlink: Permission denied) 2024-03-07T07:43:30.655 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35725 dashboard ceph NOFD /proc/32200/task/35725/fd (opendir: Permission denied) 2024-03-07T07:43:30.655 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35726 dashboard ceph cwd unknown /proc/32200/task/35726/cwd (readlink: Permission denied) 2024-03-07T07:43:30.655 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35726 dashboard ceph rtd unknown /proc/32200/task/35726/root (readlink: Permission denied) 2024-03-07T07:43:30.655 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35726 dashboard ceph txt unknown /proc/32200/task/35726/exe (readlink: Permission denied) 2024-03-07T07:43:30.655 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35726 dashboard ceph NOFD /proc/32200/task/35726/fd (opendir: Permission denied) 2024-03-07T07:43:30.655 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35727 dashboard ceph cwd unknown /proc/32200/task/35727/cwd (readlink: Permission denied) 2024-03-07T07:43:30.655 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35727 dashboard ceph rtd unknown /proc/32200/task/35727/root (readlink: Permission denied) 2024-03-07T07:43:30.655 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35727 dashboard ceph txt unknown /proc/32200/task/35727/exe (readlink: Permission denied) 2024-03-07T07:43:30.655 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35727 dashboard ceph NOFD /proc/32200/task/35727/fd (opendir: Permission denied) 2024-03-07T07:43:30.656 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35728 dashboard ceph cwd unknown /proc/32200/task/35728/cwd (readlink: Permission denied) 2024-03-07T07:43:30.656 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35728 dashboard ceph rtd unknown /proc/32200/task/35728/root (readlink: Permission denied) 2024-03-07T07:43:30.656 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35728 dashboard ceph txt unknown /proc/32200/task/35728/exe (readlink: Permission denied) 2024-03-07T07:43:30.656 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35728 dashboard ceph NOFD /proc/32200/task/35728/fd (opendir: Permission denied) 2024-03-07T07:43:30.656 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35729 dashboard ceph cwd unknown /proc/32200/task/35729/cwd (readlink: Permission denied) 2024-03-07T07:43:30.656 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35729 dashboard ceph rtd unknown /proc/32200/task/35729/root (readlink: Permission denied) 2024-03-07T07:43:30.656 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35729 dashboard ceph txt unknown /proc/32200/task/35729/exe (readlink: Permission denied) 2024-03-07T07:43:30.656 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35729 dashboard ceph NOFD /proc/32200/task/35729/fd (opendir: Permission denied) 2024-03-07T07:43:30.656 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35730 dashboard ceph cwd unknown /proc/32200/task/35730/cwd (readlink: Permission denied) 2024-03-07T07:43:30.656 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35730 dashboard ceph rtd unknown /proc/32200/task/35730/root (readlink: Permission denied) 2024-03-07T07:43:30.656 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35730 dashboard ceph txt unknown /proc/32200/task/35730/exe (readlink: Permission denied) 2024-03-07T07:43:30.656 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35730 dashboard ceph NOFD /proc/32200/task/35730/fd (opendir: Permission denied) 2024-03-07T07:43:30.656 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35731 dashboard ceph cwd unknown /proc/32200/task/35731/cwd (readlink: Permission denied) 2024-03-07T07:43:30.657 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35731 dashboard ceph rtd unknown /proc/32200/task/35731/root (readlink: Permission denied) 2024-03-07T07:43:30.657 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35731 dashboard ceph txt unknown /proc/32200/task/35731/exe (readlink: Permission denied) 2024-03-07T07:43:30.657 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35731 dashboard ceph NOFD /proc/32200/task/35731/fd (opendir: Permission denied) 2024-03-07T07:43:30.657 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35733 dashboard ceph cwd unknown /proc/32200/task/35733/cwd (readlink: Permission denied) 2024-03-07T07:43:30.657 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35733 dashboard ceph rtd unknown /proc/32200/task/35733/root (readlink: Permission denied) 2024-03-07T07:43:30.657 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35733 dashboard ceph txt unknown /proc/32200/task/35733/exe (readlink: Permission denied) 2024-03-07T07:43:30.657 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 35733 dashboard ceph NOFD /proc/32200/task/35733/fd (opendir: Permission denied) 2024-03-07T07:43:30.657 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42201 cephadm ceph cwd unknown /proc/32200/task/42201/cwd (readlink: Permission denied) 2024-03-07T07:43:30.657 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42201 cephadm ceph rtd unknown /proc/32200/task/42201/root (readlink: Permission denied) 2024-03-07T07:43:30.657 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42201 cephadm ceph txt unknown /proc/32200/task/42201/exe (readlink: Permission denied) 2024-03-07T07:43:30.657 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42201 cephadm ceph NOFD /proc/32200/task/42201/fd (opendir: Permission denied) 2024-03-07T07:43:30.657 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42202 cephadm ceph cwd unknown /proc/32200/task/42202/cwd (readlink: Permission denied) 2024-03-07T07:43:30.657 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42202 cephadm ceph rtd unknown /proc/32200/task/42202/root (readlink: Permission denied) 2024-03-07T07:43:30.657 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42202 cephadm ceph txt unknown /proc/32200/task/42202/exe (readlink: Permission denied) 2024-03-07T07:43:30.657 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42202 cephadm ceph NOFD /proc/32200/task/42202/fd (opendir: Permission denied) 2024-03-07T07:43:30.658 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42203 cephadm ceph cwd unknown /proc/32200/task/42203/cwd (readlink: Permission denied) 2024-03-07T07:43:30.658 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42203 cephadm ceph rtd unknown /proc/32200/task/42203/root (readlink: Permission denied) 2024-03-07T07:43:30.658 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42203 cephadm ceph txt unknown /proc/32200/task/42203/exe (readlink: Permission denied) 2024-03-07T07:43:30.658 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42203 cephadm ceph NOFD /proc/32200/task/42203/fd (opendir: Permission denied) 2024-03-07T07:43:30.658 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42204 cephadm ceph cwd unknown /proc/32200/task/42204/cwd (readlink: Permission denied) 2024-03-07T07:43:30.658 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42204 cephadm ceph rtd unknown /proc/32200/task/42204/root (readlink: Permission denied) 2024-03-07T07:43:30.658 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42204 cephadm ceph txt unknown /proc/32200/task/42204/exe (readlink: Permission denied) 2024-03-07T07:43:30.658 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42204 cephadm ceph NOFD /proc/32200/task/42204/fd (opendir: Permission denied) 2024-03-07T07:43:30.658 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42205 cephadm ceph cwd unknown /proc/32200/task/42205/cwd (readlink: Permission denied) 2024-03-07T07:43:30.658 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42205 cephadm ceph rtd unknown /proc/32200/task/42205/root (readlink: Permission denied) 2024-03-07T07:43:30.658 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42205 cephadm ceph txt unknown /proc/32200/task/42205/exe (readlink: Permission denied) 2024-03-07T07:43:30.658 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42205 cephadm ceph NOFD /proc/32200/task/42205/fd (opendir: Permission denied) 2024-03-07T07:43:30.658 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42206 cephadm ceph cwd unknown /proc/32200/task/42206/cwd (readlink: Permission denied) 2024-03-07T07:43:30.659 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42206 cephadm ceph rtd unknown /proc/32200/task/42206/root (readlink: Permission denied) 2024-03-07T07:43:30.659 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42206 cephadm ceph txt unknown /proc/32200/task/42206/exe (readlink: Permission denied) 2024-03-07T07:43:30.659 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42206 cephadm ceph NOFD /proc/32200/task/42206/fd (opendir: Permission denied) 2024-03-07T07:43:30.659 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42207 cephadm ceph cwd unknown /proc/32200/task/42207/cwd (readlink: Permission denied) 2024-03-07T07:43:30.659 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42207 cephadm ceph rtd unknown /proc/32200/task/42207/root (readlink: Permission denied) 2024-03-07T07:43:30.659 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42207 cephadm ceph txt unknown /proc/32200/task/42207/exe (readlink: Permission denied) 2024-03-07T07:43:30.659 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42207 cephadm ceph NOFD /proc/32200/task/42207/fd (opendir: Permission denied) 2024-03-07T07:43:30.659 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42208 cephadm ceph cwd unknown /proc/32200/task/42208/cwd (readlink: Permission denied) 2024-03-07T07:43:30.659 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42208 cephadm ceph rtd unknown /proc/32200/task/42208/root (readlink: Permission denied) 2024-03-07T07:43:30.659 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42208 cephadm ceph txt unknown /proc/32200/task/42208/exe (readlink: Permission denied) 2024-03-07T07:43:30.659 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42208 cephadm ceph NOFD /proc/32200/task/42208/fd (opendir: Permission denied) 2024-03-07T07:43:30.659 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42209 cephadm ceph cwd unknown /proc/32200/task/42209/cwd (readlink: Permission denied) 2024-03-07T07:43:30.659 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42209 cephadm ceph rtd unknown /proc/32200/task/42209/root (readlink: Permission denied) 2024-03-07T07:43:30.660 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42209 cephadm ceph txt unknown /proc/32200/task/42209/exe (readlink: Permission denied) 2024-03-07T07:43:30.660 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42209 cephadm ceph NOFD /proc/32200/task/42209/fd (opendir: Permission denied) 2024-03-07T07:43:30.660 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42210 cephadm ceph cwd unknown /proc/32200/task/42210/cwd (readlink: Permission denied) 2024-03-07T07:43:30.660 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42210 cephadm ceph rtd unknown /proc/32200/task/42210/root (readlink: Permission denied) 2024-03-07T07:43:30.660 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42210 cephadm ceph txt unknown /proc/32200/task/42210/exe (readlink: Permission denied) 2024-03-07T07:43:30.660 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42210 cephadm ceph NOFD /proc/32200/task/42210/fd (opendir: Permission denied) 2024-03-07T07:43:30.660 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42527 cephadm ceph cwd unknown /proc/32200/task/42527/cwd (readlink: Permission denied) 2024-03-07T07:43:30.660 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42527 cephadm ceph rtd unknown /proc/32200/task/42527/root (readlink: Permission denied) 2024-03-07T07:43:30.660 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42527 cephadm ceph txt unknown /proc/32200/task/42527/exe (readlink: Permission denied) 2024-03-07T07:43:30.660 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42527 cephadm ceph NOFD /proc/32200/task/42527/fd (opendir: Permission denied) 2024-03-07T07:43:30.660 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42528 cephadm ceph cwd unknown /proc/32200/task/42528/cwd (readlink: Permission denied) 2024-03-07T07:43:30.660 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42528 cephadm ceph rtd unknown /proc/32200/task/42528/root (readlink: Permission denied) 2024-03-07T07:43:30.660 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42528 cephadm ceph txt unknown /proc/32200/task/42528/exe (readlink: Permission denied) 2024-03-07T07:43:30.660 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42528 cephadm ceph NOFD /proc/32200/task/42528/fd (opendir: Permission denied) 2024-03-07T07:43:30.661 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42529 cephadm ceph cwd unknown /proc/32200/task/42529/cwd (readlink: Permission denied) 2024-03-07T07:43:30.661 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42529 cephadm ceph rtd unknown /proc/32200/task/42529/root (readlink: Permission denied) 2024-03-07T07:43:30.661 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42529 cephadm ceph txt unknown /proc/32200/task/42529/exe (readlink: Permission denied) 2024-03-07T07:43:30.661 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42529 cephadm ceph NOFD /proc/32200/task/42529/fd (opendir: Permission denied) 2024-03-07T07:43:30.661 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42530 cephadm ceph cwd unknown /proc/32200/task/42530/cwd (readlink: Permission denied) 2024-03-07T07:43:30.661 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42530 cephadm ceph rtd unknown /proc/32200/task/42530/root (readlink: Permission denied) 2024-03-07T07:43:30.661 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42530 cephadm ceph txt unknown /proc/32200/task/42530/exe (readlink: Permission denied) 2024-03-07T07:43:30.661 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42530 cephadm ceph NOFD /proc/32200/task/42530/fd (opendir: Permission denied) 2024-03-07T07:43:30.661 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42531 cephadm ceph cwd unknown /proc/32200/task/42531/cwd (readlink: Permission denied) 2024-03-07T07:43:30.661 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42531 cephadm ceph rtd unknown /proc/32200/task/42531/root (readlink: Permission denied) 2024-03-07T07:43:30.661 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42531 cephadm ceph txt unknown /proc/32200/task/42531/exe (readlink: Permission denied) 2024-03-07T07:43:30.661 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42531 cephadm ceph NOFD /proc/32200/task/42531/fd (opendir: Permission denied) 2024-03-07T07:43:30.661 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42532 cephadm ceph cwd unknown /proc/32200/task/42532/cwd (readlink: Permission denied) 2024-03-07T07:43:30.661 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42532 cephadm ceph rtd unknown /proc/32200/task/42532/root (readlink: Permission denied) 2024-03-07T07:43:30.661 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42532 cephadm ceph txt unknown /proc/32200/task/42532/exe (readlink: Permission denied) 2024-03-07T07:43:30.662 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42532 cephadm ceph NOFD /proc/32200/task/42532/fd (opendir: Permission denied) 2024-03-07T07:43:30.662 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42533 cephadm ceph cwd unknown /proc/32200/task/42533/cwd (readlink: Permission denied) 2024-03-07T07:43:30.662 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42533 cephadm ceph rtd unknown /proc/32200/task/42533/root (readlink: Permission denied) 2024-03-07T07:43:30.662 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42533 cephadm ceph txt unknown /proc/32200/task/42533/exe (readlink: Permission denied) 2024-03-07T07:43:30.662 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42533 cephadm ceph NOFD /proc/32200/task/42533/fd (opendir: Permission denied) 2024-03-07T07:43:30.662 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42534 cephadm ceph cwd unknown /proc/32200/task/42534/cwd (readlink: Permission denied) 2024-03-07T07:43:30.662 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42534 cephadm ceph rtd unknown /proc/32200/task/42534/root (readlink: Permission denied) 2024-03-07T07:43:30.662 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42534 cephadm ceph txt unknown /proc/32200/task/42534/exe (readlink: Permission denied) 2024-03-07T07:43:30.662 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42534 cephadm ceph NOFD /proc/32200/task/42534/fd (opendir: Permission denied) 2024-03-07T07:43:30.662 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42535 cephadm ceph cwd unknown /proc/32200/task/42535/cwd (readlink: Permission denied) 2024-03-07T07:43:30.662 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42535 cephadm ceph rtd unknown /proc/32200/task/42535/root (readlink: Permission denied) 2024-03-07T07:43:30.662 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42535 cephadm ceph txt unknown /proc/32200/task/42535/exe (readlink: Permission denied) 2024-03-07T07:43:30.662 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42535 cephadm ceph NOFD /proc/32200/task/42535/fd (opendir: Permission denied) 2024-03-07T07:43:30.662 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42536 cephadm ceph cwd unknown /proc/32200/task/42536/cwd (readlink: Permission denied) 2024-03-07T07:43:30.662 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42536 cephadm ceph rtd unknown /proc/32200/task/42536/root (readlink: Permission denied) 2024-03-07T07:43:30.663 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42536 cephadm ceph txt unknown /proc/32200/task/42536/exe (readlink: Permission denied) 2024-03-07T07:43:30.663 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 42536 cephadm ceph NOFD /proc/32200/task/42536/fd (opendir: Permission denied) 2024-03-07T07:43:30.663 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 51298 devicehea ceph cwd unknown /proc/32200/task/51298/cwd (readlink: Permission denied) 2024-03-07T07:43:30.663 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 51298 devicehea ceph rtd unknown /proc/32200/task/51298/root (readlink: Permission denied) 2024-03-07T07:43:30.663 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 51298 devicehea ceph txt unknown /proc/32200/task/51298/exe (readlink: Permission denied) 2024-03-07T07:43:30.663 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 51298 devicehea ceph NOFD /proc/32200/task/51298/fd (opendir: Permission denied) 2024-03-07T07:43:30.663 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 51302 io_contex ceph cwd unknown /proc/32200/task/51302/cwd (readlink: Permission denied) 2024-03-07T07:43:30.663 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 51302 io_contex ceph rtd unknown /proc/32200/task/51302/root (readlink: Permission denied) 2024-03-07T07:43:30.663 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 51302 io_contex ceph txt unknown /proc/32200/task/51302/exe (readlink: Permission denied) 2024-03-07T07:43:30.663 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 51302 io_contex ceph NOFD /proc/32200/task/51302/fd (opendir: Permission denied) 2024-03-07T07:43:30.663 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 51303 io_contex ceph cwd unknown /proc/32200/task/51303/cwd (readlink: Permission denied) 2024-03-07T07:43:30.663 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 51303 io_contex ceph rtd unknown /proc/32200/task/51303/root (readlink: Permission denied) 2024-03-07T07:43:30.663 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 51303 io_contex ceph txt unknown /proc/32200/task/51303/exe (readlink: Permission denied) 2024-03-07T07:43:30.663 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 51303 io_contex ceph NOFD /proc/32200/task/51303/fd (opendir: Permission denied) 2024-03-07T07:43:30.664 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 51304 ceph_time ceph cwd unknown /proc/32200/task/51304/cwd (readlink: Permission denied) 2024-03-07T07:43:30.664 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 51304 ceph_time ceph rtd unknown /proc/32200/task/51304/root (readlink: Permission denied) 2024-03-07T07:43:30.664 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 51304 ceph_time ceph txt unknown /proc/32200/task/51304/exe (readlink: Permission denied) 2024-03-07T07:43:30.664 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 51304 ceph_time ceph NOFD /proc/32200/task/51304/fd (opendir: Permission denied) 2024-03-07T07:43:30.664 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 51305 ms_dispat ceph cwd unknown /proc/32200/task/51305/cwd (readlink: Permission denied) 2024-03-07T07:43:30.664 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 51305 ms_dispat ceph rtd unknown /proc/32200/task/51305/root (readlink: Permission denied) 2024-03-07T07:43:30.664 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 51305 ms_dispat ceph txt unknown /proc/32200/task/51305/exe (readlink: Permission denied) 2024-03-07T07:43:30.664 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 51305 ms_dispat ceph NOFD /proc/32200/task/51305/fd (opendir: Permission denied) 2024-03-07T07:43:30.664 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 51306 ms_local ceph cwd unknown /proc/32200/task/51306/cwd (readlink: Permission denied) 2024-03-07T07:43:30.664 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 51306 ms_local ceph rtd unknown /proc/32200/task/51306/root (readlink: Permission denied) 2024-03-07T07:43:30.664 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 51306 ms_local ceph txt unknown /proc/32200/task/51306/exe (readlink: Permission denied) 2024-03-07T07:43:30.664 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 51306 ms_local ceph NOFD /proc/32200/task/51306/fd (opendir: Permission denied) 2024-03-07T07:43:30.664 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 51307 safe_time ceph cwd unknown /proc/32200/task/51307/cwd (readlink: Permission denied) 2024-03-07T07:43:30.664 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 51307 safe_time ceph rtd unknown /proc/32200/task/51307/root (readlink: Permission denied) 2024-03-07T07:43:30.665 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 51307 safe_time ceph txt unknown /proc/32200/task/51307/exe (readlink: Permission denied) 2024-03-07T07:43:30.665 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 51307 safe_time ceph NOFD /proc/32200/task/51307/fd (opendir: Permission denied) 2024-03-07T07:43:30.665 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 51308 safe_time ceph cwd unknown /proc/32200/task/51308/cwd (readlink: Permission denied) 2024-03-07T07:43:30.665 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 51308 safe_time ceph rtd unknown /proc/32200/task/51308/root (readlink: Permission denied) 2024-03-07T07:43:30.665 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 51308 safe_time ceph txt unknown /proc/32200/task/51308/exe (readlink: Permission denied) 2024-03-07T07:43:30.665 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mgr 32200 51308 safe_time ceph NOFD /proc/32200/task/51308/fd (opendir: Permission denied) 2024-03-07T07:43:30.665 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/7 32312 root cwd unknown /proc/32312/cwd (readlink: Permission denied) 2024-03-07T07:43:30.665 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/7 32312 root rtd unknown /proc/32312/root (readlink: Permission denied) 2024-03-07T07:43:30.665 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/7 32312 root txt unknown /proc/32312/exe (readlink: Permission denied) 2024-03-07T07:43:30.665 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/7 32312 root NOFD /proc/32312/fd (opendir: Permission denied) 2024-03-07T07:43:30.665 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 33346 root cwd unknown /proc/33346/cwd (readlink: Permission denied) 2024-03-07T07:43:30.665 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 33346 root rtd unknown /proc/33346/root (readlink: Permission denied) 2024-03-07T07:43:30.665 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 33346 root txt unknown /proc/33346/exe (readlink: Permission denied) 2024-03-07T07:43:30.666 INFO:tasks.workunit.client.0.smithi160.stdout:systemd 33346 root NOFD /proc/33346/fd (opendir: Permission denied) 2024-03-07T07:43:30.666 INFO:tasks.workunit.client.0.smithi160.stdout:(sd-pam) 33349 root cwd unknown /proc/33349/cwd (readlink: Permission denied) 2024-03-07T07:43:30.666 INFO:tasks.workunit.client.0.smithi160.stdout:(sd-pam) 33349 root rtd unknown /proc/33349/root (readlink: Permission denied) 2024-03-07T07:43:30.666 INFO:tasks.workunit.client.0.smithi160.stdout:(sd-pam) 33349 root txt unknown /proc/33349/exe (readlink: Permission denied) 2024-03-07T07:43:30.666 INFO:tasks.workunit.client.0.smithi160.stdout:(sd-pam) 33349 root NOFD /proc/33349/fd (opendir: Permission denied) 2024-03-07T07:43:30.666 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/3 34749 root cwd unknown /proc/34749/cwd (readlink: Permission denied) 2024-03-07T07:43:30.666 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/3 34749 root rtd unknown /proc/34749/root (readlink: Permission denied) 2024-03-07T07:43:30.666 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/3 34749 root txt unknown /proc/34749/exe (readlink: Permission denied) 2024-03-07T07:43:30.666 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/3 34749 root NOFD /proc/34749/fd (opendir: Permission denied) 2024-03-07T07:43:30.666 INFO:tasks.workunit.client.0.smithi160.stdout:sshd 35029 root cwd unknown /proc/35029/cwd (readlink: Permission denied) 2024-03-07T07:43:30.666 INFO:tasks.workunit.client.0.smithi160.stdout:sshd 35029 root rtd unknown /proc/35029/root (readlink: Permission denied) 2024-03-07T07:43:30.666 INFO:tasks.workunit.client.0.smithi160.stdout:sshd 35029 root txt unknown /proc/35029/exe (readlink: Permission denied) 2024-03-07T07:43:30.666 INFO:tasks.workunit.client.0.smithi160.stdout:sshd 35029 root NOFD /proc/35029/fd (opendir: Permission denied) 2024-03-07T07:43:30.667 INFO:tasks.workunit.client.0.smithi160.stdout:sshd 35053 root cwd unknown /proc/35053/cwd (readlink: Permission denied) 2024-03-07T07:43:30.667 INFO:tasks.workunit.client.0.smithi160.stdout:sshd 35053 root rtd unknown /proc/35053/root (readlink: Permission denied) 2024-03-07T07:43:30.667 INFO:tasks.workunit.client.0.smithi160.stdout:sshd 35053 root txt unknown /proc/35053/exe (readlink: Permission denied) 2024-03-07T07:43:30.667 INFO:tasks.workunit.client.0.smithi160.stdout:sshd 35053 root NOFD /proc/35053/fd (opendir: Permission denied) 2024-03-07T07:43:30.667 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/4 43610 root cwd unknown /proc/43610/cwd (readlink: Permission denied) 2024-03-07T07:43:30.667 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/4 43610 root rtd unknown /proc/43610/root (readlink: Permission denied) 2024-03-07T07:43:30.667 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/4 43610 root txt unknown /proc/43610/exe (readlink: Permission denied) 2024-03-07T07:43:30.667 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/4 43610 root NOFD /proc/43610/fd (opendir: Permission denied) 2024-03-07T07:43:30.667 INFO:tasks.workunit.client.0.smithi160.stdout:conmon 46912 root cwd unknown /proc/46912/cwd (readlink: Permission denied) 2024-03-07T07:43:30.667 INFO:tasks.workunit.client.0.smithi160.stdout:conmon 46912 root rtd unknown /proc/46912/root (readlink: Permission denied) 2024-03-07T07:43:30.667 INFO:tasks.workunit.client.0.smithi160.stdout:conmon 46912 root txt unknown /proc/46912/exe (readlink: Permission denied) 2024-03-07T07:43:30.667 INFO:tasks.workunit.client.0.smithi160.stdout:conmon 46912 root NOFD /proc/46912/fd (opendir: Permission denied) 2024-03-07T07:43:30.667 INFO:tasks.workunit.client.0.smithi160.stdout:podman-in 46914 root cwd unknown /proc/46914/cwd (readlink: Permission denied) 2024-03-07T07:43:30.667 INFO:tasks.workunit.client.0.smithi160.stdout:podman-in 46914 root rtd unknown /proc/46914/root (readlink: Permission denied) 2024-03-07T07:43:30.667 INFO:tasks.workunit.client.0.smithi160.stdout:podman-in 46914 root txt unknown /proc/46914/exe (readlink: Permission denied) 2024-03-07T07:43:30.668 INFO:tasks.workunit.client.0.smithi160.stdout:podman-in 46914 root NOFD /proc/46914/fd (opendir: Permission denied) 2024-03-07T07:43:30.668 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 ceph cwd unknown /proc/46916/cwd (readlink: Permission denied) 2024-03-07T07:43:30.668 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 ceph rtd unknown /proc/46916/root (readlink: Permission denied) 2024-03-07T07:43:30.668 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 ceph txt unknown /proc/46916/exe (readlink: Permission denied) 2024-03-07T07:43:30.668 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 ceph NOFD /proc/46916/fd (opendir: Permission denied) 2024-03-07T07:43:30.668 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 46917 log ceph cwd unknown /proc/46916/task/46917/cwd (readlink: Permission denied) 2024-03-07T07:43:30.668 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 46917 log ceph rtd unknown /proc/46916/task/46917/root (readlink: Permission denied) 2024-03-07T07:43:30.668 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 46917 log ceph txt unknown /proc/46916/task/46917/exe (readlink: Permission denied) 2024-03-07T07:43:30.668 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 46917 log ceph NOFD /proc/46916/task/46917/fd (opendir: Permission denied) 2024-03-07T07:43:30.668 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 46919 msgr-work ceph cwd unknown /proc/46916/task/46919/cwd (readlink: Permission denied) 2024-03-07T07:43:30.668 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 46919 msgr-work ceph rtd unknown /proc/46916/task/46919/root (readlink: Permission denied) 2024-03-07T07:43:30.668 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 46919 msgr-work ceph txt unknown /proc/46916/task/46919/exe (readlink: Permission denied) 2024-03-07T07:43:30.668 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 46919 msgr-work ceph NOFD /proc/46916/task/46919/fd (opendir: Permission denied) 2024-03-07T07:43:30.669 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 46920 msgr-work ceph cwd unknown /proc/46916/task/46920/cwd (readlink: Permission denied) 2024-03-07T07:43:30.669 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 46920 msgr-work ceph rtd unknown /proc/46916/task/46920/root (readlink: Permission denied) 2024-03-07T07:43:30.669 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 46920 msgr-work ceph txt unknown /proc/46916/task/46920/exe (readlink: Permission denied) 2024-03-07T07:43:30.669 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 46920 msgr-work ceph NOFD /proc/46916/task/46920/fd (opendir: Permission denied) 2024-03-07T07:43:30.669 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 46921 msgr-work ceph cwd unknown /proc/46916/task/46921/cwd (readlink: Permission denied) 2024-03-07T07:43:30.669 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 46921 msgr-work ceph rtd unknown /proc/46916/task/46921/root (readlink: Permission denied) 2024-03-07T07:43:30.669 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 46921 msgr-work ceph txt unknown /proc/46916/task/46921/exe (readlink: Permission denied) 2024-03-07T07:43:30.669 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 46921 msgr-work ceph NOFD /proc/46916/task/46921/fd (opendir: Permission denied) 2024-03-07T07:43:30.669 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 46925 service ceph cwd unknown /proc/46916/task/46925/cwd (readlink: Permission denied) 2024-03-07T07:43:30.669 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 46925 service ceph rtd unknown /proc/46916/task/46925/root (readlink: Permission denied) 2024-03-07T07:43:30.669 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 46925 service ceph txt unknown /proc/46916/task/46925/exe (readlink: Permission denied) 2024-03-07T07:43:30.669 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 46925 service ceph NOFD /proc/46916/task/46925/fd (opendir: Permission denied) 2024-03-07T07:43:30.669 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 46926 admin_soc ceph cwd unknown /proc/46916/task/46926/cwd (readlink: Permission denied) 2024-03-07T07:43:30.669 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 46926 admin_soc ceph rtd unknown /proc/46916/task/46926/root (readlink: Permission denied) 2024-03-07T07:43:30.669 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 46926 admin_soc ceph txt unknown /proc/46916/task/46926/exe (readlink: Permission denied) 2024-03-07T07:43:30.670 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 46926 admin_soc ceph NOFD /proc/46916/task/46926/fd (opendir: Permission denied) 2024-03-07T07:43:30.670 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47068 signal_ha ceph cwd unknown /proc/46916/task/47068/cwd (readlink: Permission denied) 2024-03-07T07:43:30.670 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47068 signal_ha ceph rtd unknown /proc/46916/task/47068/root (readlink: Permission denied) 2024-03-07T07:43:30.670 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47068 signal_ha ceph txt unknown /proc/46916/task/47068/exe (readlink: Permission denied) 2024-03-07T07:43:30.670 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47068 signal_ha ceph NOFD /proc/46916/task/47068/fd (opendir: Permission denied) 2024-03-07T07:43:30.670 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47069 io_contex ceph cwd unknown /proc/46916/task/47069/cwd (readlink: Permission denied) 2024-03-07T07:43:30.670 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47069 io_contex ceph rtd unknown /proc/46916/task/47069/root (readlink: Permission denied) 2024-03-07T07:43:30.670 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47069 io_contex ceph txt unknown /proc/46916/task/47069/exe (readlink: Permission denied) 2024-03-07T07:43:30.670 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47069 io_contex ceph NOFD /proc/46916/task/47069/fd (opendir: Permission denied) 2024-03-07T07:43:30.670 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47070 io_contex ceph cwd unknown /proc/46916/task/47070/cwd (readlink: Permission denied) 2024-03-07T07:43:30.670 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47070 io_contex ceph rtd unknown /proc/46916/task/47070/root (readlink: Permission denied) 2024-03-07T07:43:30.670 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47070 io_contex ceph txt unknown /proc/46916/task/47070/exe (readlink: Permission denied) 2024-03-07T07:43:30.670 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47070 io_contex ceph NOFD /proc/46916/task/47070/fd (opendir: Permission denied) 2024-03-07T07:43:30.671 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47182 OpHistory ceph cwd unknown /proc/46916/task/47182/cwd (readlink: Permission denied) 2024-03-07T07:43:30.671 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47182 OpHistory ceph rtd unknown /proc/46916/task/47182/root (readlink: Permission denied) 2024-03-07T07:43:30.671 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47182 OpHistory ceph txt unknown /proc/46916/task/47182/exe (readlink: Permission denied) 2024-03-07T07:43:30.671 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47182 OpHistory ceph NOFD /proc/46916/task/47182/fd (opendir: Permission denied) 2024-03-07T07:43:30.671 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47183 ceph_time ceph cwd unknown /proc/46916/task/47183/cwd (readlink: Permission denied) 2024-03-07T07:43:30.671 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47183 ceph_time ceph rtd unknown /proc/46916/task/47183/root (readlink: Permission denied) 2024-03-07T07:43:30.671 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47183 ceph_time ceph txt unknown /proc/46916/task/47183/exe (readlink: Permission denied) 2024-03-07T07:43:30.671 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47183 ceph_time ceph NOFD /proc/46916/task/47183/fd (opendir: Permission denied) 2024-03-07T07:43:30.671 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47344 ceph-osd ceph cwd unknown /proc/46916/task/47344/cwd (readlink: Permission denied) 2024-03-07T07:43:30.671 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47344 ceph-osd ceph rtd unknown /proc/46916/task/47344/root (readlink: Permission denied) 2024-03-07T07:43:30.671 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47344 ceph-osd ceph txt unknown /proc/46916/task/47344/exe (readlink: Permission denied) 2024-03-07T07:43:30.671 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47344 ceph-osd ceph NOFD /proc/46916/task/47344/fd (opendir: Permission denied) 2024-03-07T07:43:30.671 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47345 safe_time ceph cwd unknown /proc/46916/task/47345/cwd (readlink: Permission denied) 2024-03-07T07:43:30.671 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47345 safe_time ceph rtd unknown /proc/46916/task/47345/root (readlink: Permission denied) 2024-03-07T07:43:30.671 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47345 safe_time ceph txt unknown /proc/46916/task/47345/exe (readlink: Permission denied) 2024-03-07T07:43:30.672 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47345 safe_time ceph NOFD /proc/46916/task/47345/fd (opendir: Permission denied) 2024-03-07T07:43:30.672 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47346 safe_time ceph cwd unknown /proc/46916/task/47346/cwd (readlink: Permission denied) 2024-03-07T07:43:30.672 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47346 safe_time ceph rtd unknown /proc/46916/task/47346/root (readlink: Permission denied) 2024-03-07T07:43:30.672 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47346 safe_time ceph txt unknown /proc/46916/task/47346/exe (readlink: Permission denied) 2024-03-07T07:43:30.672 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47346 safe_time ceph NOFD /proc/46916/task/47346/fd (opendir: Permission denied) 2024-03-07T07:43:30.672 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47347 safe_time ceph cwd unknown /proc/46916/task/47347/cwd (readlink: Permission denied) 2024-03-07T07:43:30.672 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47347 safe_time ceph rtd unknown /proc/46916/task/47347/root (readlink: Permission denied) 2024-03-07T07:43:30.672 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47347 safe_time ceph txt unknown /proc/46916/task/47347/exe (readlink: Permission denied) 2024-03-07T07:43:30.672 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47347 safe_time ceph NOFD /proc/46916/task/47347/fd (opendir: Permission denied) 2024-03-07T07:43:30.672 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47348 safe_time ceph cwd unknown /proc/46916/task/47348/cwd (readlink: Permission denied) 2024-03-07T07:43:30.672 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47348 safe_time ceph rtd unknown /proc/46916/task/47348/root (readlink: Permission denied) 2024-03-07T07:43:30.672 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47348 safe_time ceph txt unknown /proc/46916/task/47348/exe (readlink: Permission denied) 2024-03-07T07:43:30.672 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47348 safe_time ceph NOFD /proc/46916/task/47348/fd (opendir: Permission denied) 2024-03-07T07:43:30.672 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47349 fn_anonym ceph cwd unknown /proc/46916/task/47349/cwd (readlink: Permission denied) 2024-03-07T07:43:30.673 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47349 fn_anonym ceph rtd unknown /proc/46916/task/47349/root (readlink: Permission denied) 2024-03-07T07:43:30.673 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47349 fn_anonym ceph txt unknown /proc/46916/task/47349/exe (readlink: Permission denied) 2024-03-07T07:43:30.673 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47349 fn_anonym ceph NOFD /proc/46916/task/47349/fd (opendir: Permission denied) 2024-03-07T07:43:30.673 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47354 rocksdb:l ceph cwd unknown /proc/46916/task/47354/cwd (readlink: Permission denied) 2024-03-07T07:43:30.673 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47354 rocksdb:l ceph rtd unknown /proc/46916/task/47354/root (readlink: Permission denied) 2024-03-07T07:43:30.673 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47354 rocksdb:l ceph txt unknown /proc/46916/task/47354/exe (readlink: Permission denied) 2024-03-07T07:43:30.673 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47354 rocksdb:l ceph NOFD /proc/46916/task/47354/fd (opendir: Permission denied) 2024-03-07T07:43:30.673 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47356 rocksdb:l ceph cwd unknown /proc/46916/task/47356/cwd (readlink: Permission denied) 2024-03-07T07:43:30.673 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47356 rocksdb:l ceph rtd unknown /proc/46916/task/47356/root (readlink: Permission denied) 2024-03-07T07:43:30.673 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47356 rocksdb:l ceph txt unknown /proc/46916/task/47356/exe (readlink: Permission denied) 2024-03-07T07:43:30.673 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47356 rocksdb:l ceph NOFD /proc/46916/task/47356/fd (opendir: Permission denied) 2024-03-07T07:43:30.673 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47357 rocksdb:l ceph cwd unknown /proc/46916/task/47357/cwd (readlink: Permission denied) 2024-03-07T07:43:30.673 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47357 rocksdb:l ceph rtd unknown /proc/46916/task/47357/root (readlink: Permission denied) 2024-03-07T07:43:30.673 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47357 rocksdb:l ceph txt unknown /proc/46916/task/47357/exe (readlink: Permission denied) 2024-03-07T07:43:30.674 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47357 rocksdb:l ceph NOFD /proc/46916/task/47357/fd (opendir: Permission denied) 2024-03-07T07:43:30.674 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47358 rocksdb:h ceph cwd unknown /proc/46916/task/47358/cwd (readlink: Permission denied) 2024-03-07T07:43:30.674 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47358 rocksdb:h ceph rtd unknown /proc/46916/task/47358/root (readlink: Permission denied) 2024-03-07T07:43:30.674 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47358 rocksdb:h ceph txt unknown /proc/46916/task/47358/exe (readlink: Permission denied) 2024-03-07T07:43:30.674 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47358 rocksdb:h ceph NOFD /proc/46916/task/47358/fd (opendir: Permission denied) 2024-03-07T07:43:30.674 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47815 bstore_ai ceph cwd unknown /proc/46916/task/47815/cwd (readlink: Permission denied) 2024-03-07T07:43:30.674 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47815 bstore_ai ceph rtd unknown /proc/46916/task/47815/root (readlink: Permission denied) 2024-03-07T07:43:30.674 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47815 bstore_ai ceph txt unknown /proc/46916/task/47815/exe (readlink: Permission denied) 2024-03-07T07:43:30.674 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 47815 bstore_ai ceph NOFD /proc/46916/task/47815/fd (opendir: Permission denied) 2024-03-07T07:43:30.674 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48024 bstore_ai ceph cwd unknown /proc/46916/task/48024/cwd (readlink: Permission denied) 2024-03-07T07:43:30.674 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48024 bstore_ai ceph rtd unknown /proc/46916/task/48024/root (readlink: Permission denied) 2024-03-07T07:43:30.674 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48024 bstore_ai ceph txt unknown /proc/46916/task/48024/exe (readlink: Permission denied) 2024-03-07T07:43:30.674 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48024 bstore_ai ceph NOFD /proc/46916/task/48024/fd (opendir: Permission denied) 2024-03-07T07:43:30.674 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48206 ceph-osd ceph cwd unknown /proc/46916/task/48206/cwd (readlink: Permission denied) 2024-03-07T07:43:30.675 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48206 ceph-osd ceph rtd unknown /proc/46916/task/48206/root (readlink: Permission denied) 2024-03-07T07:43:30.675 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48206 ceph-osd ceph txt unknown /proc/46916/task/48206/exe (readlink: Permission denied) 2024-03-07T07:43:30.675 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48206 ceph-osd ceph NOFD /proc/46916/task/48206/fd (opendir: Permission denied) 2024-03-07T07:43:30.675 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48207 cfin ceph cwd unknown /proc/46916/task/48207/cwd (readlink: Permission denied) 2024-03-07T07:43:30.675 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48207 cfin ceph rtd unknown /proc/46916/task/48207/root (readlink: Permission denied) 2024-03-07T07:43:30.675 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48207 cfin ceph txt unknown /proc/46916/task/48207/exe (readlink: Permission denied) 2024-03-07T07:43:30.675 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48207 cfin ceph NOFD /proc/46916/task/48207/fd (opendir: Permission denied) 2024-03-07T07:43:30.675 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48208 bstore_kv ceph cwd unknown /proc/46916/task/48208/cwd (readlink: Permission denied) 2024-03-07T07:43:30.675 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48208 bstore_kv ceph rtd unknown /proc/46916/task/48208/root (readlink: Permission denied) 2024-03-07T07:43:30.675 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48208 bstore_kv ceph txt unknown /proc/46916/task/48208/exe (readlink: Permission denied) 2024-03-07T07:43:30.675 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48208 bstore_kv ceph NOFD /proc/46916/task/48208/fd (opendir: Permission denied) 2024-03-07T07:43:30.675 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48209 bstore_kv ceph cwd unknown /proc/46916/task/48209/cwd (readlink: Permission denied) 2024-03-07T07:43:30.675 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48209 bstore_kv ceph rtd unknown /proc/46916/task/48209/root (readlink: Permission denied) 2024-03-07T07:43:30.675 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48209 bstore_kv ceph txt unknown /proc/46916/task/48209/exe (readlink: Permission denied) 2024-03-07T07:43:30.675 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48209 bstore_kv ceph NOFD /proc/46916/task/48209/fd (opendir: Permission denied) 2024-03-07T07:43:30.676 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48210 bstore_me ceph cwd unknown /proc/46916/task/48210/cwd (readlink: Permission denied) 2024-03-07T07:43:30.676 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48210 bstore_me ceph rtd unknown /proc/46916/task/48210/root (readlink: Permission denied) 2024-03-07T07:43:30.676 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48210 bstore_me ceph txt unknown /proc/46916/task/48210/exe (readlink: Permission denied) 2024-03-07T07:43:30.676 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48210 bstore_me ceph NOFD /proc/46916/task/48210/fd (opendir: Permission denied) 2024-03-07T07:43:30.676 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48211 ms_dispat ceph cwd unknown /proc/46916/task/48211/cwd (readlink: Permission denied) 2024-03-07T07:43:30.676 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48211 ms_dispat ceph rtd unknown /proc/46916/task/48211/root (readlink: Permission denied) 2024-03-07T07:43:30.676 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48211 ms_dispat ceph txt unknown /proc/46916/task/48211/exe (readlink: Permission denied) 2024-03-07T07:43:30.676 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48211 ms_dispat ceph NOFD /proc/46916/task/48211/fd (opendir: Permission denied) 2024-03-07T07:43:30.676 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48212 ms_local ceph cwd unknown /proc/46916/task/48212/cwd (readlink: Permission denied) 2024-03-07T07:43:30.676 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48212 ms_local ceph rtd unknown /proc/46916/task/48212/root (readlink: Permission denied) 2024-03-07T07:43:30.676 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48212 ms_local ceph txt unknown /proc/46916/task/48212/exe (readlink: Permission denied) 2024-03-07T07:43:30.676 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48212 ms_local ceph NOFD /proc/46916/task/48212/fd (opendir: Permission denied) 2024-03-07T07:43:30.676 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48213 safe_time ceph cwd unknown /proc/46916/task/48213/cwd (readlink: Permission denied) 2024-03-07T07:43:30.676 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48213 safe_time ceph rtd unknown /proc/46916/task/48213/root (readlink: Permission denied) 2024-03-07T07:43:30.677 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48213 safe_time ceph txt unknown /proc/46916/task/48213/exe (readlink: Permission denied) 2024-03-07T07:43:30.677 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48213 safe_time ceph NOFD /proc/46916/task/48213/fd (opendir: Permission denied) 2024-03-07T07:43:30.677 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48214 safe_time ceph cwd unknown /proc/46916/task/48214/cwd (readlink: Permission denied) 2024-03-07T07:43:30.677 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48214 safe_time ceph rtd unknown /proc/46916/task/48214/root (readlink: Permission denied) 2024-03-07T07:43:30.677 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48214 safe_time ceph txt unknown /proc/46916/task/48214/exe (readlink: Permission denied) 2024-03-07T07:43:30.677 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48214 safe_time ceph NOFD /proc/46916/task/48214/fd (opendir: Permission denied) 2024-03-07T07:43:30.677 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48215 ms_dispat ceph cwd unknown /proc/46916/task/48215/cwd (readlink: Permission denied) 2024-03-07T07:43:30.677 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48215 ms_dispat ceph rtd unknown /proc/46916/task/48215/root (readlink: Permission denied) 2024-03-07T07:43:30.677 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48215 ms_dispat ceph txt unknown /proc/46916/task/48215/exe (readlink: Permission denied) 2024-03-07T07:43:30.677 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48215 ms_dispat ceph NOFD /proc/46916/task/48215/fd (opendir: Permission denied) 2024-03-07T07:43:30.677 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48216 ms_local ceph cwd unknown /proc/46916/task/48216/cwd (readlink: Permission denied) 2024-03-07T07:43:30.677 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48216 ms_local ceph rtd unknown /proc/46916/task/48216/root (readlink: Permission denied) 2024-03-07T07:43:30.677 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48216 ms_local ceph txt unknown /proc/46916/task/48216/exe (readlink: Permission denied) 2024-03-07T07:43:30.678 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48216 ms_local ceph NOFD /proc/46916/task/48216/fd (opendir: Permission denied) 2024-03-07T07:43:30.678 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48217 ms_dispat ceph cwd unknown /proc/46916/task/48217/cwd (readlink: Permission denied) 2024-03-07T07:43:30.678 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48217 ms_dispat ceph rtd unknown /proc/46916/task/48217/root (readlink: Permission denied) 2024-03-07T07:43:30.678 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48217 ms_dispat ceph txt unknown /proc/46916/task/48217/exe (readlink: Permission denied) 2024-03-07T07:43:30.678 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48217 ms_dispat ceph NOFD /proc/46916/task/48217/fd (opendir: Permission denied) 2024-03-07T07:43:30.678 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48218 ms_local ceph cwd unknown /proc/46916/task/48218/cwd (readlink: Permission denied) 2024-03-07T07:43:30.678 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48218 ms_local ceph rtd unknown /proc/46916/task/48218/root (readlink: Permission denied) 2024-03-07T07:43:30.678 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48218 ms_local ceph txt unknown /proc/46916/task/48218/exe (readlink: Permission denied) 2024-03-07T07:43:30.678 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48218 ms_local ceph NOFD /proc/46916/task/48218/fd (opendir: Permission denied) 2024-03-07T07:43:30.678 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48219 ms_dispat ceph cwd unknown /proc/46916/task/48219/cwd (readlink: Permission denied) 2024-03-07T07:43:30.678 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48219 ms_dispat ceph rtd unknown /proc/46916/task/48219/root (readlink: Permission denied) 2024-03-07T07:43:30.678 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48219 ms_dispat ceph txt unknown /proc/46916/task/48219/exe (readlink: Permission denied) 2024-03-07T07:43:30.678 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48219 ms_dispat ceph NOFD /proc/46916/task/48219/fd (opendir: Permission denied) 2024-03-07T07:43:30.678 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48220 ms_local ceph cwd unknown /proc/46916/task/48220/cwd (readlink: Permission denied) 2024-03-07T07:43:30.679 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48220 ms_local ceph rtd unknown /proc/46916/task/48220/root (readlink: Permission denied) 2024-03-07T07:43:30.679 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48220 ms_local ceph txt unknown /proc/46916/task/48220/exe (readlink: Permission denied) 2024-03-07T07:43:30.679 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48220 ms_local ceph NOFD /proc/46916/task/48220/fd (opendir: Permission denied) 2024-03-07T07:43:30.679 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48221 ms_dispat ceph cwd unknown /proc/46916/task/48221/cwd (readlink: Permission denied) 2024-03-07T07:43:30.679 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48221 ms_dispat ceph rtd unknown /proc/46916/task/48221/root (readlink: Permission denied) 2024-03-07T07:43:30.679 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48221 ms_dispat ceph txt unknown /proc/46916/task/48221/exe (readlink: Permission denied) 2024-03-07T07:43:30.679 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48221 ms_dispat ceph NOFD /proc/46916/task/48221/fd (opendir: Permission denied) 2024-03-07T07:43:30.679 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48222 ms_local ceph cwd unknown /proc/46916/task/48222/cwd (readlink: Permission denied) 2024-03-07T07:43:30.679 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48222 ms_local ceph rtd unknown /proc/46916/task/48222/root (readlink: Permission denied) 2024-03-07T07:43:30.679 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48222 ms_local ceph txt unknown /proc/46916/task/48222/exe (readlink: Permission denied) 2024-03-07T07:43:30.679 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48222 ms_local ceph NOFD /proc/46916/task/48222/fd (opendir: Permission denied) 2024-03-07T07:43:30.679 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48223 ms_dispat ceph cwd unknown /proc/46916/task/48223/cwd (readlink: Permission denied) 2024-03-07T07:43:30.679 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48223 ms_dispat ceph rtd unknown /proc/46916/task/48223/root (readlink: Permission denied) 2024-03-07T07:43:30.679 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48223 ms_dispat ceph txt unknown /proc/46916/task/48223/exe (readlink: Permission denied) 2024-03-07T07:43:30.680 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48223 ms_dispat ceph NOFD /proc/46916/task/48223/fd (opendir: Permission denied) 2024-03-07T07:43:30.680 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48224 ms_local ceph cwd unknown /proc/46916/task/48224/cwd (readlink: Permission denied) 2024-03-07T07:43:30.680 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48224 ms_local ceph rtd unknown /proc/46916/task/48224/root (readlink: Permission denied) 2024-03-07T07:43:30.680 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48224 ms_local ceph txt unknown /proc/46916/task/48224/exe (readlink: Permission denied) 2024-03-07T07:43:30.680 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48224 ms_local ceph NOFD /proc/46916/task/48224/fd (opendir: Permission denied) 2024-03-07T07:43:30.680 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48225 ms_dispat ceph cwd unknown /proc/46916/task/48225/cwd (readlink: Permission denied) 2024-03-07T07:43:30.680 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48225 ms_dispat ceph rtd unknown /proc/46916/task/48225/root (readlink: Permission denied) 2024-03-07T07:43:30.680 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48225 ms_dispat ceph txt unknown /proc/46916/task/48225/exe (readlink: Permission denied) 2024-03-07T07:43:30.680 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48225 ms_dispat ceph NOFD /proc/46916/task/48225/fd (opendir: Permission denied) 2024-03-07T07:43:30.680 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48226 ms_local ceph cwd unknown /proc/46916/task/48226/cwd (readlink: Permission denied) 2024-03-07T07:43:30.680 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48226 ms_local ceph rtd unknown /proc/46916/task/48226/root (readlink: Permission denied) 2024-03-07T07:43:30.680 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48226 ms_local ceph txt unknown /proc/46916/task/48226/exe (readlink: Permission denied) 2024-03-07T07:43:30.680 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48226 ms_local ceph NOFD /proc/46916/task/48226/fd (opendir: Permission denied) 2024-03-07T07:43:30.680 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48227 fn_anonym ceph cwd unknown /proc/46916/task/48227/cwd (readlink: Permission denied) 2024-03-07T07:43:30.681 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48227 fn_anonym ceph rtd unknown /proc/46916/task/48227/root (readlink: Permission denied) 2024-03-07T07:43:30.681 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48227 fn_anonym ceph txt unknown /proc/46916/task/48227/exe (readlink: Permission denied) 2024-03-07T07:43:30.681 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48227 fn_anonym ceph NOFD /proc/46916/task/48227/fd (opendir: Permission denied) 2024-03-07T07:43:30.681 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48228 finisher ceph cwd unknown /proc/46916/task/48228/cwd (readlink: Permission denied) 2024-03-07T07:43:30.681 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48228 finisher ceph rtd unknown /proc/46916/task/48228/root (readlink: Permission denied) 2024-03-07T07:43:30.681 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48228 finisher ceph txt unknown /proc/46916/task/48228/exe (readlink: Permission denied) 2024-03-07T07:43:30.681 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48228 finisher ceph NOFD /proc/46916/task/48228/fd (opendir: Permission denied) 2024-03-07T07:43:30.681 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48229 safe_time ceph cwd unknown /proc/46916/task/48229/cwd (readlink: Permission denied) 2024-03-07T07:43:30.681 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48229 safe_time ceph rtd unknown /proc/46916/task/48229/root (readlink: Permission denied) 2024-03-07T07:43:30.681 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48229 safe_time ceph txt unknown /proc/46916/task/48229/exe (readlink: Permission denied) 2024-03-07T07:43:30.681 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48229 safe_time ceph NOFD /proc/46916/task/48229/fd (opendir: Permission denied) 2024-03-07T07:43:30.681 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48230 safe_time ceph cwd unknown /proc/46916/task/48230/cwd (readlink: Permission denied) 2024-03-07T07:43:30.681 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48230 safe_time ceph rtd unknown /proc/46916/task/48230/root (readlink: Permission denied) 2024-03-07T07:43:30.681 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48230 safe_time ceph txt unknown /proc/46916/task/48230/exe (readlink: Permission denied) 2024-03-07T07:43:30.682 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48230 safe_time ceph NOFD /proc/46916/task/48230/fd (opendir: Permission denied) 2024-03-07T07:43:30.682 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48231 ceph-osd ceph cwd unknown /proc/46916/task/48231/cwd (readlink: Permission denied) 2024-03-07T07:43:30.682 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48231 ceph-osd ceph rtd unknown /proc/46916/task/48231/root (readlink: Permission denied) 2024-03-07T07:43:30.682 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48231 ceph-osd ceph txt unknown /proc/46916/task/48231/exe (readlink: Permission denied) 2024-03-07T07:43:30.682 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48231 ceph-osd ceph NOFD /proc/46916/task/48231/fd (opendir: Permission denied) 2024-03-07T07:43:30.682 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48232 osd_srv_a ceph cwd unknown /proc/46916/task/48232/cwd (readlink: Permission denied) 2024-03-07T07:43:30.682 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48232 osd_srv_a ceph rtd unknown /proc/46916/task/48232/root (readlink: Permission denied) 2024-03-07T07:43:30.682 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48232 osd_srv_a ceph txt unknown /proc/46916/task/48232/exe (readlink: Permission denied) 2024-03-07T07:43:30.682 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48232 osd_srv_a ceph NOFD /proc/46916/task/48232/fd (opendir: Permission denied) 2024-03-07T07:43:30.682 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48233 tp_osd_tp ceph cwd unknown /proc/46916/task/48233/cwd (readlink: Permission denied) 2024-03-07T07:43:30.682 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48233 tp_osd_tp ceph rtd unknown /proc/46916/task/48233/root (readlink: Permission denied) 2024-03-07T07:43:30.682 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48233 tp_osd_tp ceph txt unknown /proc/46916/task/48233/exe (readlink: Permission denied) 2024-03-07T07:43:30.682 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48233 tp_osd_tp ceph NOFD /proc/46916/task/48233/fd (opendir: Permission denied) 2024-03-07T07:43:30.683 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48234 tp_osd_tp ceph cwd unknown /proc/46916/task/48234/cwd (readlink: Permission denied) 2024-03-07T07:43:30.683 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48234 tp_osd_tp ceph rtd unknown /proc/46916/task/48234/root (readlink: Permission denied) 2024-03-07T07:43:30.683 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48234 tp_osd_tp ceph txt unknown /proc/46916/task/48234/exe (readlink: Permission denied) 2024-03-07T07:43:30.683 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48234 tp_osd_tp ceph NOFD /proc/46916/task/48234/fd (opendir: Permission denied) 2024-03-07T07:43:30.683 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48235 tp_osd_tp ceph cwd unknown /proc/46916/task/48235/cwd (readlink: Permission denied) 2024-03-07T07:43:30.683 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48235 tp_osd_tp ceph rtd unknown /proc/46916/task/48235/root (readlink: Permission denied) 2024-03-07T07:43:30.683 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48235 tp_osd_tp ceph txt unknown /proc/46916/task/48235/exe (readlink: Permission denied) 2024-03-07T07:43:30.683 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48235 tp_osd_tp ceph NOFD /proc/46916/task/48235/fd (opendir: Permission denied) 2024-03-07T07:43:30.683 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48236 tp_osd_tp ceph cwd unknown /proc/46916/task/48236/cwd (readlink: Permission denied) 2024-03-07T07:43:30.683 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48236 tp_osd_tp ceph rtd unknown /proc/46916/task/48236/root (readlink: Permission denied) 2024-03-07T07:43:30.683 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48236 tp_osd_tp ceph txt unknown /proc/46916/task/48236/exe (readlink: Permission denied) 2024-03-07T07:43:30.683 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48236 tp_osd_tp ceph NOFD /proc/46916/task/48236/fd (opendir: Permission denied) 2024-03-07T07:43:30.683 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48237 tp_osd_tp ceph cwd unknown /proc/46916/task/48237/cwd (readlink: Permission denied) 2024-03-07T07:43:30.683 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48237 tp_osd_tp ceph rtd unknown /proc/46916/task/48237/root (readlink: Permission denied) 2024-03-07T07:43:30.683 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48237 tp_osd_tp ceph txt unknown /proc/46916/task/48237/exe (readlink: Permission denied) 2024-03-07T07:43:30.684 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48237 tp_osd_tp ceph NOFD /proc/46916/task/48237/fd (opendir: Permission denied) 2024-03-07T07:43:30.684 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48238 tp_osd_tp ceph cwd unknown /proc/46916/task/48238/cwd (readlink: Permission denied) 2024-03-07T07:43:30.684 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48238 tp_osd_tp ceph rtd unknown /proc/46916/task/48238/root (readlink: Permission denied) 2024-03-07T07:43:30.684 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48238 tp_osd_tp ceph txt unknown /proc/46916/task/48238/exe (readlink: Permission denied) 2024-03-07T07:43:30.684 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48238 tp_osd_tp ceph NOFD /proc/46916/task/48238/fd (opendir: Permission denied) 2024-03-07T07:43:30.684 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48239 tp_osd_tp ceph cwd unknown /proc/46916/task/48239/cwd (readlink: Permission denied) 2024-03-07T07:43:30.684 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48239 tp_osd_tp ceph rtd unknown /proc/46916/task/48239/root (readlink: Permission denied) 2024-03-07T07:43:30.684 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48239 tp_osd_tp ceph txt unknown /proc/46916/task/48239/exe (readlink: Permission denied) 2024-03-07T07:43:30.684 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48239 tp_osd_tp ceph NOFD /proc/46916/task/48239/fd (opendir: Permission denied) 2024-03-07T07:43:30.684 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48240 tp_osd_tp ceph cwd unknown /proc/46916/task/48240/cwd (readlink: Permission denied) 2024-03-07T07:43:30.684 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48240 tp_osd_tp ceph rtd unknown /proc/46916/task/48240/root (readlink: Permission denied) 2024-03-07T07:43:30.684 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48240 tp_osd_tp ceph txt unknown /proc/46916/task/48240/exe (readlink: Permission denied) 2024-03-07T07:43:30.684 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48240 tp_osd_tp ceph NOFD /proc/46916/task/48240/fd (opendir: Permission denied) 2024-03-07T07:43:30.685 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48241 tp_osd_tp ceph cwd unknown /proc/46916/task/48241/cwd (readlink: Permission denied) 2024-03-07T07:43:30.685 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48241 tp_osd_tp ceph rtd unknown /proc/46916/task/48241/root (readlink: Permission denied) 2024-03-07T07:43:30.685 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48241 tp_osd_tp ceph txt unknown /proc/46916/task/48241/exe (readlink: Permission denied) 2024-03-07T07:43:30.685 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48241 tp_osd_tp ceph NOFD /proc/46916/task/48241/fd (opendir: Permission denied) 2024-03-07T07:43:30.685 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48242 tp_osd_tp ceph cwd unknown /proc/46916/task/48242/cwd (readlink: Permission denied) 2024-03-07T07:43:30.685 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48242 tp_osd_tp ceph rtd unknown /proc/46916/task/48242/root (readlink: Permission denied) 2024-03-07T07:43:30.685 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48242 tp_osd_tp ceph txt unknown /proc/46916/task/48242/exe (readlink: Permission denied) 2024-03-07T07:43:30.685 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48242 tp_osd_tp ceph NOFD /proc/46916/task/48242/fd (opendir: Permission denied) 2024-03-07T07:43:30.685 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48243 tp_osd_tp ceph cwd unknown /proc/46916/task/48243/cwd (readlink: Permission denied) 2024-03-07T07:43:30.685 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48243 tp_osd_tp ceph rtd unknown /proc/46916/task/48243/root (readlink: Permission denied) 2024-03-07T07:43:30.685 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48243 tp_osd_tp ceph txt unknown /proc/46916/task/48243/exe (readlink: Permission denied) 2024-03-07T07:43:30.685 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48243 tp_osd_tp ceph NOFD /proc/46916/task/48243/fd (opendir: Permission denied) 2024-03-07T07:43:30.685 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48244 tp_osd_tp ceph cwd unknown /proc/46916/task/48244/cwd (readlink: Permission denied) 2024-03-07T07:43:30.686 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48244 tp_osd_tp ceph rtd unknown /proc/46916/task/48244/root (readlink: Permission denied) 2024-03-07T07:43:30.686 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48244 tp_osd_tp ceph txt unknown /proc/46916/task/48244/exe (readlink: Permission denied) 2024-03-07T07:43:30.686 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48244 tp_osd_tp ceph NOFD /proc/46916/task/48244/fd (opendir: Permission denied) 2024-03-07T07:43:30.686 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48245 tp_osd_tp ceph cwd unknown /proc/46916/task/48245/cwd (readlink: Permission denied) 2024-03-07T07:43:30.686 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48245 tp_osd_tp ceph rtd unknown /proc/46916/task/48245/root (readlink: Permission denied) 2024-03-07T07:43:30.686 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48245 tp_osd_tp ceph txt unknown /proc/46916/task/48245/exe (readlink: Permission denied) 2024-03-07T07:43:30.686 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48245 tp_osd_tp ceph NOFD /proc/46916/task/48245/fd (opendir: Permission denied) 2024-03-07T07:43:30.686 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48246 tp_osd_tp ceph cwd unknown /proc/46916/task/48246/cwd (readlink: Permission denied) 2024-03-07T07:43:30.686 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48246 tp_osd_tp ceph rtd unknown /proc/46916/task/48246/root (readlink: Permission denied) 2024-03-07T07:43:30.686 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48246 tp_osd_tp ceph txt unknown /proc/46916/task/48246/exe (readlink: Permission denied) 2024-03-07T07:43:30.686 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48246 tp_osd_tp ceph NOFD /proc/46916/task/48246/fd (opendir: Permission denied) 2024-03-07T07:43:30.686 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48247 tp_osd_tp ceph cwd unknown /proc/46916/task/48247/cwd (readlink: Permission denied) 2024-03-07T07:43:30.686 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48247 tp_osd_tp ceph rtd unknown /proc/46916/task/48247/root (readlink: Permission denied) 2024-03-07T07:43:30.686 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48247 tp_osd_tp ceph txt unknown /proc/46916/task/48247/exe (readlink: Permission denied) 2024-03-07T07:43:30.687 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48247 tp_osd_tp ceph NOFD /proc/46916/task/48247/fd (opendir: Permission denied) 2024-03-07T07:43:30.687 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48248 tp_osd_tp ceph cwd unknown /proc/46916/task/48248/cwd (readlink: Permission denied) 2024-03-07T07:43:30.687 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48248 tp_osd_tp ceph rtd unknown /proc/46916/task/48248/root (readlink: Permission denied) 2024-03-07T07:43:30.687 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48248 tp_osd_tp ceph txt unknown /proc/46916/task/48248/exe (readlink: Permission denied) 2024-03-07T07:43:30.687 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48248 tp_osd_tp ceph NOFD /proc/46916/task/48248/fd (opendir: Permission denied) 2024-03-07T07:43:30.687 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48249 osd_srv_h ceph cwd unknown /proc/46916/task/48249/cwd (readlink: Permission denied) 2024-03-07T07:43:30.687 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48249 osd_srv_h ceph rtd unknown /proc/46916/task/48249/root (readlink: Permission denied) 2024-03-07T07:43:30.687 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48249 osd_srv_h ceph txt unknown /proc/46916/task/48249/exe (readlink: Permission denied) 2024-03-07T07:43:30.687 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 46916 48249 osd_srv_h ceph NOFD /proc/46916/task/48249/fd (opendir: Permission denied) 2024-03-07T07:43:30.687 INFO:tasks.workunit.client.0.smithi160.stdout:sudo 48291 root cwd unknown /proc/48291/cwd (readlink: Permission denied) 2024-03-07T07:43:30.687 INFO:tasks.workunit.client.0.smithi160.stdout:sudo 48291 root rtd unknown /proc/48291/root (readlink: Permission denied) 2024-03-07T07:43:30.687 INFO:tasks.workunit.client.0.smithi160.stdout:sudo 48291 root txt unknown /proc/48291/exe (readlink: Permission denied) 2024-03-07T07:43:30.687 INFO:tasks.workunit.client.0.smithi160.stdout:sudo 48291 root NOFD /proc/48291/fd (opendir: Permission denied) 2024-03-07T07:43:30.687 INFO:tasks.workunit.client.0.smithi160.stdout:journalct 48320 root cwd unknown /proc/48320/cwd (readlink: Permission denied) 2024-03-07T07:43:30.687 INFO:tasks.workunit.client.0.smithi160.stdout:journalct 48320 root rtd unknown /proc/48320/root (readlink: Permission denied) 2024-03-07T07:43:30.688 INFO:tasks.workunit.client.0.smithi160.stdout:journalct 48320 root txt unknown /proc/48320/exe (readlink: Permission denied) 2024-03-07T07:43:30.688 INFO:tasks.workunit.client.0.smithi160.stdout:journalct 48320 root NOFD /proc/48320/fd (opendir: Permission denied) 2024-03-07T07:43:30.688 INFO:tasks.workunit.client.0.smithi160.stdout:conmon 53426 root cwd unknown /proc/53426/cwd (readlink: Permission denied) 2024-03-07T07:43:30.688 INFO:tasks.workunit.client.0.smithi160.stdout:conmon 53426 root rtd unknown /proc/53426/root (readlink: Permission denied) 2024-03-07T07:43:30.688 INFO:tasks.workunit.client.0.smithi160.stdout:conmon 53426 root txt unknown /proc/53426/exe (readlink: Permission denied) 2024-03-07T07:43:30.688 INFO:tasks.workunit.client.0.smithi160.stdout:conmon 53426 root NOFD /proc/53426/fd (opendir: Permission denied) 2024-03-07T07:43:30.688 INFO:tasks.workunit.client.0.smithi160.stdout:podman-in 53428 root cwd unknown /proc/53428/cwd (readlink: Permission denied) 2024-03-07T07:43:30.688 INFO:tasks.workunit.client.0.smithi160.stdout:podman-in 53428 root rtd unknown /proc/53428/root (readlink: Permission denied) 2024-03-07T07:43:30.688 INFO:tasks.workunit.client.0.smithi160.stdout:podman-in 53428 root txt unknown /proc/53428/exe (readlink: Permission denied) 2024-03-07T07:43:30.688 INFO:tasks.workunit.client.0.smithi160.stdout:podman-in 53428 root NOFD /proc/53428/fd (opendir: Permission denied) 2024-03-07T07:43:30.688 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 ceph cwd unknown /proc/53430/cwd (readlink: Permission denied) 2024-03-07T07:43:30.688 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 ceph rtd unknown /proc/53430/root (readlink: Permission denied) 2024-03-07T07:43:30.688 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 ceph txt unknown /proc/53430/exe (readlink: Permission denied) 2024-03-07T07:43:30.689 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 ceph NOFD /proc/53430/fd (opendir: Permission denied) 2024-03-07T07:43:30.689 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53431 log ceph cwd unknown /proc/53430/task/53431/cwd (readlink: Permission denied) 2024-03-07T07:43:30.689 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53431 log ceph rtd unknown /proc/53430/task/53431/root (readlink: Permission denied) 2024-03-07T07:43:30.689 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53431 log ceph txt unknown /proc/53430/task/53431/exe (readlink: Permission denied) 2024-03-07T07:43:30.689 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53431 log ceph NOFD /proc/53430/task/53431/fd (opendir: Permission denied) 2024-03-07T07:43:30.689 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53433 msgr-work ceph cwd unknown /proc/53430/task/53433/cwd (readlink: Permission denied) 2024-03-07T07:43:30.689 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53433 msgr-work ceph rtd unknown /proc/53430/task/53433/root (readlink: Permission denied) 2024-03-07T07:43:30.689 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53433 msgr-work ceph txt unknown /proc/53430/task/53433/exe (readlink: Permission denied) 2024-03-07T07:43:30.689 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53433 msgr-work ceph NOFD /proc/53430/task/53433/fd (opendir: Permission denied) 2024-03-07T07:43:30.689 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53434 msgr-work ceph cwd unknown /proc/53430/task/53434/cwd (readlink: Permission denied) 2024-03-07T07:43:30.689 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53434 msgr-work ceph rtd unknown /proc/53430/task/53434/root (readlink: Permission denied) 2024-03-07T07:43:30.689 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53434 msgr-work ceph txt unknown /proc/53430/task/53434/exe (readlink: Permission denied) 2024-03-07T07:43:30.689 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53434 msgr-work ceph NOFD /proc/53430/task/53434/fd (opendir: Permission denied) 2024-03-07T07:43:30.690 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53435 msgr-work ceph cwd unknown /proc/53430/task/53435/cwd (readlink: Permission denied) 2024-03-07T07:43:30.690 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53435 msgr-work ceph rtd unknown /proc/53430/task/53435/root (readlink: Permission denied) 2024-03-07T07:43:30.690 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53435 msgr-work ceph txt unknown /proc/53430/task/53435/exe (readlink: Permission denied) 2024-03-07T07:43:30.690 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53435 msgr-work ceph NOFD /proc/53430/task/53435/fd (opendir: Permission denied) 2024-03-07T07:43:30.690 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53439 service ceph cwd unknown /proc/53430/task/53439/cwd (readlink: Permission denied) 2024-03-07T07:43:30.690 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53439 service ceph rtd unknown /proc/53430/task/53439/root (readlink: Permission denied) 2024-03-07T07:43:30.690 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53439 service ceph txt unknown /proc/53430/task/53439/exe (readlink: Permission denied) 2024-03-07T07:43:30.690 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53439 service ceph NOFD /proc/53430/task/53439/fd (opendir: Permission denied) 2024-03-07T07:43:30.690 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53440 admin_soc ceph cwd unknown /proc/53430/task/53440/cwd (readlink: Permission denied) 2024-03-07T07:43:30.690 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53440 admin_soc ceph rtd unknown /proc/53430/task/53440/root (readlink: Permission denied) 2024-03-07T07:43:30.690 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53440 admin_soc ceph txt unknown /proc/53430/task/53440/exe (readlink: Permission denied) 2024-03-07T07:43:30.690 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53440 admin_soc ceph NOFD /proc/53430/task/53440/fd (opendir: Permission denied) 2024-03-07T07:43:30.690 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53582 signal_ha ceph cwd unknown /proc/53430/task/53582/cwd (readlink: Permission denied) 2024-03-07T07:43:30.690 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53582 signal_ha ceph rtd unknown /proc/53430/task/53582/root (readlink: Permission denied) 2024-03-07T07:43:30.691 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53582 signal_ha ceph txt unknown /proc/53430/task/53582/exe (readlink: Permission denied) 2024-03-07T07:43:30.691 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53582 signal_ha ceph NOFD /proc/53430/task/53582/fd (opendir: Permission denied) 2024-03-07T07:43:30.691 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53583 io_contex ceph cwd unknown /proc/53430/task/53583/cwd (readlink: Permission denied) 2024-03-07T07:43:30.691 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53583 io_contex ceph rtd unknown /proc/53430/task/53583/root (readlink: Permission denied) 2024-03-07T07:43:30.691 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53583 io_contex ceph txt unknown /proc/53430/task/53583/exe (readlink: Permission denied) 2024-03-07T07:43:30.691 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53583 io_contex ceph NOFD /proc/53430/task/53583/fd (opendir: Permission denied) 2024-03-07T07:43:30.691 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53584 io_contex ceph cwd unknown /proc/53430/task/53584/cwd (readlink: Permission denied) 2024-03-07T07:43:30.691 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53584 io_contex ceph rtd unknown /proc/53430/task/53584/root (readlink: Permission denied) 2024-03-07T07:43:30.691 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53584 io_contex ceph txt unknown /proc/53430/task/53584/exe (readlink: Permission denied) 2024-03-07T07:43:30.691 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53584 io_contex ceph NOFD /proc/53430/task/53584/fd (opendir: Permission denied) 2024-03-07T07:43:30.691 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53711 OpHistory ceph cwd unknown /proc/53430/task/53711/cwd (readlink: Permission denied) 2024-03-07T07:43:30.691 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53711 OpHistory ceph rtd unknown /proc/53430/task/53711/root (readlink: Permission denied) 2024-03-07T07:43:30.691 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53711 OpHistory ceph txt unknown /proc/53430/task/53711/exe (readlink: Permission denied) 2024-03-07T07:43:30.691 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53711 OpHistory ceph NOFD /proc/53430/task/53711/fd (opendir: Permission denied) 2024-03-07T07:43:30.692 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53712 ceph_time ceph cwd unknown /proc/53430/task/53712/cwd (readlink: Permission denied) 2024-03-07T07:43:30.692 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53712 ceph_time ceph rtd unknown /proc/53430/task/53712/root (readlink: Permission denied) 2024-03-07T07:43:30.692 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53712 ceph_time ceph txt unknown /proc/53430/task/53712/exe (readlink: Permission denied) 2024-03-07T07:43:30.692 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53712 ceph_time ceph NOFD /proc/53430/task/53712/fd (opendir: Permission denied) 2024-03-07T07:43:30.692 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53872 ceph-osd ceph cwd unknown /proc/53430/task/53872/cwd (readlink: Permission denied) 2024-03-07T07:43:30.692 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53872 ceph-osd ceph rtd unknown /proc/53430/task/53872/root (readlink: Permission denied) 2024-03-07T07:43:30.692 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53872 ceph-osd ceph txt unknown /proc/53430/task/53872/exe (readlink: Permission denied) 2024-03-07T07:43:30.692 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53872 ceph-osd ceph NOFD /proc/53430/task/53872/fd (opendir: Permission denied) 2024-03-07T07:43:30.692 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53873 safe_time ceph cwd unknown /proc/53430/task/53873/cwd (readlink: Permission denied) 2024-03-07T07:43:30.692 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53873 safe_time ceph rtd unknown /proc/53430/task/53873/root (readlink: Permission denied) 2024-03-07T07:43:30.692 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53873 safe_time ceph txt unknown /proc/53430/task/53873/exe (readlink: Permission denied) 2024-03-07T07:43:30.692 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53873 safe_time ceph NOFD /proc/53430/task/53873/fd (opendir: Permission denied) 2024-03-07T07:43:30.692 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53874 safe_time ceph cwd unknown /proc/53430/task/53874/cwd (readlink: Permission denied) 2024-03-07T07:43:30.693 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53874 safe_time ceph rtd unknown /proc/53430/task/53874/root (readlink: Permission denied) 2024-03-07T07:43:30.693 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53874 safe_time ceph txt unknown /proc/53430/task/53874/exe (readlink: Permission denied) 2024-03-07T07:43:30.693 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53874 safe_time ceph NOFD /proc/53430/task/53874/fd (opendir: Permission denied) 2024-03-07T07:43:30.693 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53875 safe_time ceph cwd unknown /proc/53430/task/53875/cwd (readlink: Permission denied) 2024-03-07T07:43:30.693 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53875 safe_time ceph rtd unknown /proc/53430/task/53875/root (readlink: Permission denied) 2024-03-07T07:43:30.693 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53875 safe_time ceph txt unknown /proc/53430/task/53875/exe (readlink: Permission denied) 2024-03-07T07:43:30.693 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53875 safe_time ceph NOFD /proc/53430/task/53875/fd (opendir: Permission denied) 2024-03-07T07:43:30.693 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53876 safe_time ceph cwd unknown /proc/53430/task/53876/cwd (readlink: Permission denied) 2024-03-07T07:43:30.693 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53876 safe_time ceph rtd unknown /proc/53430/task/53876/root (readlink: Permission denied) 2024-03-07T07:43:30.693 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53876 safe_time ceph txt unknown /proc/53430/task/53876/exe (readlink: Permission denied) 2024-03-07T07:43:30.693 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53876 safe_time ceph NOFD /proc/53430/task/53876/fd (opendir: Permission denied) 2024-03-07T07:43:30.693 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53877 fn_anonym ceph cwd unknown /proc/53430/task/53877/cwd (readlink: Permission denied) 2024-03-07T07:43:30.693 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53877 fn_anonym ceph rtd unknown /proc/53430/task/53877/root (readlink: Permission denied) 2024-03-07T07:43:30.693 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53877 fn_anonym ceph txt unknown /proc/53430/task/53877/exe (readlink: Permission denied) 2024-03-07T07:43:30.693 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53877 fn_anonym ceph NOFD /proc/53430/task/53877/fd (opendir: Permission denied) 2024-03-07T07:43:30.694 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53881 rocksdb:l ceph cwd unknown /proc/53430/task/53881/cwd (readlink: Permission denied) 2024-03-07T07:43:30.694 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53881 rocksdb:l ceph rtd unknown /proc/53430/task/53881/root (readlink: Permission denied) 2024-03-07T07:43:30.694 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53881 rocksdb:l ceph txt unknown /proc/53430/task/53881/exe (readlink: Permission denied) 2024-03-07T07:43:30.694 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53881 rocksdb:l ceph NOFD /proc/53430/task/53881/fd (opendir: Permission denied) 2024-03-07T07:43:30.694 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53882 rocksdb:l ceph cwd unknown /proc/53430/task/53882/cwd (readlink: Permission denied) 2024-03-07T07:43:30.694 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53882 rocksdb:l ceph rtd unknown /proc/53430/task/53882/root (readlink: Permission denied) 2024-03-07T07:43:30.694 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53882 rocksdb:l ceph txt unknown /proc/53430/task/53882/exe (readlink: Permission denied) 2024-03-07T07:43:30.694 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53882 rocksdb:l ceph NOFD /proc/53430/task/53882/fd (opendir: Permission denied) 2024-03-07T07:43:30.694 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53883 rocksdb:l ceph cwd unknown /proc/53430/task/53883/cwd (readlink: Permission denied) 2024-03-07T07:43:30.694 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53883 rocksdb:l ceph rtd unknown /proc/53430/task/53883/root (readlink: Permission denied) 2024-03-07T07:43:30.694 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53883 rocksdb:l ceph txt unknown /proc/53430/task/53883/exe (readlink: Permission denied) 2024-03-07T07:43:30.694 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53883 rocksdb:l ceph NOFD /proc/53430/task/53883/fd (opendir: Permission denied) 2024-03-07T07:43:30.694 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53884 rocksdb:h ceph cwd unknown /proc/53430/task/53884/cwd (readlink: Permission denied) 2024-03-07T07:43:30.694 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53884 rocksdb:h ceph rtd unknown /proc/53430/task/53884/root (readlink: Permission denied) 2024-03-07T07:43:30.694 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53884 rocksdb:h ceph txt unknown /proc/53430/task/53884/exe (readlink: Permission denied) 2024-03-07T07:43:30.695 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 53884 rocksdb:h ceph NOFD /proc/53430/task/53884/fd (opendir: Permission denied) 2024-03-07T07:43:30.695 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54327 bstore_ai ceph cwd unknown /proc/53430/task/54327/cwd (readlink: Permission denied) 2024-03-07T07:43:30.695 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54327 bstore_ai ceph rtd unknown /proc/53430/task/54327/root (readlink: Permission denied) 2024-03-07T07:43:30.695 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54327 bstore_ai ceph txt unknown /proc/53430/task/54327/exe (readlink: Permission denied) 2024-03-07T07:43:30.695 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54327 bstore_ai ceph NOFD /proc/53430/task/54327/fd (opendir: Permission denied) 2024-03-07T07:43:30.695 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54513 bstore_ai ceph cwd unknown /proc/53430/task/54513/cwd (readlink: Permission denied) 2024-03-07T07:43:30.695 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54513 bstore_ai ceph rtd unknown /proc/53430/task/54513/root (readlink: Permission denied) 2024-03-07T07:43:30.695 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54513 bstore_ai ceph txt unknown /proc/53430/task/54513/exe (readlink: Permission denied) 2024-03-07T07:43:30.695 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54513 bstore_ai ceph NOFD /proc/53430/task/54513/fd (opendir: Permission denied) 2024-03-07T07:43:30.695 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54711 ceph-osd ceph cwd unknown /proc/53430/task/54711/cwd (readlink: Permission denied) 2024-03-07T07:43:30.695 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54711 ceph-osd ceph rtd unknown /proc/53430/task/54711/root (readlink: Permission denied) 2024-03-07T07:43:30.695 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54711 ceph-osd ceph txt unknown /proc/53430/task/54711/exe (readlink: Permission denied) 2024-03-07T07:43:30.695 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54711 ceph-osd ceph NOFD /proc/53430/task/54711/fd (opendir: Permission denied) 2024-03-07T07:43:30.695 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54713 cfin ceph cwd unknown /proc/53430/task/54713/cwd (readlink: Permission denied) 2024-03-07T07:43:30.695 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54713 cfin ceph rtd unknown /proc/53430/task/54713/root (readlink: Permission denied) 2024-03-07T07:43:30.696 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54713 cfin ceph txt unknown /proc/53430/task/54713/exe (readlink: Permission denied) 2024-03-07T07:43:30.696 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54713 cfin ceph NOFD /proc/53430/task/54713/fd (opendir: Permission denied) 2024-03-07T07:43:30.696 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54714 bstore_kv ceph cwd unknown /proc/53430/task/54714/cwd (readlink: Permission denied) 2024-03-07T07:43:30.696 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54714 bstore_kv ceph rtd unknown /proc/53430/task/54714/root (readlink: Permission denied) 2024-03-07T07:43:30.696 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54714 bstore_kv ceph txt unknown /proc/53430/task/54714/exe (readlink: Permission denied) 2024-03-07T07:43:30.696 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54714 bstore_kv ceph NOFD /proc/53430/task/54714/fd (opendir: Permission denied) 2024-03-07T07:43:30.696 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54715 bstore_kv ceph cwd unknown /proc/53430/task/54715/cwd (readlink: Permission denied) 2024-03-07T07:43:30.696 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54715 bstore_kv ceph rtd unknown /proc/53430/task/54715/root (readlink: Permission denied) 2024-03-07T07:43:30.696 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54715 bstore_kv ceph txt unknown /proc/53430/task/54715/exe (readlink: Permission denied) 2024-03-07T07:43:30.696 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54715 bstore_kv ceph NOFD /proc/53430/task/54715/fd (opendir: Permission denied) 2024-03-07T07:43:30.696 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54716 bstore_me ceph cwd unknown /proc/53430/task/54716/cwd (readlink: Permission denied) 2024-03-07T07:43:30.696 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54716 bstore_me ceph rtd unknown /proc/53430/task/54716/root (readlink: Permission denied) 2024-03-07T07:43:30.697 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54716 bstore_me ceph txt unknown /proc/53430/task/54716/exe (readlink: Permission denied) 2024-03-07T07:43:30.697 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54716 bstore_me ceph NOFD /proc/53430/task/54716/fd (opendir: Permission denied) 2024-03-07T07:43:30.697 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54730 ms_dispat ceph cwd unknown /proc/53430/task/54730/cwd (readlink: Permission denied) 2024-03-07T07:43:30.697 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54730 ms_dispat ceph rtd unknown /proc/53430/task/54730/root (readlink: Permission denied) 2024-03-07T07:43:30.697 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54730 ms_dispat ceph txt unknown /proc/53430/task/54730/exe (readlink: Permission denied) 2024-03-07T07:43:30.697 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54730 ms_dispat ceph NOFD /proc/53430/task/54730/fd (opendir: Permission denied) 2024-03-07T07:43:30.697 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54731 ms_local ceph cwd unknown /proc/53430/task/54731/cwd (readlink: Permission denied) 2024-03-07T07:43:30.697 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54731 ms_local ceph rtd unknown /proc/53430/task/54731/root (readlink: Permission denied) 2024-03-07T07:43:30.697 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54731 ms_local ceph txt unknown /proc/53430/task/54731/exe (readlink: Permission denied) 2024-03-07T07:43:30.697 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54731 ms_local ceph NOFD /proc/53430/task/54731/fd (opendir: Permission denied) 2024-03-07T07:43:30.697 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54732 safe_time ceph cwd unknown /proc/53430/task/54732/cwd (readlink: Permission denied) 2024-03-07T07:43:30.697 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54732 safe_time ceph rtd unknown /proc/53430/task/54732/root (readlink: Permission denied) 2024-03-07T07:43:30.697 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54732 safe_time ceph txt unknown /proc/53430/task/54732/exe (readlink: Permission denied) 2024-03-07T07:43:30.697 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54732 safe_time ceph NOFD /proc/53430/task/54732/fd (opendir: Permission denied) 2024-03-07T07:43:30.698 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54733 safe_time ceph cwd unknown /proc/53430/task/54733/cwd (readlink: Permission denied) 2024-03-07T07:43:30.698 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54733 safe_time ceph rtd unknown /proc/53430/task/54733/root (readlink: Permission denied) 2024-03-07T07:43:30.698 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54733 safe_time ceph txt unknown /proc/53430/task/54733/exe (readlink: Permission denied) 2024-03-07T07:43:30.698 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54733 safe_time ceph NOFD /proc/53430/task/54733/fd (opendir: Permission denied) 2024-03-07T07:43:30.698 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54734 ms_dispat ceph cwd unknown /proc/53430/task/54734/cwd (readlink: Permission denied) 2024-03-07T07:43:30.698 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54734 ms_dispat ceph rtd unknown /proc/53430/task/54734/root (readlink: Permission denied) 2024-03-07T07:43:30.698 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54734 ms_dispat ceph txt unknown /proc/53430/task/54734/exe (readlink: Permission denied) 2024-03-07T07:43:30.698 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54734 ms_dispat ceph NOFD /proc/53430/task/54734/fd (opendir: Permission denied) 2024-03-07T07:43:30.698 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54735 ms_local ceph cwd unknown /proc/53430/task/54735/cwd (readlink: Permission denied) 2024-03-07T07:43:30.698 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54735 ms_local ceph rtd unknown /proc/53430/task/54735/root (readlink: Permission denied) 2024-03-07T07:43:30.698 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54735 ms_local ceph txt unknown /proc/53430/task/54735/exe (readlink: Permission denied) 2024-03-07T07:43:30.698 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54735 ms_local ceph NOFD /proc/53430/task/54735/fd (opendir: Permission denied) 2024-03-07T07:43:30.698 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54736 ms_dispat ceph cwd unknown /proc/53430/task/54736/cwd (readlink: Permission denied) 2024-03-07T07:43:30.698 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54736 ms_dispat ceph rtd unknown /proc/53430/task/54736/root (readlink: Permission denied) 2024-03-07T07:43:30.698 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54736 ms_dispat ceph txt unknown /proc/53430/task/54736/exe (readlink: Permission denied) 2024-03-07T07:43:30.699 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54736 ms_dispat ceph NOFD /proc/53430/task/54736/fd (opendir: Permission denied) 2024-03-07T07:43:30.699 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54737 ms_local ceph cwd unknown /proc/53430/task/54737/cwd (readlink: Permission denied) 2024-03-07T07:43:30.699 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54737 ms_local ceph rtd unknown /proc/53430/task/54737/root (readlink: Permission denied) 2024-03-07T07:43:30.699 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54737 ms_local ceph txt unknown /proc/53430/task/54737/exe (readlink: Permission denied) 2024-03-07T07:43:30.699 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54737 ms_local ceph NOFD /proc/53430/task/54737/fd (opendir: Permission denied) 2024-03-07T07:43:30.699 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54738 ms_dispat ceph cwd unknown /proc/53430/task/54738/cwd (readlink: Permission denied) 2024-03-07T07:43:30.699 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54738 ms_dispat ceph rtd unknown /proc/53430/task/54738/root (readlink: Permission denied) 2024-03-07T07:43:30.699 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54738 ms_dispat ceph txt unknown /proc/53430/task/54738/exe (readlink: Permission denied) 2024-03-07T07:43:30.699 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54738 ms_dispat ceph NOFD /proc/53430/task/54738/fd (opendir: Permission denied) 2024-03-07T07:43:30.699 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54739 ms_local ceph cwd unknown /proc/53430/task/54739/cwd (readlink: Permission denied) 2024-03-07T07:43:30.699 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54739 ms_local ceph rtd unknown /proc/53430/task/54739/root (readlink: Permission denied) 2024-03-07T07:43:30.699 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54739 ms_local ceph txt unknown /proc/53430/task/54739/exe (readlink: Permission denied) 2024-03-07T07:43:30.699 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54739 ms_local ceph NOFD /proc/53430/task/54739/fd (opendir: Permission denied) 2024-03-07T07:43:30.700 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54740 ms_dispat ceph cwd unknown /proc/53430/task/54740/cwd (readlink: Permission denied) 2024-03-07T07:43:30.700 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54740 ms_dispat ceph rtd unknown /proc/53430/task/54740/root (readlink: Permission denied) 2024-03-07T07:43:30.700 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54740 ms_dispat ceph txt unknown /proc/53430/task/54740/exe (readlink: Permission denied) 2024-03-07T07:43:30.700 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54740 ms_dispat ceph NOFD /proc/53430/task/54740/fd (opendir: Permission denied) 2024-03-07T07:43:30.700 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54741 ms_local ceph cwd unknown /proc/53430/task/54741/cwd (readlink: Permission denied) 2024-03-07T07:43:30.700 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54741 ms_local ceph rtd unknown /proc/53430/task/54741/root (readlink: Permission denied) 2024-03-07T07:43:30.700 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54741 ms_local ceph txt unknown /proc/53430/task/54741/exe (readlink: Permission denied) 2024-03-07T07:43:30.700 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54741 ms_local ceph NOFD /proc/53430/task/54741/fd (opendir: Permission denied) 2024-03-07T07:43:30.700 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54743 ms_dispat ceph cwd unknown /proc/53430/task/54743/cwd (readlink: Permission denied) 2024-03-07T07:43:30.700 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54743 ms_dispat ceph rtd unknown /proc/53430/task/54743/root (readlink: Permission denied) 2024-03-07T07:43:30.700 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54743 ms_dispat ceph txt unknown /proc/53430/task/54743/exe (readlink: Permission denied) 2024-03-07T07:43:30.700 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54743 ms_dispat ceph NOFD /proc/53430/task/54743/fd (opendir: Permission denied) 2024-03-07T07:43:30.700 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54744 ms_local ceph cwd unknown /proc/53430/task/54744/cwd (readlink: Permission denied) 2024-03-07T07:43:30.700 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54744 ms_local ceph rtd unknown /proc/53430/task/54744/root (readlink: Permission denied) 2024-03-07T07:43:30.701 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54744 ms_local ceph txt unknown /proc/53430/task/54744/exe (readlink: Permission denied) 2024-03-07T07:43:30.701 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54744 ms_local ceph NOFD /proc/53430/task/54744/fd (opendir: Permission denied) 2024-03-07T07:43:30.701 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54745 ms_dispat ceph cwd unknown /proc/53430/task/54745/cwd (readlink: Permission denied) 2024-03-07T07:43:30.701 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54745 ms_dispat ceph rtd unknown /proc/53430/task/54745/root (readlink: Permission denied) 2024-03-07T07:43:30.701 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54745 ms_dispat ceph txt unknown /proc/53430/task/54745/exe (readlink: Permission denied) 2024-03-07T07:43:30.701 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54745 ms_dispat ceph NOFD /proc/53430/task/54745/fd (opendir: Permission denied) 2024-03-07T07:43:30.701 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54746 ms_local ceph cwd unknown /proc/53430/task/54746/cwd (readlink: Permission denied) 2024-03-07T07:43:30.701 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54746 ms_local ceph rtd unknown /proc/53430/task/54746/root (readlink: Permission denied) 2024-03-07T07:43:30.701 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54746 ms_local ceph txt unknown /proc/53430/task/54746/exe (readlink: Permission denied) 2024-03-07T07:43:30.701 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54746 ms_local ceph NOFD /proc/53430/task/54746/fd (opendir: Permission denied) 2024-03-07T07:43:30.701 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54747 fn_anonym ceph cwd unknown /proc/53430/task/54747/cwd (readlink: Permission denied) 2024-03-07T07:43:30.701 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54747 fn_anonym ceph rtd unknown /proc/53430/task/54747/root (readlink: Permission denied) 2024-03-07T07:43:30.701 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54747 fn_anonym ceph txt unknown /proc/53430/task/54747/exe (readlink: Permission denied) 2024-03-07T07:43:30.701 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54747 fn_anonym ceph NOFD /proc/53430/task/54747/fd (opendir: Permission denied) 2024-03-07T07:43:30.702 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54748 finisher ceph cwd unknown /proc/53430/task/54748/cwd (readlink: Permission denied) 2024-03-07T07:43:30.702 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54748 finisher ceph rtd unknown /proc/53430/task/54748/root (readlink: Permission denied) 2024-03-07T07:43:30.702 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54748 finisher ceph txt unknown /proc/53430/task/54748/exe (readlink: Permission denied) 2024-03-07T07:43:30.702 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54748 finisher ceph NOFD /proc/53430/task/54748/fd (opendir: Permission denied) 2024-03-07T07:43:30.702 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54749 safe_time ceph cwd unknown /proc/53430/task/54749/cwd (readlink: Permission denied) 2024-03-07T07:43:30.702 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54749 safe_time ceph rtd unknown /proc/53430/task/54749/root (readlink: Permission denied) 2024-03-07T07:43:30.702 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54749 safe_time ceph txt unknown /proc/53430/task/54749/exe (readlink: Permission denied) 2024-03-07T07:43:30.702 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54749 safe_time ceph NOFD /proc/53430/task/54749/fd (opendir: Permission denied) 2024-03-07T07:43:30.702 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54750 safe_time ceph cwd unknown /proc/53430/task/54750/cwd (readlink: Permission denied) 2024-03-07T07:43:30.702 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54750 safe_time ceph rtd unknown /proc/53430/task/54750/root (readlink: Permission denied) 2024-03-07T07:43:30.702 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54750 safe_time ceph txt unknown /proc/53430/task/54750/exe (readlink: Permission denied) 2024-03-07T07:43:30.702 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54750 safe_time ceph NOFD /proc/53430/task/54750/fd (opendir: Permission denied) 2024-03-07T07:43:30.702 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54751 ceph-osd ceph cwd unknown /proc/53430/task/54751/cwd (readlink: Permission denied) 2024-03-07T07:43:30.702 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54751 ceph-osd ceph rtd unknown /proc/53430/task/54751/root (readlink: Permission denied) 2024-03-07T07:43:30.702 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54751 ceph-osd ceph txt unknown /proc/53430/task/54751/exe (readlink: Permission denied) 2024-03-07T07:43:30.703 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54751 ceph-osd ceph NOFD /proc/53430/task/54751/fd (opendir: Permission denied) 2024-03-07T07:43:30.703 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54752 osd_srv_a ceph cwd unknown /proc/53430/task/54752/cwd (readlink: Permission denied) 2024-03-07T07:43:30.703 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54752 osd_srv_a ceph rtd unknown /proc/53430/task/54752/root (readlink: Permission denied) 2024-03-07T07:43:30.703 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54752 osd_srv_a ceph txt unknown /proc/53430/task/54752/exe (readlink: Permission denied) 2024-03-07T07:43:30.703 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54752 osd_srv_a ceph NOFD /proc/53430/task/54752/fd (opendir: Permission denied) 2024-03-07T07:43:30.703 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54753 tp_osd_tp ceph cwd unknown /proc/53430/task/54753/cwd (readlink: Permission denied) 2024-03-07T07:43:30.703 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54753 tp_osd_tp ceph rtd unknown /proc/53430/task/54753/root (readlink: Permission denied) 2024-03-07T07:43:30.703 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54753 tp_osd_tp ceph txt unknown /proc/53430/task/54753/exe (readlink: Permission denied) 2024-03-07T07:43:30.703 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54753 tp_osd_tp ceph NOFD /proc/53430/task/54753/fd (opendir: Permission denied) 2024-03-07T07:43:30.703 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54754 tp_osd_tp ceph cwd unknown /proc/53430/task/54754/cwd (readlink: Permission denied) 2024-03-07T07:43:30.703 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54754 tp_osd_tp ceph rtd unknown /proc/53430/task/54754/root (readlink: Permission denied) 2024-03-07T07:43:30.703 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54754 tp_osd_tp ceph txt unknown /proc/53430/task/54754/exe (readlink: Permission denied) 2024-03-07T07:43:30.703 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54754 tp_osd_tp ceph NOFD /proc/53430/task/54754/fd (opendir: Permission denied) 2024-03-07T07:43:30.703 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54755 tp_osd_tp ceph cwd unknown /proc/53430/task/54755/cwd (readlink: Permission denied) 2024-03-07T07:43:30.704 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54755 tp_osd_tp ceph rtd unknown /proc/53430/task/54755/root (readlink: Permission denied) 2024-03-07T07:43:30.704 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54755 tp_osd_tp ceph txt unknown /proc/53430/task/54755/exe (readlink: Permission denied) 2024-03-07T07:43:30.704 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54755 tp_osd_tp ceph NOFD /proc/53430/task/54755/fd (opendir: Permission denied) 2024-03-07T07:43:30.704 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54756 tp_osd_tp ceph cwd unknown /proc/53430/task/54756/cwd (readlink: Permission denied) 2024-03-07T07:43:30.704 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54756 tp_osd_tp ceph rtd unknown /proc/53430/task/54756/root (readlink: Permission denied) 2024-03-07T07:43:30.704 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54756 tp_osd_tp ceph txt unknown /proc/53430/task/54756/exe (readlink: Permission denied) 2024-03-07T07:43:30.704 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54756 tp_osd_tp ceph NOFD /proc/53430/task/54756/fd (opendir: Permission denied) 2024-03-07T07:43:30.704 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54757 tp_osd_tp ceph cwd unknown /proc/53430/task/54757/cwd (readlink: Permission denied) 2024-03-07T07:43:30.704 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54757 tp_osd_tp ceph rtd unknown /proc/53430/task/54757/root (readlink: Permission denied) 2024-03-07T07:43:30.704 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54757 tp_osd_tp ceph txt unknown /proc/53430/task/54757/exe (readlink: Permission denied) 2024-03-07T07:43:30.704 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54757 tp_osd_tp ceph NOFD /proc/53430/task/54757/fd (opendir: Permission denied) 2024-03-07T07:43:30.704 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54758 tp_osd_tp ceph cwd unknown /proc/53430/task/54758/cwd (readlink: Permission denied) 2024-03-07T07:43:30.704 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54758 tp_osd_tp ceph rtd unknown /proc/53430/task/54758/root (readlink: Permission denied) 2024-03-07T07:43:30.705 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54758 tp_osd_tp ceph txt unknown /proc/53430/task/54758/exe (readlink: Permission denied) 2024-03-07T07:43:30.705 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54758 tp_osd_tp ceph NOFD /proc/53430/task/54758/fd (opendir: Permission denied) 2024-03-07T07:43:30.705 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54759 tp_osd_tp ceph cwd unknown /proc/53430/task/54759/cwd (readlink: Permission denied) 2024-03-07T07:43:30.705 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54759 tp_osd_tp ceph rtd unknown /proc/53430/task/54759/root (readlink: Permission denied) 2024-03-07T07:43:30.705 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54759 tp_osd_tp ceph txt unknown /proc/53430/task/54759/exe (readlink: Permission denied) 2024-03-07T07:43:30.705 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54759 tp_osd_tp ceph NOFD /proc/53430/task/54759/fd (opendir: Permission denied) 2024-03-07T07:43:30.705 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54760 tp_osd_tp ceph cwd unknown /proc/53430/task/54760/cwd (readlink: Permission denied) 2024-03-07T07:43:30.705 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54760 tp_osd_tp ceph rtd unknown /proc/53430/task/54760/root (readlink: Permission denied) 2024-03-07T07:43:30.705 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54760 tp_osd_tp ceph txt unknown /proc/53430/task/54760/exe (readlink: Permission denied) 2024-03-07T07:43:30.705 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54760 tp_osd_tp ceph NOFD /proc/53430/task/54760/fd (opendir: Permission denied) 2024-03-07T07:43:30.705 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54761 tp_osd_tp ceph cwd unknown /proc/53430/task/54761/cwd (readlink: Permission denied) 2024-03-07T07:43:30.705 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54761 tp_osd_tp ceph rtd unknown /proc/53430/task/54761/root (readlink: Permission denied) 2024-03-07T07:43:30.705 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54761 tp_osd_tp ceph txt unknown /proc/53430/task/54761/exe (readlink: Permission denied) 2024-03-07T07:43:30.705 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54761 tp_osd_tp ceph NOFD /proc/53430/task/54761/fd (opendir: Permission denied) 2024-03-07T07:43:30.706 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54762 tp_osd_tp ceph cwd unknown /proc/53430/task/54762/cwd (readlink: Permission denied) 2024-03-07T07:43:30.706 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54762 tp_osd_tp ceph rtd unknown /proc/53430/task/54762/root (readlink: Permission denied) 2024-03-07T07:43:30.706 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54762 tp_osd_tp ceph txt unknown /proc/53430/task/54762/exe (readlink: Permission denied) 2024-03-07T07:43:30.706 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54762 tp_osd_tp ceph NOFD /proc/53430/task/54762/fd (opendir: Permission denied) 2024-03-07T07:43:30.706 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54763 tp_osd_tp ceph cwd unknown /proc/53430/task/54763/cwd (readlink: Permission denied) 2024-03-07T07:43:30.706 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54763 tp_osd_tp ceph rtd unknown /proc/53430/task/54763/root (readlink: Permission denied) 2024-03-07T07:43:30.706 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54763 tp_osd_tp ceph txt unknown /proc/53430/task/54763/exe (readlink: Permission denied) 2024-03-07T07:43:30.706 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54763 tp_osd_tp ceph NOFD /proc/53430/task/54763/fd (opendir: Permission denied) 2024-03-07T07:43:30.706 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54764 tp_osd_tp ceph cwd unknown /proc/53430/task/54764/cwd (readlink: Permission denied) 2024-03-07T07:43:30.706 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54764 tp_osd_tp ceph rtd unknown /proc/53430/task/54764/root (readlink: Permission denied) 2024-03-07T07:43:30.706 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54764 tp_osd_tp ceph txt unknown /proc/53430/task/54764/exe (readlink: Permission denied) 2024-03-07T07:43:30.706 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54764 tp_osd_tp ceph NOFD /proc/53430/task/54764/fd (opendir: Permission denied) 2024-03-07T07:43:30.706 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54765 tp_osd_tp ceph cwd unknown /proc/53430/task/54765/cwd (readlink: Permission denied) 2024-03-07T07:43:30.706 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54765 tp_osd_tp ceph rtd unknown /proc/53430/task/54765/root (readlink: Permission denied) 2024-03-07T07:43:30.707 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54765 tp_osd_tp ceph txt unknown /proc/53430/task/54765/exe (readlink: Permission denied) 2024-03-07T07:43:30.707 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54765 tp_osd_tp ceph NOFD /proc/53430/task/54765/fd (opendir: Permission denied) 2024-03-07T07:43:30.707 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54766 tp_osd_tp ceph cwd unknown /proc/53430/task/54766/cwd (readlink: Permission denied) 2024-03-07T07:43:30.707 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54766 tp_osd_tp ceph rtd unknown /proc/53430/task/54766/root (readlink: Permission denied) 2024-03-07T07:43:30.707 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54766 tp_osd_tp ceph txt unknown /proc/53430/task/54766/exe (readlink: Permission denied) 2024-03-07T07:43:30.707 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54766 tp_osd_tp ceph NOFD /proc/53430/task/54766/fd (opendir: Permission denied) 2024-03-07T07:43:30.707 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54767 tp_osd_tp ceph cwd unknown /proc/53430/task/54767/cwd (readlink: Permission denied) 2024-03-07T07:43:30.707 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54767 tp_osd_tp ceph rtd unknown /proc/53430/task/54767/root (readlink: Permission denied) 2024-03-07T07:43:30.707 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54767 tp_osd_tp ceph txt unknown /proc/53430/task/54767/exe (readlink: Permission denied) 2024-03-07T07:43:30.707 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54767 tp_osd_tp ceph NOFD /proc/53430/task/54767/fd (opendir: Permission denied) 2024-03-07T07:43:30.707 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54768 tp_osd_tp ceph cwd unknown /proc/53430/task/54768/cwd (readlink: Permission denied) 2024-03-07T07:43:30.707 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54768 tp_osd_tp ceph rtd unknown /proc/53430/task/54768/root (readlink: Permission denied) 2024-03-07T07:43:30.707 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54768 tp_osd_tp ceph txt unknown /proc/53430/task/54768/exe (readlink: Permission denied) 2024-03-07T07:43:30.707 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54768 tp_osd_tp ceph NOFD /proc/53430/task/54768/fd (opendir: Permission denied) 2024-03-07T07:43:30.708 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54769 osd_srv_h ceph cwd unknown /proc/53430/task/54769/cwd (readlink: Permission denied) 2024-03-07T07:43:30.708 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54769 osd_srv_h ceph rtd unknown /proc/53430/task/54769/root (readlink: Permission denied) 2024-03-07T07:43:30.708 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54769 osd_srv_h ceph txt unknown /proc/53430/task/54769/exe (readlink: Permission denied) 2024-03-07T07:43:30.708 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 53430 54769 osd_srv_h ceph NOFD /proc/53430/task/54769/fd (opendir: Permission denied) 2024-03-07T07:43:30.708 INFO:tasks.workunit.client.0.smithi160.stdout:sudo 54831 root cwd unknown /proc/54831/cwd (readlink: Permission denied) 2024-03-07T07:43:30.708 INFO:tasks.workunit.client.0.smithi160.stdout:sudo 54831 root rtd unknown /proc/54831/root (readlink: Permission denied) 2024-03-07T07:43:30.708 INFO:tasks.workunit.client.0.smithi160.stdout:sudo 54831 root txt unknown /proc/54831/exe (readlink: Permission denied) 2024-03-07T07:43:30.708 INFO:tasks.workunit.client.0.smithi160.stdout:sudo 54831 root NOFD /proc/54831/fd (opendir: Permission denied) 2024-03-07T07:43:30.708 INFO:tasks.workunit.client.0.smithi160.stdout:journalct 54848 root cwd unknown /proc/54848/cwd (readlink: Permission denied) 2024-03-07T07:43:30.708 INFO:tasks.workunit.client.0.smithi160.stdout:journalct 54848 root rtd unknown /proc/54848/root (readlink: Permission denied) 2024-03-07T07:43:30.708 INFO:tasks.workunit.client.0.smithi160.stdout:journalct 54848 root txt unknown /proc/54848/exe (readlink: Permission denied) 2024-03-07T07:43:30.708 INFO:tasks.workunit.client.0.smithi160.stdout:journalct 54848 root NOFD /proc/54848/fd (opendir: Permission denied) 2024-03-07T07:43:30.708 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/1 60059 root cwd unknown /proc/60059/cwd (readlink: Permission denied) 2024-03-07T07:43:30.709 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/1 60059 root rtd unknown /proc/60059/root (readlink: Permission denied) 2024-03-07T07:43:30.709 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/1 60059 root txt unknown /proc/60059/exe (readlink: Permission denied) 2024-03-07T07:43:30.709 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/1 60059 root NOFD /proc/60059/fd (opendir: Permission denied) 2024-03-07T07:43:30.709 INFO:tasks.workunit.client.0.smithi160.stdout:conmon 60423 root cwd unknown /proc/60423/cwd (readlink: Permission denied) 2024-03-07T07:43:30.709 INFO:tasks.workunit.client.0.smithi160.stdout:conmon 60423 root rtd unknown /proc/60423/root (readlink: Permission denied) 2024-03-07T07:43:30.709 INFO:tasks.workunit.client.0.smithi160.stdout:conmon 60423 root txt unknown /proc/60423/exe (readlink: Permission denied) 2024-03-07T07:43:30.709 INFO:tasks.workunit.client.0.smithi160.stdout:conmon 60423 root NOFD /proc/60423/fd (opendir: Permission denied) 2024-03-07T07:43:30.709 INFO:tasks.workunit.client.0.smithi160.stdout:podman-in 60425 root cwd unknown /proc/60425/cwd (readlink: Permission denied) 2024-03-07T07:43:30.709 INFO:tasks.workunit.client.0.smithi160.stdout:podman-in 60425 root rtd unknown /proc/60425/root (readlink: Permission denied) 2024-03-07T07:43:30.709 INFO:tasks.workunit.client.0.smithi160.stdout:podman-in 60425 root txt unknown /proc/60425/exe (readlink: Permission denied) 2024-03-07T07:43:30.709 INFO:tasks.workunit.client.0.smithi160.stdout:podman-in 60425 root NOFD /proc/60425/fd (opendir: Permission denied) 2024-03-07T07:43:30.709 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 ceph cwd unknown /proc/60427/cwd (readlink: Permission denied) 2024-03-07T07:43:30.709 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 ceph rtd unknown /proc/60427/root (readlink: Permission denied) 2024-03-07T07:43:30.709 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 ceph txt unknown /proc/60427/exe (readlink: Permission denied) 2024-03-07T07:43:30.709 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 ceph NOFD /proc/60427/fd (opendir: Permission denied) 2024-03-07T07:43:30.710 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60428 log ceph cwd unknown /proc/60427/task/60428/cwd (readlink: Permission denied) 2024-03-07T07:43:30.710 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60428 log ceph rtd unknown /proc/60427/task/60428/root (readlink: Permission denied) 2024-03-07T07:43:30.710 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60428 log ceph txt unknown /proc/60427/task/60428/exe (readlink: Permission denied) 2024-03-07T07:43:30.710 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60428 log ceph NOFD /proc/60427/task/60428/fd (opendir: Permission denied) 2024-03-07T07:43:30.710 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60430 msgr-work ceph cwd unknown /proc/60427/task/60430/cwd (readlink: Permission denied) 2024-03-07T07:43:30.710 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60430 msgr-work ceph rtd unknown /proc/60427/task/60430/root (readlink: Permission denied) 2024-03-07T07:43:30.710 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60430 msgr-work ceph txt unknown /proc/60427/task/60430/exe (readlink: Permission denied) 2024-03-07T07:43:30.710 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60430 msgr-work ceph NOFD /proc/60427/task/60430/fd (opendir: Permission denied) 2024-03-07T07:43:30.710 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60431 msgr-work ceph cwd unknown /proc/60427/task/60431/cwd (readlink: Permission denied) 2024-03-07T07:43:30.710 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60431 msgr-work ceph rtd unknown /proc/60427/task/60431/root (readlink: Permission denied) 2024-03-07T07:43:30.710 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60431 msgr-work ceph txt unknown /proc/60427/task/60431/exe (readlink: Permission denied) 2024-03-07T07:43:30.710 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60431 msgr-work ceph NOFD /proc/60427/task/60431/fd (opendir: Permission denied) 2024-03-07T07:43:30.710 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60432 msgr-work ceph cwd unknown /proc/60427/task/60432/cwd (readlink: Permission denied) 2024-03-07T07:43:30.710 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60432 msgr-work ceph rtd unknown /proc/60427/task/60432/root (readlink: Permission denied) 2024-03-07T07:43:30.711 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60432 msgr-work ceph txt unknown /proc/60427/task/60432/exe (readlink: Permission denied) 2024-03-07T07:43:30.711 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60432 msgr-work ceph NOFD /proc/60427/task/60432/fd (opendir: Permission denied) 2024-03-07T07:43:30.711 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60436 service ceph cwd unknown /proc/60427/task/60436/cwd (readlink: Permission denied) 2024-03-07T07:43:30.711 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60436 service ceph rtd unknown /proc/60427/task/60436/root (readlink: Permission denied) 2024-03-07T07:43:30.711 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60436 service ceph txt unknown /proc/60427/task/60436/exe (readlink: Permission denied) 2024-03-07T07:43:30.711 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60436 service ceph NOFD /proc/60427/task/60436/fd (opendir: Permission denied) 2024-03-07T07:43:30.711 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60437 admin_soc ceph cwd unknown /proc/60427/task/60437/cwd (readlink: Permission denied) 2024-03-07T07:43:30.711 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60437 admin_soc ceph rtd unknown /proc/60427/task/60437/root (readlink: Permission denied) 2024-03-07T07:43:30.711 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60437 admin_soc ceph txt unknown /proc/60427/task/60437/exe (readlink: Permission denied) 2024-03-07T07:43:30.711 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60437 admin_soc ceph NOFD /proc/60427/task/60437/fd (opendir: Permission denied) 2024-03-07T07:43:30.711 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60579 signal_ha ceph cwd unknown /proc/60427/task/60579/cwd (readlink: Permission denied) 2024-03-07T07:43:30.711 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60579 signal_ha ceph rtd unknown /proc/60427/task/60579/root (readlink: Permission denied) 2024-03-07T07:43:30.711 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60579 signal_ha ceph txt unknown /proc/60427/task/60579/exe (readlink: Permission denied) 2024-03-07T07:43:30.712 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60579 signal_ha ceph NOFD /proc/60427/task/60579/fd (opendir: Permission denied) 2024-03-07T07:43:30.712 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60580 io_contex ceph cwd unknown /proc/60427/task/60580/cwd (readlink: Permission denied) 2024-03-07T07:43:30.712 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60580 io_contex ceph rtd unknown /proc/60427/task/60580/root (readlink: Permission denied) 2024-03-07T07:43:30.712 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60580 io_contex ceph txt unknown /proc/60427/task/60580/exe (readlink: Permission denied) 2024-03-07T07:43:30.712 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60580 io_contex ceph NOFD /proc/60427/task/60580/fd (opendir: Permission denied) 2024-03-07T07:43:30.712 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60581 io_contex ceph cwd unknown /proc/60427/task/60581/cwd (readlink: Permission denied) 2024-03-07T07:43:30.712 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60581 io_contex ceph rtd unknown /proc/60427/task/60581/root (readlink: Permission denied) 2024-03-07T07:43:30.712 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60581 io_contex ceph txt unknown /proc/60427/task/60581/exe (readlink: Permission denied) 2024-03-07T07:43:30.712 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60581 io_contex ceph NOFD /proc/60427/task/60581/fd (opendir: Permission denied) 2024-03-07T07:43:30.712 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60694 OpHistory ceph cwd unknown /proc/60427/task/60694/cwd (readlink: Permission denied) 2024-03-07T07:43:30.712 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60694 OpHistory ceph rtd unknown /proc/60427/task/60694/root (readlink: Permission denied) 2024-03-07T07:43:30.712 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60694 OpHistory ceph txt unknown /proc/60427/task/60694/exe (readlink: Permission denied) 2024-03-07T07:43:30.712 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60694 OpHistory ceph NOFD /proc/60427/task/60694/fd (opendir: Permission denied) 2024-03-07T07:43:30.712 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60695 ceph_time ceph cwd unknown /proc/60427/task/60695/cwd (readlink: Permission denied) 2024-03-07T07:43:30.713 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60695 ceph_time ceph rtd unknown /proc/60427/task/60695/root (readlink: Permission denied) 2024-03-07T07:43:30.713 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60695 ceph_time ceph txt unknown /proc/60427/task/60695/exe (readlink: Permission denied) 2024-03-07T07:43:30.713 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60695 ceph_time ceph NOFD /proc/60427/task/60695/fd (opendir: Permission denied) 2024-03-07T07:43:30.713 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60728 ceph-osd ceph cwd unknown /proc/60427/task/60728/cwd (readlink: Permission denied) 2024-03-07T07:43:30.713 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60728 ceph-osd ceph rtd unknown /proc/60427/task/60728/root (readlink: Permission denied) 2024-03-07T07:43:30.713 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60728 ceph-osd ceph txt unknown /proc/60427/task/60728/exe (readlink: Permission denied) 2024-03-07T07:43:30.713 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60728 ceph-osd ceph NOFD /proc/60427/task/60728/fd (opendir: Permission denied) 2024-03-07T07:43:30.713 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60732 ceph-osd ceph cwd unknown /proc/60427/task/60732/cwd (readlink: Permission denied) 2024-03-07T07:43:30.713 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60732 ceph-osd ceph rtd unknown /proc/60427/task/60732/root (readlink: Permission denied) 2024-03-07T07:43:30.713 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60732 ceph-osd ceph txt unknown /proc/60427/task/60732/exe (readlink: Permission denied) 2024-03-07T07:43:30.713 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60732 ceph-osd ceph NOFD /proc/60427/task/60732/fd (opendir: Permission denied) 2024-03-07T07:43:30.713 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60753 ceph-osd ceph cwd unknown /proc/60427/task/60753/cwd (readlink: Permission denied) 2024-03-07T07:43:30.713 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60753 ceph-osd ceph rtd unknown /proc/60427/task/60753/root (readlink: Permission denied) 2024-03-07T07:43:30.713 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60753 ceph-osd ceph txt unknown /proc/60427/task/60753/exe (readlink: Permission denied) 2024-03-07T07:43:30.714 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60753 ceph-osd ceph NOFD /proc/60427/task/60753/fd (opendir: Permission denied) 2024-03-07T07:43:30.714 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60758 ceph-osd ceph cwd unknown /proc/60427/task/60758/cwd (readlink: Permission denied) 2024-03-07T07:43:30.714 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60758 ceph-osd ceph rtd unknown /proc/60427/task/60758/root (readlink: Permission denied) 2024-03-07T07:43:30.714 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60758 ceph-osd ceph txt unknown /proc/60427/task/60758/exe (readlink: Permission denied) 2024-03-07T07:43:30.714 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60758 ceph-osd ceph NOFD /proc/60427/task/60758/fd (opendir: Permission denied) 2024-03-07T07:43:30.714 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60780 ceph-osd ceph cwd unknown /proc/60427/task/60780/cwd (readlink: Permission denied) 2024-03-07T07:43:30.714 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60780 ceph-osd ceph rtd unknown /proc/60427/task/60780/root (readlink: Permission denied) 2024-03-07T07:43:30.714 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60780 ceph-osd ceph txt unknown /proc/60427/task/60780/exe (readlink: Permission denied) 2024-03-07T07:43:30.714 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60780 ceph-osd ceph NOFD /proc/60427/task/60780/fd (opendir: Permission denied) 2024-03-07T07:43:30.714 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60801 ceph-osd ceph cwd unknown /proc/60427/task/60801/cwd (readlink: Permission denied) 2024-03-07T07:43:30.714 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60801 ceph-osd ceph rtd unknown /proc/60427/task/60801/root (readlink: Permission denied) 2024-03-07T07:43:30.714 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60801 ceph-osd ceph txt unknown /proc/60427/task/60801/exe (readlink: Permission denied) 2024-03-07T07:43:30.714 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60801 ceph-osd ceph NOFD /proc/60427/task/60801/fd (opendir: Permission denied) 2024-03-07T07:43:30.714 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60822 ceph-osd ceph cwd unknown /proc/60427/task/60822/cwd (readlink: Permission denied) 2024-03-07T07:43:30.715 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60822 ceph-osd ceph rtd unknown /proc/60427/task/60822/root (readlink: Permission denied) 2024-03-07T07:43:30.715 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60822 ceph-osd ceph txt unknown /proc/60427/task/60822/exe (readlink: Permission denied) 2024-03-07T07:43:30.715 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60822 ceph-osd ceph NOFD /proc/60427/task/60822/fd (opendir: Permission denied) 2024-03-07T07:43:30.715 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60832 ceph-osd ceph cwd unknown /proc/60427/task/60832/cwd (readlink: Permission denied) 2024-03-07T07:43:30.715 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60832 ceph-osd ceph rtd unknown /proc/60427/task/60832/root (readlink: Permission denied) 2024-03-07T07:43:30.715 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60832 ceph-osd ceph txt unknown /proc/60427/task/60832/exe (readlink: Permission denied) 2024-03-07T07:43:30.715 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60832 ceph-osd ceph NOFD /proc/60427/task/60832/fd (opendir: Permission denied) 2024-03-07T07:43:30.715 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60834 ceph-osd ceph cwd unknown /proc/60427/task/60834/cwd (readlink: Permission denied) 2024-03-07T07:43:30.715 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60834 ceph-osd ceph rtd unknown /proc/60427/task/60834/root (readlink: Permission denied) 2024-03-07T07:43:30.715 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60834 ceph-osd ceph txt unknown /proc/60427/task/60834/exe (readlink: Permission denied) 2024-03-07T07:43:30.715 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60834 ceph-osd ceph NOFD /proc/60427/task/60834/fd (opendir: Permission denied) 2024-03-07T07:43:30.715 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60835 safe_time ceph cwd unknown /proc/60427/task/60835/cwd (readlink: Permission denied) 2024-03-07T07:43:30.715 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60835 safe_time ceph rtd unknown /proc/60427/task/60835/root (readlink: Permission denied) 2024-03-07T07:43:30.715 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60835 safe_time ceph txt unknown /proc/60427/task/60835/exe (readlink: Permission denied) 2024-03-07T07:43:30.716 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60835 safe_time ceph NOFD /proc/60427/task/60835/fd (opendir: Permission denied) 2024-03-07T07:43:30.716 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60836 safe_time ceph cwd unknown /proc/60427/task/60836/cwd (readlink: Permission denied) 2024-03-07T07:43:30.716 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60836 safe_time ceph rtd unknown /proc/60427/task/60836/root (readlink: Permission denied) 2024-03-07T07:43:30.716 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60836 safe_time ceph txt unknown /proc/60427/task/60836/exe (readlink: Permission denied) 2024-03-07T07:43:30.716 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60836 safe_time ceph NOFD /proc/60427/task/60836/fd (opendir: Permission denied) 2024-03-07T07:43:30.716 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60837 safe_time ceph cwd unknown /proc/60427/task/60837/cwd (readlink: Permission denied) 2024-03-07T07:43:30.716 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60837 safe_time ceph rtd unknown /proc/60427/task/60837/root (readlink: Permission denied) 2024-03-07T07:43:30.716 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60837 safe_time ceph txt unknown /proc/60427/task/60837/exe (readlink: Permission denied) 2024-03-07T07:43:30.716 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60837 safe_time ceph NOFD /proc/60427/task/60837/fd (opendir: Permission denied) 2024-03-07T07:43:30.716 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60838 safe_time ceph cwd unknown /proc/60427/task/60838/cwd (readlink: Permission denied) 2024-03-07T07:43:30.716 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60838 safe_time ceph rtd unknown /proc/60427/task/60838/root (readlink: Permission denied) 2024-03-07T07:43:30.716 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60838 safe_time ceph txt unknown /proc/60427/task/60838/exe (readlink: Permission denied) 2024-03-07T07:43:30.716 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60838 safe_time ceph NOFD /proc/60427/task/60838/fd (opendir: Permission denied) 2024-03-07T07:43:30.717 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60839 fn_anonym ceph cwd unknown /proc/60427/task/60839/cwd (readlink: Permission denied) 2024-03-07T07:43:30.717 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60839 fn_anonym ceph rtd unknown /proc/60427/task/60839/root (readlink: Permission denied) 2024-03-07T07:43:30.717 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60839 fn_anonym ceph txt unknown /proc/60427/task/60839/exe (readlink: Permission denied) 2024-03-07T07:43:30.717 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60839 fn_anonym ceph NOFD /proc/60427/task/60839/fd (opendir: Permission denied) 2024-03-07T07:43:30.717 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60842 rocksdb:l ceph cwd unknown /proc/60427/task/60842/cwd (readlink: Permission denied) 2024-03-07T07:43:30.717 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60842 rocksdb:l ceph rtd unknown /proc/60427/task/60842/root (readlink: Permission denied) 2024-03-07T07:43:30.717 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60842 rocksdb:l ceph txt unknown /proc/60427/task/60842/exe (readlink: Permission denied) 2024-03-07T07:43:30.717 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60842 rocksdb:l ceph NOFD /proc/60427/task/60842/fd (opendir: Permission denied) 2024-03-07T07:43:30.717 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60843 rocksdb:l ceph cwd unknown /proc/60427/task/60843/cwd (readlink: Permission denied) 2024-03-07T07:43:30.717 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60843 rocksdb:l ceph rtd unknown /proc/60427/task/60843/root (readlink: Permission denied) 2024-03-07T07:43:30.717 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60843 rocksdb:l ceph txt unknown /proc/60427/task/60843/exe (readlink: Permission denied) 2024-03-07T07:43:30.717 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60843 rocksdb:l ceph NOFD /proc/60427/task/60843/fd (opendir: Permission denied) 2024-03-07T07:43:30.717 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60844 rocksdb:l ceph cwd unknown /proc/60427/task/60844/cwd (readlink: Permission denied) 2024-03-07T07:43:30.717 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60844 rocksdb:l ceph rtd unknown /proc/60427/task/60844/root (readlink: Permission denied) 2024-03-07T07:43:30.718 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60844 rocksdb:l ceph txt unknown /proc/60427/task/60844/exe (readlink: Permission denied) 2024-03-07T07:43:30.718 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60844 rocksdb:l ceph NOFD /proc/60427/task/60844/fd (opendir: Permission denied) 2024-03-07T07:43:30.718 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60845 rocksdb:h ceph cwd unknown /proc/60427/task/60845/cwd (readlink: Permission denied) 2024-03-07T07:43:30.718 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60845 rocksdb:h ceph rtd unknown /proc/60427/task/60845/root (readlink: Permission denied) 2024-03-07T07:43:30.718 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60845 rocksdb:h ceph txt unknown /proc/60427/task/60845/exe (readlink: Permission denied) 2024-03-07T07:43:30.718 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 60845 rocksdb:h ceph NOFD /proc/60427/task/60845/fd (opendir: Permission denied) 2024-03-07T07:43:30.718 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61342 bstore_ai ceph cwd unknown /proc/60427/task/61342/cwd (readlink: Permission denied) 2024-03-07T07:43:30.718 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61342 bstore_ai ceph rtd unknown /proc/60427/task/61342/root (readlink: Permission denied) 2024-03-07T07:43:30.718 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61342 bstore_ai ceph txt unknown /proc/60427/task/61342/exe (readlink: Permission denied) 2024-03-07T07:43:30.718 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61342 bstore_ai ceph NOFD /proc/60427/task/61342/fd (opendir: Permission denied) 2024-03-07T07:43:30.718 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61527 bstore_ai ceph cwd unknown /proc/60427/task/61527/cwd (readlink: Permission denied) 2024-03-07T07:43:30.718 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61527 bstore_ai ceph rtd unknown /proc/60427/task/61527/root (readlink: Permission denied) 2024-03-07T07:43:30.718 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61527 bstore_ai ceph txt unknown /proc/60427/task/61527/exe (readlink: Permission denied) 2024-03-07T07:43:30.718 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61527 bstore_ai ceph NOFD /proc/60427/task/61527/fd (opendir: Permission denied) 2024-03-07T07:43:30.719 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61709 ceph-osd ceph cwd unknown /proc/60427/task/61709/cwd (readlink: Permission denied) 2024-03-07T07:43:30.719 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61709 ceph-osd ceph rtd unknown /proc/60427/task/61709/root (readlink: Permission denied) 2024-03-07T07:43:30.719 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61709 ceph-osd ceph txt unknown /proc/60427/task/61709/exe (readlink: Permission denied) 2024-03-07T07:43:30.719 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61709 ceph-osd ceph NOFD /proc/60427/task/61709/fd (opendir: Permission denied) 2024-03-07T07:43:30.719 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61710 cfin ceph cwd unknown /proc/60427/task/61710/cwd (readlink: Permission denied) 2024-03-07T07:43:30.719 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61710 cfin ceph rtd unknown /proc/60427/task/61710/root (readlink: Permission denied) 2024-03-07T07:43:30.719 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61710 cfin ceph txt unknown /proc/60427/task/61710/exe (readlink: Permission denied) 2024-03-07T07:43:30.719 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61710 cfin ceph NOFD /proc/60427/task/61710/fd (opendir: Permission denied) 2024-03-07T07:43:30.719 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61711 bstore_kv ceph cwd unknown /proc/60427/task/61711/cwd (readlink: Permission denied) 2024-03-07T07:43:30.719 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61711 bstore_kv ceph rtd unknown /proc/60427/task/61711/root (readlink: Permission denied) 2024-03-07T07:43:30.719 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61711 bstore_kv ceph txt unknown /proc/60427/task/61711/exe (readlink: Permission denied) 2024-03-07T07:43:30.719 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61711 bstore_kv ceph NOFD /proc/60427/task/61711/fd (opendir: Permission denied) 2024-03-07T07:43:30.719 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61712 bstore_kv ceph cwd unknown /proc/60427/task/61712/cwd (readlink: Permission denied) 2024-03-07T07:43:30.719 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61712 bstore_kv ceph rtd unknown /proc/60427/task/61712/root (readlink: Permission denied) 2024-03-07T07:43:30.720 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61712 bstore_kv ceph txt unknown /proc/60427/task/61712/exe (readlink: Permission denied) 2024-03-07T07:43:30.720 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61712 bstore_kv ceph NOFD /proc/60427/task/61712/fd (opendir: Permission denied) 2024-03-07T07:43:30.720 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61713 bstore_me ceph cwd unknown /proc/60427/task/61713/cwd (readlink: Permission denied) 2024-03-07T07:43:30.720 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61713 bstore_me ceph rtd unknown /proc/60427/task/61713/root (readlink: Permission denied) 2024-03-07T07:43:30.720 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61713 bstore_me ceph txt unknown /proc/60427/task/61713/exe (readlink: Permission denied) 2024-03-07T07:43:30.720 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61713 bstore_me ceph NOFD /proc/60427/task/61713/fd (opendir: Permission denied) 2024-03-07T07:43:30.720 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61714 ms_dispat ceph cwd unknown /proc/60427/task/61714/cwd (readlink: Permission denied) 2024-03-07T07:43:30.720 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61714 ms_dispat ceph rtd unknown /proc/60427/task/61714/root (readlink: Permission denied) 2024-03-07T07:43:30.720 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61714 ms_dispat ceph txt unknown /proc/60427/task/61714/exe (readlink: Permission denied) 2024-03-07T07:43:30.720 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61714 ms_dispat ceph NOFD /proc/60427/task/61714/fd (opendir: Permission denied) 2024-03-07T07:43:30.720 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61715 ms_local ceph cwd unknown /proc/60427/task/61715/cwd (readlink: Permission denied) 2024-03-07T07:43:30.720 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61715 ms_local ceph rtd unknown /proc/60427/task/61715/root (readlink: Permission denied) 2024-03-07T07:43:30.720 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61715 ms_local ceph txt unknown /proc/60427/task/61715/exe (readlink: Permission denied) 2024-03-07T07:43:30.720 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61715 ms_local ceph NOFD /proc/60427/task/61715/fd (opendir: Permission denied) 2024-03-07T07:43:30.721 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61716 safe_time ceph cwd unknown /proc/60427/task/61716/cwd (readlink: Permission denied) 2024-03-07T07:43:30.721 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61716 safe_time ceph rtd unknown /proc/60427/task/61716/root (readlink: Permission denied) 2024-03-07T07:43:30.721 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61716 safe_time ceph txt unknown /proc/60427/task/61716/exe (readlink: Permission denied) 2024-03-07T07:43:30.721 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61716 safe_time ceph NOFD /proc/60427/task/61716/fd (opendir: Permission denied) 2024-03-07T07:43:30.721 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61717 safe_time ceph cwd unknown /proc/60427/task/61717/cwd (readlink: Permission denied) 2024-03-07T07:43:30.721 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61717 safe_time ceph rtd unknown /proc/60427/task/61717/root (readlink: Permission denied) 2024-03-07T07:43:30.721 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61717 safe_time ceph txt unknown /proc/60427/task/61717/exe (readlink: Permission denied) 2024-03-07T07:43:30.721 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61717 safe_time ceph NOFD /proc/60427/task/61717/fd (opendir: Permission denied) 2024-03-07T07:43:30.721 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61718 ms_dispat ceph cwd unknown /proc/60427/task/61718/cwd (readlink: Permission denied) 2024-03-07T07:43:30.721 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61718 ms_dispat ceph rtd unknown /proc/60427/task/61718/root (readlink: Permission denied) 2024-03-07T07:43:30.721 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61718 ms_dispat ceph txt unknown /proc/60427/task/61718/exe (readlink: Permission denied) 2024-03-07T07:43:30.721 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61718 ms_dispat ceph NOFD /proc/60427/task/61718/fd (opendir: Permission denied) 2024-03-07T07:43:30.721 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61719 ms_local ceph cwd unknown /proc/60427/task/61719/cwd (readlink: Permission denied) 2024-03-07T07:43:30.721 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61719 ms_local ceph rtd unknown /proc/60427/task/61719/root (readlink: Permission denied) 2024-03-07T07:43:30.722 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61719 ms_local ceph txt unknown /proc/60427/task/61719/exe (readlink: Permission denied) 2024-03-07T07:43:30.722 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61719 ms_local ceph NOFD /proc/60427/task/61719/fd (opendir: Permission denied) 2024-03-07T07:43:30.722 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61720 ms_dispat ceph cwd unknown /proc/60427/task/61720/cwd (readlink: Permission denied) 2024-03-07T07:43:30.722 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61720 ms_dispat ceph rtd unknown /proc/60427/task/61720/root (readlink: Permission denied) 2024-03-07T07:43:30.722 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61720 ms_dispat ceph txt unknown /proc/60427/task/61720/exe (readlink: Permission denied) 2024-03-07T07:43:30.722 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61720 ms_dispat ceph NOFD /proc/60427/task/61720/fd (opendir: Permission denied) 2024-03-07T07:43:30.722 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61721 ms_local ceph cwd unknown /proc/60427/task/61721/cwd (readlink: Permission denied) 2024-03-07T07:43:30.722 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61721 ms_local ceph rtd unknown /proc/60427/task/61721/root (readlink: Permission denied) 2024-03-07T07:43:30.722 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61721 ms_local ceph txt unknown /proc/60427/task/61721/exe (readlink: Permission denied) 2024-03-07T07:43:30.722 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61721 ms_local ceph NOFD /proc/60427/task/61721/fd (opendir: Permission denied) 2024-03-07T07:43:30.722 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61722 ms_dispat ceph cwd unknown /proc/60427/task/61722/cwd (readlink: Permission denied) 2024-03-07T07:43:30.722 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61722 ms_dispat ceph rtd unknown /proc/60427/task/61722/root (readlink: Permission denied) 2024-03-07T07:43:30.722 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61722 ms_dispat ceph txt unknown /proc/60427/task/61722/exe (readlink: Permission denied) 2024-03-07T07:43:30.723 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61722 ms_dispat ceph NOFD /proc/60427/task/61722/fd (opendir: Permission denied) 2024-03-07T07:43:30.723 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61723 ms_local ceph cwd unknown /proc/60427/task/61723/cwd (readlink: Permission denied) 2024-03-07T07:43:30.723 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61723 ms_local ceph rtd unknown /proc/60427/task/61723/root (readlink: Permission denied) 2024-03-07T07:43:30.723 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61723 ms_local ceph txt unknown /proc/60427/task/61723/exe (readlink: Permission denied) 2024-03-07T07:43:30.723 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61723 ms_local ceph NOFD /proc/60427/task/61723/fd (opendir: Permission denied) 2024-03-07T07:43:30.723 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61724 ms_dispat ceph cwd unknown /proc/60427/task/61724/cwd (readlink: Permission denied) 2024-03-07T07:43:30.723 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61724 ms_dispat ceph rtd unknown /proc/60427/task/61724/root (readlink: Permission denied) 2024-03-07T07:43:30.723 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61724 ms_dispat ceph txt unknown /proc/60427/task/61724/exe (readlink: Permission denied) 2024-03-07T07:43:30.723 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61724 ms_dispat ceph NOFD /proc/60427/task/61724/fd (opendir: Permission denied) 2024-03-07T07:43:30.723 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61725 ms_local ceph cwd unknown /proc/60427/task/61725/cwd (readlink: Permission denied) 2024-03-07T07:43:30.723 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61725 ms_local ceph rtd unknown /proc/60427/task/61725/root (readlink: Permission denied) 2024-03-07T07:43:30.723 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61725 ms_local ceph txt unknown /proc/60427/task/61725/exe (readlink: Permission denied) 2024-03-07T07:43:30.723 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61725 ms_local ceph NOFD /proc/60427/task/61725/fd (opendir: Permission denied) 2024-03-07T07:43:30.723 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61726 ms_dispat ceph cwd unknown /proc/60427/task/61726/cwd (readlink: Permission denied) 2024-03-07T07:43:30.724 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61726 ms_dispat ceph rtd unknown /proc/60427/task/61726/root (readlink: Permission denied) 2024-03-07T07:43:30.724 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61726 ms_dispat ceph txt unknown /proc/60427/task/61726/exe (readlink: Permission denied) 2024-03-07T07:43:30.724 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61726 ms_dispat ceph NOFD /proc/60427/task/61726/fd (opendir: Permission denied) 2024-03-07T07:43:30.724 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61727 ms_local ceph cwd unknown /proc/60427/task/61727/cwd (readlink: Permission denied) 2024-03-07T07:43:30.724 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61727 ms_local ceph rtd unknown /proc/60427/task/61727/root (readlink: Permission denied) 2024-03-07T07:43:30.724 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61727 ms_local ceph txt unknown /proc/60427/task/61727/exe (readlink: Permission denied) 2024-03-07T07:43:30.724 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61727 ms_local ceph NOFD /proc/60427/task/61727/fd (opendir: Permission denied) 2024-03-07T07:43:30.724 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61728 ms_dispat ceph cwd unknown /proc/60427/task/61728/cwd (readlink: Permission denied) 2024-03-07T07:43:30.724 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61728 ms_dispat ceph rtd unknown /proc/60427/task/61728/root (readlink: Permission denied) 2024-03-07T07:43:30.724 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61728 ms_dispat ceph txt unknown /proc/60427/task/61728/exe (readlink: Permission denied) 2024-03-07T07:43:30.724 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61728 ms_dispat ceph NOFD /proc/60427/task/61728/fd (opendir: Permission denied) 2024-03-07T07:43:30.724 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61729 ms_local ceph cwd unknown /proc/60427/task/61729/cwd (readlink: Permission denied) 2024-03-07T07:43:30.724 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61729 ms_local ceph rtd unknown /proc/60427/task/61729/root (readlink: Permission denied) 2024-03-07T07:43:30.724 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61729 ms_local ceph txt unknown /proc/60427/task/61729/exe (readlink: Permission denied) 2024-03-07T07:43:30.725 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61729 ms_local ceph NOFD /proc/60427/task/61729/fd (opendir: Permission denied) 2024-03-07T07:43:30.725 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61730 fn_anonym ceph cwd unknown /proc/60427/task/61730/cwd (readlink: Permission denied) 2024-03-07T07:43:30.725 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61730 fn_anonym ceph rtd unknown /proc/60427/task/61730/root (readlink: Permission denied) 2024-03-07T07:43:30.725 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61730 fn_anonym ceph txt unknown /proc/60427/task/61730/exe (readlink: Permission denied) 2024-03-07T07:43:30.725 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61730 fn_anonym ceph NOFD /proc/60427/task/61730/fd (opendir: Permission denied) 2024-03-07T07:43:30.725 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61731 finisher ceph cwd unknown /proc/60427/task/61731/cwd (readlink: Permission denied) 2024-03-07T07:43:30.725 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61731 finisher ceph rtd unknown /proc/60427/task/61731/root (readlink: Permission denied) 2024-03-07T07:43:30.725 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61731 finisher ceph txt unknown /proc/60427/task/61731/exe (readlink: Permission denied) 2024-03-07T07:43:30.725 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61731 finisher ceph NOFD /proc/60427/task/61731/fd (opendir: Permission denied) 2024-03-07T07:43:30.725 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61732 safe_time ceph cwd unknown /proc/60427/task/61732/cwd (readlink: Permission denied) 2024-03-07T07:43:30.725 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61732 safe_time ceph rtd unknown /proc/60427/task/61732/root (readlink: Permission denied) 2024-03-07T07:43:30.725 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61732 safe_time ceph txt unknown /proc/60427/task/61732/exe (readlink: Permission denied) 2024-03-07T07:43:30.725 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61732 safe_time ceph NOFD /proc/60427/task/61732/fd (opendir: Permission denied) 2024-03-07T07:43:30.726 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61733 safe_time ceph cwd unknown /proc/60427/task/61733/cwd (readlink: Permission denied) 2024-03-07T07:43:30.726 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61733 safe_time ceph rtd unknown /proc/60427/task/61733/root (readlink: Permission denied) 2024-03-07T07:43:30.726 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61733 safe_time ceph txt unknown /proc/60427/task/61733/exe (readlink: Permission denied) 2024-03-07T07:43:30.726 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61733 safe_time ceph NOFD /proc/60427/task/61733/fd (opendir: Permission denied) 2024-03-07T07:43:30.726 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61734 ceph-osd ceph cwd unknown /proc/60427/task/61734/cwd (readlink: Permission denied) 2024-03-07T07:43:30.726 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61734 ceph-osd ceph rtd unknown /proc/60427/task/61734/root (readlink: Permission denied) 2024-03-07T07:43:30.726 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61734 ceph-osd ceph txt unknown /proc/60427/task/61734/exe (readlink: Permission denied) 2024-03-07T07:43:30.726 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61734 ceph-osd ceph NOFD /proc/60427/task/61734/fd (opendir: Permission denied) 2024-03-07T07:43:30.726 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61735 osd_srv_a ceph cwd unknown /proc/60427/task/61735/cwd (readlink: Permission denied) 2024-03-07T07:43:30.726 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61735 osd_srv_a ceph rtd unknown /proc/60427/task/61735/root (readlink: Permission denied) 2024-03-07T07:43:30.726 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61735 osd_srv_a ceph txt unknown /proc/60427/task/61735/exe (readlink: Permission denied) 2024-03-07T07:43:30.726 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61735 osd_srv_a ceph NOFD /proc/60427/task/61735/fd (opendir: Permission denied) 2024-03-07T07:43:30.726 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61736 tp_osd_tp ceph cwd unknown /proc/60427/task/61736/cwd (readlink: Permission denied) 2024-03-07T07:43:30.727 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61736 tp_osd_tp ceph rtd unknown /proc/60427/task/61736/root (readlink: Permission denied) 2024-03-07T07:43:30.727 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61736 tp_osd_tp ceph txt unknown /proc/60427/task/61736/exe (readlink: Permission denied) 2024-03-07T07:43:30.727 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61736 tp_osd_tp ceph NOFD /proc/60427/task/61736/fd (opendir: Permission denied) 2024-03-07T07:43:30.727 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61737 tp_osd_tp ceph cwd unknown /proc/60427/task/61737/cwd (readlink: Permission denied) 2024-03-07T07:43:30.727 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61737 tp_osd_tp ceph rtd unknown /proc/60427/task/61737/root (readlink: Permission denied) 2024-03-07T07:43:30.727 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61737 tp_osd_tp ceph txt unknown /proc/60427/task/61737/exe (readlink: Permission denied) 2024-03-07T07:43:30.727 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61737 tp_osd_tp ceph NOFD /proc/60427/task/61737/fd (opendir: Permission denied) 2024-03-07T07:43:30.727 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61738 tp_osd_tp ceph cwd unknown /proc/60427/task/61738/cwd (readlink: Permission denied) 2024-03-07T07:43:30.727 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61738 tp_osd_tp ceph rtd unknown /proc/60427/task/61738/root (readlink: Permission denied) 2024-03-07T07:43:30.727 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61738 tp_osd_tp ceph txt unknown /proc/60427/task/61738/exe (readlink: Permission denied) 2024-03-07T07:43:30.727 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61738 tp_osd_tp ceph NOFD /proc/60427/task/61738/fd (opendir: Permission denied) 2024-03-07T07:43:30.727 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61739 tp_osd_tp ceph cwd unknown /proc/60427/task/61739/cwd (readlink: Permission denied) 2024-03-07T07:43:30.727 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61739 tp_osd_tp ceph rtd unknown /proc/60427/task/61739/root (readlink: Permission denied) 2024-03-07T07:43:30.727 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61739 tp_osd_tp ceph txt unknown /proc/60427/task/61739/exe (readlink: Permission denied) 2024-03-07T07:43:30.727 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61739 tp_osd_tp ceph NOFD /proc/60427/task/61739/fd (opendir: Permission denied) 2024-03-07T07:43:30.728 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61740 tp_osd_tp ceph cwd unknown /proc/60427/task/61740/cwd (readlink: Permission denied) 2024-03-07T07:43:30.728 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61740 tp_osd_tp ceph rtd unknown /proc/60427/task/61740/root (readlink: Permission denied) 2024-03-07T07:43:30.728 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61740 tp_osd_tp ceph txt unknown /proc/60427/task/61740/exe (readlink: Permission denied) 2024-03-07T07:43:30.728 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61740 tp_osd_tp ceph NOFD /proc/60427/task/61740/fd (opendir: Permission denied) 2024-03-07T07:43:30.728 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61741 tp_osd_tp ceph cwd unknown /proc/60427/task/61741/cwd (readlink: Permission denied) 2024-03-07T07:43:30.728 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61741 tp_osd_tp ceph rtd unknown /proc/60427/task/61741/root (readlink: Permission denied) 2024-03-07T07:43:30.728 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61741 tp_osd_tp ceph txt unknown /proc/60427/task/61741/exe (readlink: Permission denied) 2024-03-07T07:43:30.728 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61741 tp_osd_tp ceph NOFD /proc/60427/task/61741/fd (opendir: Permission denied) 2024-03-07T07:43:30.728 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61742 tp_osd_tp ceph cwd unknown /proc/60427/task/61742/cwd (readlink: Permission denied) 2024-03-07T07:43:30.728 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61742 tp_osd_tp ceph rtd unknown /proc/60427/task/61742/root (readlink: Permission denied) 2024-03-07T07:43:30.728 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61742 tp_osd_tp ceph txt unknown /proc/60427/task/61742/exe (readlink: Permission denied) 2024-03-07T07:43:30.728 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61742 tp_osd_tp ceph NOFD /proc/60427/task/61742/fd (opendir: Permission denied) 2024-03-07T07:43:30.728 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61743 tp_osd_tp ceph cwd unknown /proc/60427/task/61743/cwd (readlink: Permission denied) 2024-03-07T07:43:30.729 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61743 tp_osd_tp ceph rtd unknown /proc/60427/task/61743/root (readlink: Permission denied) 2024-03-07T07:43:30.729 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61743 tp_osd_tp ceph txt unknown /proc/60427/task/61743/exe (readlink: Permission denied) 2024-03-07T07:43:30.729 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61743 tp_osd_tp ceph NOFD /proc/60427/task/61743/fd (opendir: Permission denied) 2024-03-07T07:43:30.729 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61744 tp_osd_tp ceph cwd unknown /proc/60427/task/61744/cwd (readlink: Permission denied) 2024-03-07T07:43:30.729 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61744 tp_osd_tp ceph rtd unknown /proc/60427/task/61744/root (readlink: Permission denied) 2024-03-07T07:43:30.729 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61744 tp_osd_tp ceph txt unknown /proc/60427/task/61744/exe (readlink: Permission denied) 2024-03-07T07:43:30.729 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61744 tp_osd_tp ceph NOFD /proc/60427/task/61744/fd (opendir: Permission denied) 2024-03-07T07:43:30.729 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61745 tp_osd_tp ceph cwd unknown /proc/60427/task/61745/cwd (readlink: Permission denied) 2024-03-07T07:43:30.729 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61745 tp_osd_tp ceph rtd unknown /proc/60427/task/61745/root (readlink: Permission denied) 2024-03-07T07:43:30.729 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61745 tp_osd_tp ceph txt unknown /proc/60427/task/61745/exe (readlink: Permission denied) 2024-03-07T07:43:30.729 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61745 tp_osd_tp ceph NOFD /proc/60427/task/61745/fd (opendir: Permission denied) 2024-03-07T07:43:30.729 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61746 tp_osd_tp ceph cwd unknown /proc/60427/task/61746/cwd (readlink: Permission denied) 2024-03-07T07:43:30.729 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61746 tp_osd_tp ceph rtd unknown /proc/60427/task/61746/root (readlink: Permission denied) 2024-03-07T07:43:30.729 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61746 tp_osd_tp ceph txt unknown /proc/60427/task/61746/exe (readlink: Permission denied) 2024-03-07T07:43:30.730 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61746 tp_osd_tp ceph NOFD /proc/60427/task/61746/fd (opendir: Permission denied) 2024-03-07T07:43:30.730 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61747 tp_osd_tp ceph cwd unknown /proc/60427/task/61747/cwd (readlink: Permission denied) 2024-03-07T07:43:30.730 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61747 tp_osd_tp ceph rtd unknown /proc/60427/task/61747/root (readlink: Permission denied) 2024-03-07T07:43:30.730 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61747 tp_osd_tp ceph txt unknown /proc/60427/task/61747/exe (readlink: Permission denied) 2024-03-07T07:43:30.730 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61747 tp_osd_tp ceph NOFD /proc/60427/task/61747/fd (opendir: Permission denied) 2024-03-07T07:43:30.730 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61748 tp_osd_tp ceph cwd unknown /proc/60427/task/61748/cwd (readlink: Permission denied) 2024-03-07T07:43:30.730 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61748 tp_osd_tp ceph rtd unknown /proc/60427/task/61748/root (readlink: Permission denied) 2024-03-07T07:43:30.730 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61748 tp_osd_tp ceph txt unknown /proc/60427/task/61748/exe (readlink: Permission denied) 2024-03-07T07:43:30.730 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61748 tp_osd_tp ceph NOFD /proc/60427/task/61748/fd (opendir: Permission denied) 2024-03-07T07:43:30.730 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61749 tp_osd_tp ceph cwd unknown /proc/60427/task/61749/cwd (readlink: Permission denied) 2024-03-07T07:43:30.730 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61749 tp_osd_tp ceph rtd unknown /proc/60427/task/61749/root (readlink: Permission denied) 2024-03-07T07:43:30.730 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61749 tp_osd_tp ceph txt unknown /proc/60427/task/61749/exe (readlink: Permission denied) 2024-03-07T07:43:30.730 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61749 tp_osd_tp ceph NOFD /proc/60427/task/61749/fd (opendir: Permission denied) 2024-03-07T07:43:30.731 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61750 tp_osd_tp ceph cwd unknown /proc/60427/task/61750/cwd (readlink: Permission denied) 2024-03-07T07:43:30.731 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61750 tp_osd_tp ceph rtd unknown /proc/60427/task/61750/root (readlink: Permission denied) 2024-03-07T07:43:30.731 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61750 tp_osd_tp ceph txt unknown /proc/60427/task/61750/exe (readlink: Permission denied) 2024-03-07T07:43:30.731 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61750 tp_osd_tp ceph NOFD /proc/60427/task/61750/fd (opendir: Permission denied) 2024-03-07T07:43:30.731 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61751 tp_osd_tp ceph cwd unknown /proc/60427/task/61751/cwd (readlink: Permission denied) 2024-03-07T07:43:30.731 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61751 tp_osd_tp ceph rtd unknown /proc/60427/task/61751/root (readlink: Permission denied) 2024-03-07T07:43:30.731 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61751 tp_osd_tp ceph txt unknown /proc/60427/task/61751/exe (readlink: Permission denied) 2024-03-07T07:43:30.731 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61751 tp_osd_tp ceph NOFD /proc/60427/task/61751/fd (opendir: Permission denied) 2024-03-07T07:43:30.731 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61752 osd_srv_h ceph cwd unknown /proc/60427/task/61752/cwd (readlink: Permission denied) 2024-03-07T07:43:30.731 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61752 osd_srv_h ceph rtd unknown /proc/60427/task/61752/root (readlink: Permission denied) 2024-03-07T07:43:30.731 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61752 osd_srv_h ceph txt unknown /proc/60427/task/61752/exe (readlink: Permission denied) 2024-03-07T07:43:30.731 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 60427 61752 osd_srv_h ceph NOFD /proc/60427/task/61752/fd (opendir: Permission denied) 2024-03-07T07:43:30.731 INFO:tasks.workunit.client.0.smithi160.stdout:sudo 61847 root cwd unknown /proc/61847/cwd (readlink: Permission denied) 2024-03-07T07:43:30.731 INFO:tasks.workunit.client.0.smithi160.stdout:sudo 61847 root rtd unknown /proc/61847/root (readlink: Permission denied) 2024-03-07T07:43:30.732 INFO:tasks.workunit.client.0.smithi160.stdout:sudo 61847 root txt unknown /proc/61847/exe (readlink: Permission denied) 2024-03-07T07:43:30.732 INFO:tasks.workunit.client.0.smithi160.stdout:sudo 61847 root NOFD /proc/61847/fd (opendir: Permission denied) 2024-03-07T07:43:30.732 INFO:tasks.workunit.client.0.smithi160.stdout:journalct 61864 root cwd unknown /proc/61864/cwd (readlink: Permission denied) 2024-03-07T07:43:30.732 INFO:tasks.workunit.client.0.smithi160.stdout:journalct 61864 root rtd unknown /proc/61864/root (readlink: Permission denied) 2024-03-07T07:43:30.732 INFO:tasks.workunit.client.0.smithi160.stdout:journalct 61864 root txt unknown /proc/61864/exe (readlink: Permission denied) 2024-03-07T07:43:30.732 INFO:tasks.workunit.client.0.smithi160.stdout:journalct 61864 root NOFD /proc/61864/fd (opendir: Permission denied) 2024-03-07T07:43:30.732 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/0 65215 root cwd unknown /proc/65215/cwd (readlink: Permission denied) 2024-03-07T07:43:30.732 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/0 65215 root rtd unknown /proc/65215/root (readlink: Permission denied) 2024-03-07T07:43:30.732 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/0 65215 root txt unknown /proc/65215/exe (readlink: Permission denied) 2024-03-07T07:43:30.732 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/0 65215 root NOFD /proc/65215/fd (opendir: Permission denied) 2024-03-07T07:43:30.732 INFO:tasks.workunit.client.0.smithi160.stdout:conmon 67330 root cwd unknown /proc/67330/cwd (readlink: Permission denied) 2024-03-07T07:43:30.732 INFO:tasks.workunit.client.0.smithi160.stdout:conmon 67330 root rtd unknown /proc/67330/root (readlink: Permission denied) 2024-03-07T07:43:30.732 INFO:tasks.workunit.client.0.smithi160.stdout:conmon 67330 root txt unknown /proc/67330/exe (readlink: Permission denied) 2024-03-07T07:43:30.732 INFO:tasks.workunit.client.0.smithi160.stdout:conmon 67330 root NOFD /proc/67330/fd (opendir: Permission denied) 2024-03-07T07:43:30.733 INFO:tasks.workunit.client.0.smithi160.stdout:podman-in 67332 root cwd unknown /proc/67332/cwd (readlink: Permission denied) 2024-03-07T07:43:30.733 INFO:tasks.workunit.client.0.smithi160.stdout:podman-in 67332 root rtd unknown /proc/67332/root (readlink: Permission denied) 2024-03-07T07:43:30.733 INFO:tasks.workunit.client.0.smithi160.stdout:podman-in 67332 root txt unknown /proc/67332/exe (readlink: Permission denied) 2024-03-07T07:43:30.733 INFO:tasks.workunit.client.0.smithi160.stdout:podman-in 67332 root NOFD /proc/67332/fd (opendir: Permission denied) 2024-03-07T07:43:30.733 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 ceph cwd unknown /proc/67334/cwd (readlink: Permission denied) 2024-03-07T07:43:30.733 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 ceph rtd unknown /proc/67334/root (readlink: Permission denied) 2024-03-07T07:43:30.733 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 ceph txt unknown /proc/67334/exe (readlink: Permission denied) 2024-03-07T07:43:30.733 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 ceph NOFD /proc/67334/fd (opendir: Permission denied) 2024-03-07T07:43:30.733 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67335 log ceph cwd unknown /proc/67334/task/67335/cwd (readlink: Permission denied) 2024-03-07T07:43:30.733 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67335 log ceph rtd unknown /proc/67334/task/67335/root (readlink: Permission denied) 2024-03-07T07:43:30.733 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67335 log ceph txt unknown /proc/67334/task/67335/exe (readlink: Permission denied) 2024-03-07T07:43:30.733 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67335 log ceph NOFD /proc/67334/task/67335/fd (opendir: Permission denied) 2024-03-07T07:43:30.733 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67337 msgr-work ceph cwd unknown /proc/67334/task/67337/cwd (readlink: Permission denied) 2024-03-07T07:43:30.733 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67337 msgr-work ceph rtd unknown /proc/67334/task/67337/root (readlink: Permission denied) 2024-03-07T07:43:30.733 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67337 msgr-work ceph txt unknown /proc/67334/task/67337/exe (readlink: Permission denied) 2024-03-07T07:43:30.734 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67337 msgr-work ceph NOFD /proc/67334/task/67337/fd (opendir: Permission denied) 2024-03-07T07:43:30.734 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67338 msgr-work ceph cwd unknown /proc/67334/task/67338/cwd (readlink: Permission denied) 2024-03-07T07:43:30.734 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67338 msgr-work ceph rtd unknown /proc/67334/task/67338/root (readlink: Permission denied) 2024-03-07T07:43:30.734 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67338 msgr-work ceph txt unknown /proc/67334/task/67338/exe (readlink: Permission denied) 2024-03-07T07:43:30.734 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67338 msgr-work ceph NOFD /proc/67334/task/67338/fd (opendir: Permission denied) 2024-03-07T07:43:30.734 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67339 msgr-work ceph cwd unknown /proc/67334/task/67339/cwd (readlink: Permission denied) 2024-03-07T07:43:30.734 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67339 msgr-work ceph rtd unknown /proc/67334/task/67339/root (readlink: Permission denied) 2024-03-07T07:43:30.734 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67339 msgr-work ceph txt unknown /proc/67334/task/67339/exe (readlink: Permission denied) 2024-03-07T07:43:30.734 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67339 msgr-work ceph NOFD /proc/67334/task/67339/fd (opendir: Permission denied) 2024-03-07T07:43:30.734 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67343 service ceph cwd unknown /proc/67334/task/67343/cwd (readlink: Permission denied) 2024-03-07T07:43:30.734 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67343 service ceph rtd unknown /proc/67334/task/67343/root (readlink: Permission denied) 2024-03-07T07:43:30.734 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67343 service ceph txt unknown /proc/67334/task/67343/exe (readlink: Permission denied) 2024-03-07T07:43:30.734 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67343 service ceph NOFD /proc/67334/task/67343/fd (opendir: Permission denied) 2024-03-07T07:43:30.734 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67344 admin_soc ceph cwd unknown /proc/67334/task/67344/cwd (readlink: Permission denied) 2024-03-07T07:43:30.735 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67344 admin_soc ceph rtd unknown /proc/67334/task/67344/root (readlink: Permission denied) 2024-03-07T07:43:30.735 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67344 admin_soc ceph txt unknown /proc/67334/task/67344/exe (readlink: Permission denied) 2024-03-07T07:43:30.735 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67344 admin_soc ceph NOFD /proc/67334/task/67344/fd (opendir: Permission denied) 2024-03-07T07:43:30.735 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67487 signal_ha ceph cwd unknown /proc/67334/task/67487/cwd (readlink: Permission denied) 2024-03-07T07:43:30.735 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67487 signal_ha ceph rtd unknown /proc/67334/task/67487/root (readlink: Permission denied) 2024-03-07T07:43:30.735 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67487 signal_ha ceph txt unknown /proc/67334/task/67487/exe (readlink: Permission denied) 2024-03-07T07:43:30.735 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67487 signal_ha ceph NOFD /proc/67334/task/67487/fd (opendir: Permission denied) 2024-03-07T07:43:30.735 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67488 io_contex ceph cwd unknown /proc/67334/task/67488/cwd (readlink: Permission denied) 2024-03-07T07:43:30.735 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67488 io_contex ceph rtd unknown /proc/67334/task/67488/root (readlink: Permission denied) 2024-03-07T07:43:30.735 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67488 io_contex ceph txt unknown /proc/67334/task/67488/exe (readlink: Permission denied) 2024-03-07T07:43:30.735 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67488 io_contex ceph NOFD /proc/67334/task/67488/fd (opendir: Permission denied) 2024-03-07T07:43:30.735 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67489 io_contex ceph cwd unknown /proc/67334/task/67489/cwd (readlink: Permission denied) 2024-03-07T07:43:30.735 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67489 io_contex ceph rtd unknown /proc/67334/task/67489/root (readlink: Permission denied) 2024-03-07T07:43:30.735 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67489 io_contex ceph txt unknown /proc/67334/task/67489/exe (readlink: Permission denied) 2024-03-07T07:43:30.736 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67489 io_contex ceph NOFD /proc/67334/task/67489/fd (opendir: Permission denied) 2024-03-07T07:43:30.736 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67600 OpHistory ceph cwd unknown /proc/67334/task/67600/cwd (readlink: Permission denied) 2024-03-07T07:43:30.736 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67600 OpHistory ceph rtd unknown /proc/67334/task/67600/root (readlink: Permission denied) 2024-03-07T07:43:30.736 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67600 OpHistory ceph txt unknown /proc/67334/task/67600/exe (readlink: Permission denied) 2024-03-07T07:43:30.736 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67600 OpHistory ceph NOFD /proc/67334/task/67600/fd (opendir: Permission denied) 2024-03-07T07:43:30.736 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67601 ceph_time ceph cwd unknown /proc/67334/task/67601/cwd (readlink: Permission denied) 2024-03-07T07:43:30.736 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67601 ceph_time ceph rtd unknown /proc/67334/task/67601/root (readlink: Permission denied) 2024-03-07T07:43:30.736 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67601 ceph_time ceph txt unknown /proc/67334/task/67601/exe (readlink: Permission denied) 2024-03-07T07:43:30.736 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67601 ceph_time ceph NOFD /proc/67334/task/67601/fd (opendir: Permission denied) 2024-03-07T07:43:30.736 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67778 ceph-osd ceph cwd unknown /proc/67334/task/67778/cwd (readlink: Permission denied) 2024-03-07T07:43:30.736 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67778 ceph-osd ceph rtd unknown /proc/67334/task/67778/root (readlink: Permission denied) 2024-03-07T07:43:30.736 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67778 ceph-osd ceph txt unknown /proc/67334/task/67778/exe (readlink: Permission denied) 2024-03-07T07:43:30.736 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67778 ceph-osd ceph NOFD /proc/67334/task/67778/fd (opendir: Permission denied) 2024-03-07T07:43:30.737 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67779 safe_time ceph cwd unknown /proc/67334/task/67779/cwd (readlink: Permission denied) 2024-03-07T07:43:30.737 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67779 safe_time ceph rtd unknown /proc/67334/task/67779/root (readlink: Permission denied) 2024-03-07T07:43:30.737 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67779 safe_time ceph txt unknown /proc/67334/task/67779/exe (readlink: Permission denied) 2024-03-07T07:43:30.737 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67779 safe_time ceph NOFD /proc/67334/task/67779/fd (opendir: Permission denied) 2024-03-07T07:43:30.737 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67780 safe_time ceph cwd unknown /proc/67334/task/67780/cwd (readlink: Permission denied) 2024-03-07T07:43:30.737 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67780 safe_time ceph rtd unknown /proc/67334/task/67780/root (readlink: Permission denied) 2024-03-07T07:43:30.737 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67780 safe_time ceph txt unknown /proc/67334/task/67780/exe (readlink: Permission denied) 2024-03-07T07:43:30.737 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67780 safe_time ceph NOFD /proc/67334/task/67780/fd (opendir: Permission denied) 2024-03-07T07:43:30.737 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67781 safe_time ceph cwd unknown /proc/67334/task/67781/cwd (readlink: Permission denied) 2024-03-07T07:43:30.737 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67781 safe_time ceph rtd unknown /proc/67334/task/67781/root (readlink: Permission denied) 2024-03-07T07:43:30.737 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67781 safe_time ceph txt unknown /proc/67334/task/67781/exe (readlink: Permission denied) 2024-03-07T07:43:30.737 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67781 safe_time ceph NOFD /proc/67334/task/67781/fd (opendir: Permission denied) 2024-03-07T07:43:30.737 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67782 safe_time ceph cwd unknown /proc/67334/task/67782/cwd (readlink: Permission denied) 2024-03-07T07:43:30.737 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67782 safe_time ceph rtd unknown /proc/67334/task/67782/root (readlink: Permission denied) 2024-03-07T07:43:30.738 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67782 safe_time ceph txt unknown /proc/67334/task/67782/exe (readlink: Permission denied) 2024-03-07T07:43:30.738 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67782 safe_time ceph NOFD /proc/67334/task/67782/fd (opendir: Permission denied) 2024-03-07T07:43:30.738 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67783 fn_anonym ceph cwd unknown /proc/67334/task/67783/cwd (readlink: Permission denied) 2024-03-07T07:43:30.738 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67783 fn_anonym ceph rtd unknown /proc/67334/task/67783/root (readlink: Permission denied) 2024-03-07T07:43:30.738 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67783 fn_anonym ceph txt unknown /proc/67334/task/67783/exe (readlink: Permission denied) 2024-03-07T07:43:30.738 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67783 fn_anonym ceph NOFD /proc/67334/task/67783/fd (opendir: Permission denied) 2024-03-07T07:43:30.738 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67789 rocksdb:l ceph cwd unknown /proc/67334/task/67789/cwd (readlink: Permission denied) 2024-03-07T07:43:30.738 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67789 rocksdb:l ceph rtd unknown /proc/67334/task/67789/root (readlink: Permission denied) 2024-03-07T07:43:30.738 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67789 rocksdb:l ceph txt unknown /proc/67334/task/67789/exe (readlink: Permission denied) 2024-03-07T07:43:30.738 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67789 rocksdb:l ceph NOFD /proc/67334/task/67789/fd (opendir: Permission denied) 2024-03-07T07:43:30.738 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67790 rocksdb:l ceph cwd unknown /proc/67334/task/67790/cwd (readlink: Permission denied) 2024-03-07T07:43:30.738 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67790 rocksdb:l ceph rtd unknown /proc/67334/task/67790/root (readlink: Permission denied) 2024-03-07T07:43:30.738 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67790 rocksdb:l ceph txt unknown /proc/67334/task/67790/exe (readlink: Permission denied) 2024-03-07T07:43:30.738 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67790 rocksdb:l ceph NOFD /proc/67334/task/67790/fd (opendir: Permission denied) 2024-03-07T07:43:30.739 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67791 rocksdb:l ceph cwd unknown /proc/67334/task/67791/cwd (readlink: Permission denied) 2024-03-07T07:43:30.739 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67791 rocksdb:l ceph rtd unknown /proc/67334/task/67791/root (readlink: Permission denied) 2024-03-07T07:43:30.739 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67791 rocksdb:l ceph txt unknown /proc/67334/task/67791/exe (readlink: Permission denied) 2024-03-07T07:43:30.739 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67791 rocksdb:l ceph NOFD /proc/67334/task/67791/fd (opendir: Permission denied) 2024-03-07T07:43:30.739 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67792 rocksdb:h ceph cwd unknown /proc/67334/task/67792/cwd (readlink: Permission denied) 2024-03-07T07:43:30.739 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67792 rocksdb:h ceph rtd unknown /proc/67334/task/67792/root (readlink: Permission denied) 2024-03-07T07:43:30.739 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67792 rocksdb:h ceph txt unknown /proc/67334/task/67792/exe (readlink: Permission denied) 2024-03-07T07:43:30.739 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 67792 rocksdb:h ceph NOFD /proc/67334/task/67792/fd (opendir: Permission denied) 2024-03-07T07:43:30.739 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68271 bstore_ai ceph cwd unknown /proc/67334/task/68271/cwd (readlink: Permission denied) 2024-03-07T07:43:30.739 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68271 bstore_ai ceph rtd unknown /proc/67334/task/68271/root (readlink: Permission denied) 2024-03-07T07:43:30.739 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68271 bstore_ai ceph txt unknown /proc/67334/task/68271/exe (readlink: Permission denied) 2024-03-07T07:43:30.739 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68271 bstore_ai ceph NOFD /proc/67334/task/68271/fd (opendir: Permission denied) 2024-03-07T07:43:30.739 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68455 bstore_ai ceph cwd unknown /proc/67334/task/68455/cwd (readlink: Permission denied) 2024-03-07T07:43:30.739 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68455 bstore_ai ceph rtd unknown /proc/67334/task/68455/root (readlink: Permission denied) 2024-03-07T07:43:30.739 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68455 bstore_ai ceph txt unknown /proc/67334/task/68455/exe (readlink: Permission denied) 2024-03-07T07:43:30.740 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68455 bstore_ai ceph NOFD /proc/67334/task/68455/fd (opendir: Permission denied) 2024-03-07T07:43:30.740 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68637 ceph-osd ceph cwd unknown /proc/67334/task/68637/cwd (readlink: Permission denied) 2024-03-07T07:43:30.740 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68637 ceph-osd ceph rtd unknown /proc/67334/task/68637/root (readlink: Permission denied) 2024-03-07T07:43:30.740 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68637 ceph-osd ceph txt unknown /proc/67334/task/68637/exe (readlink: Permission denied) 2024-03-07T07:43:30.740 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68637 ceph-osd ceph NOFD /proc/67334/task/68637/fd (opendir: Permission denied) 2024-03-07T07:43:30.740 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68638 cfin ceph cwd unknown /proc/67334/task/68638/cwd (readlink: Permission denied) 2024-03-07T07:43:30.740 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68638 cfin ceph rtd unknown /proc/67334/task/68638/root (readlink: Permission denied) 2024-03-07T07:43:30.740 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68638 cfin ceph txt unknown /proc/67334/task/68638/exe (readlink: Permission denied) 2024-03-07T07:43:30.740 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68638 cfin ceph NOFD /proc/67334/task/68638/fd (opendir: Permission denied) 2024-03-07T07:43:30.740 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68639 bstore_kv ceph cwd unknown /proc/67334/task/68639/cwd (readlink: Permission denied) 2024-03-07T07:43:30.740 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68639 bstore_kv ceph rtd unknown /proc/67334/task/68639/root (readlink: Permission denied) 2024-03-07T07:43:30.740 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68639 bstore_kv ceph txt unknown /proc/67334/task/68639/exe (readlink: Permission denied) 2024-03-07T07:43:30.741 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68639 bstore_kv ceph NOFD /proc/67334/task/68639/fd (opendir: Permission denied) 2024-03-07T07:43:30.741 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68640 bstore_kv ceph cwd unknown /proc/67334/task/68640/cwd (readlink: Permission denied) 2024-03-07T07:43:30.741 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68640 bstore_kv ceph rtd unknown /proc/67334/task/68640/root (readlink: Permission denied) 2024-03-07T07:43:30.741 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68640 bstore_kv ceph txt unknown /proc/67334/task/68640/exe (readlink: Permission denied) 2024-03-07T07:43:30.741 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68640 bstore_kv ceph NOFD /proc/67334/task/68640/fd (opendir: Permission denied) 2024-03-07T07:43:30.741 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68641 bstore_me ceph cwd unknown /proc/67334/task/68641/cwd (readlink: Permission denied) 2024-03-07T07:43:30.741 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68641 bstore_me ceph rtd unknown /proc/67334/task/68641/root (readlink: Permission denied) 2024-03-07T07:43:30.741 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68641 bstore_me ceph txt unknown /proc/67334/task/68641/exe (readlink: Permission denied) 2024-03-07T07:43:30.741 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68641 bstore_me ceph NOFD /proc/67334/task/68641/fd (opendir: Permission denied) 2024-03-07T07:43:30.741 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68642 ms_dispat ceph cwd unknown /proc/67334/task/68642/cwd (readlink: Permission denied) 2024-03-07T07:43:30.741 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68642 ms_dispat ceph rtd unknown /proc/67334/task/68642/root (readlink: Permission denied) 2024-03-07T07:43:30.741 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68642 ms_dispat ceph txt unknown /proc/67334/task/68642/exe (readlink: Permission denied) 2024-03-07T07:43:30.741 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68642 ms_dispat ceph NOFD /proc/67334/task/68642/fd (opendir: Permission denied) 2024-03-07T07:43:30.741 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68643 ms_local ceph cwd unknown /proc/67334/task/68643/cwd (readlink: Permission denied) 2024-03-07T07:43:30.742 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68643 ms_local ceph rtd unknown /proc/67334/task/68643/root (readlink: Permission denied) 2024-03-07T07:43:30.742 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68643 ms_local ceph txt unknown /proc/67334/task/68643/exe (readlink: Permission denied) 2024-03-07T07:43:30.742 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68643 ms_local ceph NOFD /proc/67334/task/68643/fd (opendir: Permission denied) 2024-03-07T07:43:30.742 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68644 safe_time ceph cwd unknown /proc/67334/task/68644/cwd (readlink: Permission denied) 2024-03-07T07:43:30.742 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68644 safe_time ceph rtd unknown /proc/67334/task/68644/root (readlink: Permission denied) 2024-03-07T07:43:30.742 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68644 safe_time ceph txt unknown /proc/67334/task/68644/exe (readlink: Permission denied) 2024-03-07T07:43:30.742 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68644 safe_time ceph NOFD /proc/67334/task/68644/fd (opendir: Permission denied) 2024-03-07T07:43:30.742 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68645 safe_time ceph cwd unknown /proc/67334/task/68645/cwd (readlink: Permission denied) 2024-03-07T07:43:30.742 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68645 safe_time ceph rtd unknown /proc/67334/task/68645/root (readlink: Permission denied) 2024-03-07T07:43:30.742 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68645 safe_time ceph txt unknown /proc/67334/task/68645/exe (readlink: Permission denied) 2024-03-07T07:43:30.742 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68645 safe_time ceph NOFD /proc/67334/task/68645/fd (opendir: Permission denied) 2024-03-07T07:43:30.742 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68646 ms_dispat ceph cwd unknown /proc/67334/task/68646/cwd (readlink: Permission denied) 2024-03-07T07:43:30.742 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68646 ms_dispat ceph rtd unknown /proc/67334/task/68646/root (readlink: Permission denied) 2024-03-07T07:43:30.742 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68646 ms_dispat ceph txt unknown /proc/67334/task/68646/exe (readlink: Permission denied) 2024-03-07T07:43:30.743 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68646 ms_dispat ceph NOFD /proc/67334/task/68646/fd (opendir: Permission denied) 2024-03-07T07:43:30.743 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68647 ms_local ceph cwd unknown /proc/67334/task/68647/cwd (readlink: Permission denied) 2024-03-07T07:43:30.743 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68647 ms_local ceph rtd unknown /proc/67334/task/68647/root (readlink: Permission denied) 2024-03-07T07:43:30.743 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68647 ms_local ceph txt unknown /proc/67334/task/68647/exe (readlink: Permission denied) 2024-03-07T07:43:30.743 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68647 ms_local ceph NOFD /proc/67334/task/68647/fd (opendir: Permission denied) 2024-03-07T07:43:30.743 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68648 ms_dispat ceph cwd unknown /proc/67334/task/68648/cwd (readlink: Permission denied) 2024-03-07T07:43:30.743 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68648 ms_dispat ceph rtd unknown /proc/67334/task/68648/root (readlink: Permission denied) 2024-03-07T07:43:30.743 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68648 ms_dispat ceph txt unknown /proc/67334/task/68648/exe (readlink: Permission denied) 2024-03-07T07:43:30.743 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68648 ms_dispat ceph NOFD /proc/67334/task/68648/fd (opendir: Permission denied) 2024-03-07T07:43:30.743 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68649 ms_local ceph cwd unknown /proc/67334/task/68649/cwd (readlink: Permission denied) 2024-03-07T07:43:30.743 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68649 ms_local ceph rtd unknown /proc/67334/task/68649/root (readlink: Permission denied) 2024-03-07T07:43:30.743 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68649 ms_local ceph txt unknown /proc/67334/task/68649/exe (readlink: Permission denied) 2024-03-07T07:43:30.743 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68649 ms_local ceph NOFD /proc/67334/task/68649/fd (opendir: Permission denied) 2024-03-07T07:43:30.743 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68650 ms_dispat ceph cwd unknown /proc/67334/task/68650/cwd (readlink: Permission denied) 2024-03-07T07:43:30.743 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68650 ms_dispat ceph rtd unknown /proc/67334/task/68650/root (readlink: Permission denied) 2024-03-07T07:43:30.744 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68650 ms_dispat ceph txt unknown /proc/67334/task/68650/exe (readlink: Permission denied) 2024-03-07T07:43:30.744 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68650 ms_dispat ceph NOFD /proc/67334/task/68650/fd (opendir: Permission denied) 2024-03-07T07:43:30.744 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68651 ms_local ceph cwd unknown /proc/67334/task/68651/cwd (readlink: Permission denied) 2024-03-07T07:43:30.744 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68651 ms_local ceph rtd unknown /proc/67334/task/68651/root (readlink: Permission denied) 2024-03-07T07:43:30.744 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68651 ms_local ceph txt unknown /proc/67334/task/68651/exe (readlink: Permission denied) 2024-03-07T07:43:30.744 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68651 ms_local ceph NOFD /proc/67334/task/68651/fd (opendir: Permission denied) 2024-03-07T07:43:30.744 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68652 ms_dispat ceph cwd unknown /proc/67334/task/68652/cwd (readlink: Permission denied) 2024-03-07T07:43:30.744 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68652 ms_dispat ceph rtd unknown /proc/67334/task/68652/root (readlink: Permission denied) 2024-03-07T07:43:30.744 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68652 ms_dispat ceph txt unknown /proc/67334/task/68652/exe (readlink: Permission denied) 2024-03-07T07:43:30.744 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68652 ms_dispat ceph NOFD /proc/67334/task/68652/fd (opendir: Permission denied) 2024-03-07T07:43:30.744 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68653 ms_local ceph cwd unknown /proc/67334/task/68653/cwd (readlink: Permission denied) 2024-03-07T07:43:30.744 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68653 ms_local ceph rtd unknown /proc/67334/task/68653/root (readlink: Permission denied) 2024-03-07T07:43:30.744 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68653 ms_local ceph txt unknown /proc/67334/task/68653/exe (readlink: Permission denied) 2024-03-07T07:43:30.745 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68653 ms_local ceph NOFD /proc/67334/task/68653/fd (opendir: Permission denied) 2024-03-07T07:43:30.745 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68654 ms_dispat ceph cwd unknown /proc/67334/task/68654/cwd (readlink: Permission denied) 2024-03-07T07:43:30.745 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68654 ms_dispat ceph rtd unknown /proc/67334/task/68654/root (readlink: Permission denied) 2024-03-07T07:43:30.745 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68654 ms_dispat ceph txt unknown /proc/67334/task/68654/exe (readlink: Permission denied) 2024-03-07T07:43:30.745 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68654 ms_dispat ceph NOFD /proc/67334/task/68654/fd (opendir: Permission denied) 2024-03-07T07:43:30.745 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68655 ms_local ceph cwd unknown /proc/67334/task/68655/cwd (readlink: Permission denied) 2024-03-07T07:43:30.745 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68655 ms_local ceph rtd unknown /proc/67334/task/68655/root (readlink: Permission denied) 2024-03-07T07:43:30.745 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68655 ms_local ceph txt unknown /proc/67334/task/68655/exe (readlink: Permission denied) 2024-03-07T07:43:30.745 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68655 ms_local ceph NOFD /proc/67334/task/68655/fd (opendir: Permission denied) 2024-03-07T07:43:30.745 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68656 ms_dispat ceph cwd unknown /proc/67334/task/68656/cwd (readlink: Permission denied) 2024-03-07T07:43:30.745 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68656 ms_dispat ceph rtd unknown /proc/67334/task/68656/root (readlink: Permission denied) 2024-03-07T07:43:30.745 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68656 ms_dispat ceph txt unknown /proc/67334/task/68656/exe (readlink: Permission denied) 2024-03-07T07:43:30.745 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68656 ms_dispat ceph NOFD /proc/67334/task/68656/fd (opendir: Permission denied) 2024-03-07T07:43:30.746 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68657 ms_local ceph cwd unknown /proc/67334/task/68657/cwd (readlink: Permission denied) 2024-03-07T07:43:30.746 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68657 ms_local ceph rtd unknown /proc/67334/task/68657/root (readlink: Permission denied) 2024-03-07T07:43:30.746 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68657 ms_local ceph txt unknown /proc/67334/task/68657/exe (readlink: Permission denied) 2024-03-07T07:43:30.746 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68657 ms_local ceph NOFD /proc/67334/task/68657/fd (opendir: Permission denied) 2024-03-07T07:43:30.746 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68658 fn_anonym ceph cwd unknown /proc/67334/task/68658/cwd (readlink: Permission denied) 2024-03-07T07:43:30.746 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68658 fn_anonym ceph rtd unknown /proc/67334/task/68658/root (readlink: Permission denied) 2024-03-07T07:43:30.746 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68658 fn_anonym ceph txt unknown /proc/67334/task/68658/exe (readlink: Permission denied) 2024-03-07T07:43:30.746 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68658 fn_anonym ceph NOFD /proc/67334/task/68658/fd (opendir: Permission denied) 2024-03-07T07:43:30.746 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68659 finisher ceph cwd unknown /proc/67334/task/68659/cwd (readlink: Permission denied) 2024-03-07T07:43:30.746 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68659 finisher ceph rtd unknown /proc/67334/task/68659/root (readlink: Permission denied) 2024-03-07T07:43:30.746 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68659 finisher ceph txt unknown /proc/67334/task/68659/exe (readlink: Permission denied) 2024-03-07T07:43:30.746 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68659 finisher ceph NOFD /proc/67334/task/68659/fd (opendir: Permission denied) 2024-03-07T07:43:30.746 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68660 safe_time ceph cwd unknown /proc/67334/task/68660/cwd (readlink: Permission denied) 2024-03-07T07:43:30.746 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68660 safe_time ceph rtd unknown /proc/67334/task/68660/root (readlink: Permission denied) 2024-03-07T07:43:30.746 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68660 safe_time ceph txt unknown /proc/67334/task/68660/exe (readlink: Permission denied) 2024-03-07T07:43:30.747 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68660 safe_time ceph NOFD /proc/67334/task/68660/fd (opendir: Permission denied) 2024-03-07T07:43:30.747 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68661 safe_time ceph cwd unknown /proc/67334/task/68661/cwd (readlink: Permission denied) 2024-03-07T07:43:30.747 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68661 safe_time ceph rtd unknown /proc/67334/task/68661/root (readlink: Permission denied) 2024-03-07T07:43:30.747 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68661 safe_time ceph txt unknown /proc/67334/task/68661/exe (readlink: Permission denied) 2024-03-07T07:43:30.747 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68661 safe_time ceph NOFD /proc/67334/task/68661/fd (opendir: Permission denied) 2024-03-07T07:43:30.747 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68662 ceph-osd ceph cwd unknown /proc/67334/task/68662/cwd (readlink: Permission denied) 2024-03-07T07:43:30.747 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68662 ceph-osd ceph rtd unknown /proc/67334/task/68662/root (readlink: Permission denied) 2024-03-07T07:43:30.747 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68662 ceph-osd ceph txt unknown /proc/67334/task/68662/exe (readlink: Permission denied) 2024-03-07T07:43:30.747 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68662 ceph-osd ceph NOFD /proc/67334/task/68662/fd (opendir: Permission denied) 2024-03-07T07:43:30.747 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68663 osd_srv_a ceph cwd unknown /proc/67334/task/68663/cwd (readlink: Permission denied) 2024-03-07T07:43:30.747 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68663 osd_srv_a ceph rtd unknown /proc/67334/task/68663/root (readlink: Permission denied) 2024-03-07T07:43:30.747 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68663 osd_srv_a ceph txt unknown /proc/67334/task/68663/exe (readlink: Permission denied) 2024-03-07T07:43:30.747 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68663 osd_srv_a ceph NOFD /proc/67334/task/68663/fd (opendir: Permission denied) 2024-03-07T07:43:30.748 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68664 tp_osd_tp ceph cwd unknown /proc/67334/task/68664/cwd (readlink: Permission denied) 2024-03-07T07:43:30.748 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68664 tp_osd_tp ceph rtd unknown /proc/67334/task/68664/root (readlink: Permission denied) 2024-03-07T07:43:30.748 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68664 tp_osd_tp ceph txt unknown /proc/67334/task/68664/exe (readlink: Permission denied) 2024-03-07T07:43:30.748 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68664 tp_osd_tp ceph NOFD /proc/67334/task/68664/fd (opendir: Permission denied) 2024-03-07T07:43:30.748 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68665 tp_osd_tp ceph cwd unknown /proc/67334/task/68665/cwd (readlink: Permission denied) 2024-03-07T07:43:30.748 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68665 tp_osd_tp ceph rtd unknown /proc/67334/task/68665/root (readlink: Permission denied) 2024-03-07T07:43:30.748 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68665 tp_osd_tp ceph txt unknown /proc/67334/task/68665/exe (readlink: Permission denied) 2024-03-07T07:43:30.748 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68665 tp_osd_tp ceph NOFD /proc/67334/task/68665/fd (opendir: Permission denied) 2024-03-07T07:43:30.748 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68666 tp_osd_tp ceph cwd unknown /proc/67334/task/68666/cwd (readlink: Permission denied) 2024-03-07T07:43:30.748 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68666 tp_osd_tp ceph rtd unknown /proc/67334/task/68666/root (readlink: Permission denied) 2024-03-07T07:43:30.748 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68666 tp_osd_tp ceph txt unknown /proc/67334/task/68666/exe (readlink: Permission denied) 2024-03-07T07:43:30.748 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68666 tp_osd_tp ceph NOFD /proc/67334/task/68666/fd (opendir: Permission denied) 2024-03-07T07:43:30.748 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68667 tp_osd_tp ceph cwd unknown /proc/67334/task/68667/cwd (readlink: Permission denied) 2024-03-07T07:43:30.749 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68667 tp_osd_tp ceph rtd unknown /proc/67334/task/68667/root (readlink: Permission denied) 2024-03-07T07:43:30.749 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68667 tp_osd_tp ceph txt unknown /proc/67334/task/68667/exe (readlink: Permission denied) 2024-03-07T07:43:30.749 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68667 tp_osd_tp ceph NOFD /proc/67334/task/68667/fd (opendir: Permission denied) 2024-03-07T07:43:30.749 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68668 tp_osd_tp ceph cwd unknown /proc/67334/task/68668/cwd (readlink: Permission denied) 2024-03-07T07:43:30.749 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68668 tp_osd_tp ceph rtd unknown /proc/67334/task/68668/root (readlink: Permission denied) 2024-03-07T07:43:30.749 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68668 tp_osd_tp ceph txt unknown /proc/67334/task/68668/exe (readlink: Permission denied) 2024-03-07T07:43:30.749 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68668 tp_osd_tp ceph NOFD /proc/67334/task/68668/fd (opendir: Permission denied) 2024-03-07T07:43:30.749 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68669 tp_osd_tp ceph cwd unknown /proc/67334/task/68669/cwd (readlink: Permission denied) 2024-03-07T07:43:30.749 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68669 tp_osd_tp ceph rtd unknown /proc/67334/task/68669/root (readlink: Permission denied) 2024-03-07T07:43:30.749 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68669 tp_osd_tp ceph txt unknown /proc/67334/task/68669/exe (readlink: Permission denied) 2024-03-07T07:43:30.749 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68669 tp_osd_tp ceph NOFD /proc/67334/task/68669/fd (opendir: Permission denied) 2024-03-07T07:43:30.749 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68670 tp_osd_tp ceph cwd unknown /proc/67334/task/68670/cwd (readlink: Permission denied) 2024-03-07T07:43:30.749 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68670 tp_osd_tp ceph rtd unknown /proc/67334/task/68670/root (readlink: Permission denied) 2024-03-07T07:43:30.749 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68670 tp_osd_tp ceph txt unknown /proc/67334/task/68670/exe (readlink: Permission denied) 2024-03-07T07:43:30.750 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68670 tp_osd_tp ceph NOFD /proc/67334/task/68670/fd (opendir: Permission denied) 2024-03-07T07:43:30.750 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68671 tp_osd_tp ceph cwd unknown /proc/67334/task/68671/cwd (readlink: Permission denied) 2024-03-07T07:43:30.750 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68671 tp_osd_tp ceph rtd unknown /proc/67334/task/68671/root (readlink: Permission denied) 2024-03-07T07:43:30.750 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68671 tp_osd_tp ceph txt unknown /proc/67334/task/68671/exe (readlink: Permission denied) 2024-03-07T07:43:30.750 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68671 tp_osd_tp ceph NOFD /proc/67334/task/68671/fd (opendir: Permission denied) 2024-03-07T07:43:30.750 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68672 tp_osd_tp ceph cwd unknown /proc/67334/task/68672/cwd (readlink: Permission denied) 2024-03-07T07:43:30.750 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68672 tp_osd_tp ceph rtd unknown /proc/67334/task/68672/root (readlink: Permission denied) 2024-03-07T07:43:30.750 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68672 tp_osd_tp ceph txt unknown /proc/67334/task/68672/exe (readlink: Permission denied) 2024-03-07T07:43:30.750 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68672 tp_osd_tp ceph NOFD /proc/67334/task/68672/fd (opendir: Permission denied) 2024-03-07T07:43:30.750 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68673 tp_osd_tp ceph cwd unknown /proc/67334/task/68673/cwd (readlink: Permission denied) 2024-03-07T07:43:30.750 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68673 tp_osd_tp ceph rtd unknown /proc/67334/task/68673/root (readlink: Permission denied) 2024-03-07T07:43:30.750 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68673 tp_osd_tp ceph txt unknown /proc/67334/task/68673/exe (readlink: Permission denied) 2024-03-07T07:43:30.750 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68673 tp_osd_tp ceph NOFD /proc/67334/task/68673/fd (opendir: Permission denied) 2024-03-07T07:43:30.750 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68674 tp_osd_tp ceph cwd unknown /proc/67334/task/68674/cwd (readlink: Permission denied) 2024-03-07T07:43:30.751 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68674 tp_osd_tp ceph rtd unknown /proc/67334/task/68674/root (readlink: Permission denied) 2024-03-07T07:43:30.751 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68674 tp_osd_tp ceph txt unknown /proc/67334/task/68674/exe (readlink: Permission denied) 2024-03-07T07:43:30.751 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68674 tp_osd_tp ceph NOFD /proc/67334/task/68674/fd (opendir: Permission denied) 2024-03-07T07:43:30.751 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68675 tp_osd_tp ceph cwd unknown /proc/67334/task/68675/cwd (readlink: Permission denied) 2024-03-07T07:43:30.751 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68675 tp_osd_tp ceph rtd unknown /proc/67334/task/68675/root (readlink: Permission denied) 2024-03-07T07:43:30.751 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68675 tp_osd_tp ceph txt unknown /proc/67334/task/68675/exe (readlink: Permission denied) 2024-03-07T07:43:30.751 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68675 tp_osd_tp ceph NOFD /proc/67334/task/68675/fd (opendir: Permission denied) 2024-03-07T07:43:30.751 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68676 tp_osd_tp ceph cwd unknown /proc/67334/task/68676/cwd (readlink: Permission denied) 2024-03-07T07:43:30.751 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68676 tp_osd_tp ceph rtd unknown /proc/67334/task/68676/root (readlink: Permission denied) 2024-03-07T07:43:30.751 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68676 tp_osd_tp ceph txt unknown /proc/67334/task/68676/exe (readlink: Permission denied) 2024-03-07T07:43:30.751 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68676 tp_osd_tp ceph NOFD /proc/67334/task/68676/fd (opendir: Permission denied) 2024-03-07T07:43:30.751 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68677 tp_osd_tp ceph cwd unknown /proc/67334/task/68677/cwd (readlink: Permission denied) 2024-03-07T07:43:30.751 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68677 tp_osd_tp ceph rtd unknown /proc/67334/task/68677/root (readlink: Permission denied) 2024-03-07T07:43:30.751 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68677 tp_osd_tp ceph txt unknown /proc/67334/task/68677/exe (readlink: Permission denied) 2024-03-07T07:43:30.752 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68677 tp_osd_tp ceph NOFD /proc/67334/task/68677/fd (opendir: Permission denied) 2024-03-07T07:43:30.752 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68678 tp_osd_tp ceph cwd unknown /proc/67334/task/68678/cwd (readlink: Permission denied) 2024-03-07T07:43:30.752 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68678 tp_osd_tp ceph rtd unknown /proc/67334/task/68678/root (readlink: Permission denied) 2024-03-07T07:43:30.752 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68678 tp_osd_tp ceph txt unknown /proc/67334/task/68678/exe (readlink: Permission denied) 2024-03-07T07:43:30.752 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68678 tp_osd_tp ceph NOFD /proc/67334/task/68678/fd (opendir: Permission denied) 2024-03-07T07:43:30.752 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68679 tp_osd_tp ceph cwd unknown /proc/67334/task/68679/cwd (readlink: Permission denied) 2024-03-07T07:43:30.752 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68679 tp_osd_tp ceph rtd unknown /proc/67334/task/68679/root (readlink: Permission denied) 2024-03-07T07:43:30.752 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68679 tp_osd_tp ceph txt unknown /proc/67334/task/68679/exe (readlink: Permission denied) 2024-03-07T07:43:30.752 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68679 tp_osd_tp ceph NOFD /proc/67334/task/68679/fd (opendir: Permission denied) 2024-03-07T07:43:30.752 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68680 osd_srv_h ceph cwd unknown /proc/67334/task/68680/cwd (readlink: Permission denied) 2024-03-07T07:43:30.752 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68680 osd_srv_h ceph rtd unknown /proc/67334/task/68680/root (readlink: Permission denied) 2024-03-07T07:43:30.752 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68680 osd_srv_h ceph txt unknown /proc/67334/task/68680/exe (readlink: Permission denied) 2024-03-07T07:43:30.753 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-osd 67334 68680 osd_srv_h ceph NOFD /proc/67334/task/68680/fd (opendir: Permission denied) 2024-03-07T07:43:30.753 INFO:tasks.workunit.client.0.smithi160.stdout:sudo 68777 root cwd unknown /proc/68777/cwd (readlink: Permission denied) 2024-03-07T07:43:30.753 INFO:tasks.workunit.client.0.smithi160.stdout:sudo 68777 root rtd unknown /proc/68777/root (readlink: Permission denied) 2024-03-07T07:43:30.753 INFO:tasks.workunit.client.0.smithi160.stdout:sudo 68777 root txt unknown /proc/68777/exe (readlink: Permission denied) 2024-03-07T07:43:30.753 INFO:tasks.workunit.client.0.smithi160.stdout:sudo 68777 root NOFD /proc/68777/fd (opendir: Permission denied) 2024-03-07T07:43:30.753 INFO:tasks.workunit.client.0.smithi160.stdout:journalct 68807 root cwd unknown /proc/68807/cwd (readlink: Permission denied) 2024-03-07T07:43:30.753 INFO:tasks.workunit.client.0.smithi160.stdout:journalct 68807 root rtd unknown /proc/68807/root (readlink: Permission denied) 2024-03-07T07:43:30.753 INFO:tasks.workunit.client.0.smithi160.stdout:journalct 68807 root txt unknown /proc/68807/exe (readlink: Permission denied) 2024-03-07T07:43:30.753 INFO:tasks.workunit.client.0.smithi160.stdout:journalct 68807 root NOFD /proc/68807/fd (opendir: Permission denied) 2024-03-07T07:43:30.753 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/0 72109 root cwd unknown /proc/72109/cwd (readlink: Permission denied) 2024-03-07T07:43:30.753 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/0 72109 root rtd unknown /proc/72109/root (readlink: Permission denied) 2024-03-07T07:43:30.753 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/0 72109 root txt unknown /proc/72109/exe (readlink: Permission denied) 2024-03-07T07:43:30.753 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/0 72109 root NOFD /proc/72109/fd (opendir: Permission denied) 2024-03-07T07:43:30.754 INFO:tasks.workunit.client.0.smithi160.stdout:sudo 72111 root cwd unknown /proc/72111/cwd (readlink: Permission denied) 2024-03-07T07:43:30.754 INFO:tasks.workunit.client.0.smithi160.stdout:sudo 72111 root rtd unknown /proc/72111/root (readlink: Permission denied) 2024-03-07T07:43:30.754 INFO:tasks.workunit.client.0.smithi160.stdout:sudo 72111 root txt unknown /proc/72111/exe (readlink: Permission denied) 2024-03-07T07:43:30.754 INFO:tasks.workunit.client.0.smithi160.stdout:sudo 72111 root NOFD /proc/72111/fd (opendir: Permission denied) 2024-03-07T07:43:30.754 INFO:tasks.workunit.client.0.smithi160.stdout:journalct 72128 root cwd unknown /proc/72128/cwd (readlink: Permission denied) 2024-03-07T07:43:30.754 INFO:tasks.workunit.client.0.smithi160.stdout:journalct 72128 root rtd unknown /proc/72128/root (readlink: Permission denied) 2024-03-07T07:43:30.754 INFO:tasks.workunit.client.0.smithi160.stdout:journalct 72128 root txt unknown /proc/72128/exe (readlink: Permission denied) 2024-03-07T07:43:30.754 INFO:tasks.workunit.client.0.smithi160.stdout:journalct 72128 root NOFD /proc/72128/fd (opendir: Permission denied) 2024-03-07T07:43:30.754 INFO:tasks.workunit.client.0.smithi160.stdout:sudo 72129 root cwd unknown /proc/72129/cwd (readlink: Permission denied) 2024-03-07T07:43:30.754 INFO:tasks.workunit.client.0.smithi160.stdout:sudo 72129 root rtd unknown /proc/72129/root (readlink: Permission denied) 2024-03-07T07:43:30.754 INFO:tasks.workunit.client.0.smithi160.stdout:sudo 72129 root txt unknown /proc/72129/exe (readlink: Permission denied) 2024-03-07T07:43:30.754 INFO:tasks.workunit.client.0.smithi160.stdout:sudo 72129 root NOFD /proc/72129/fd (opendir: Permission denied) 2024-03-07T07:43:30.754 INFO:tasks.workunit.client.0.smithi160.stdout:journalct 72146 root cwd unknown /proc/72146/cwd (readlink: Permission denied) 2024-03-07T07:43:30.754 INFO:tasks.workunit.client.0.smithi160.stdout:journalct 72146 root rtd unknown /proc/72146/root (readlink: Permission denied) 2024-03-07T07:43:30.755 INFO:tasks.workunit.client.0.smithi160.stdout:journalct 72146 root txt unknown /proc/72146/exe (readlink: Permission denied) 2024-03-07T07:43:30.755 INFO:tasks.workunit.client.0.smithi160.stdout:journalct 72146 root NOFD /proc/72146/fd (opendir: Permission denied) 2024-03-07T07:43:30.755 INFO:tasks.workunit.client.0.smithi160.stdout:sudo 72147 root cwd unknown /proc/72147/cwd (readlink: Permission denied) 2024-03-07T07:43:30.755 INFO:tasks.workunit.client.0.smithi160.stdout:sudo 72147 root rtd unknown /proc/72147/root (readlink: Permission denied) 2024-03-07T07:43:30.755 INFO:tasks.workunit.client.0.smithi160.stdout:sudo 72147 root txt unknown /proc/72147/exe (readlink: Permission denied) 2024-03-07T07:43:30.755 INFO:tasks.workunit.client.0.smithi160.stdout:sudo 72147 root NOFD /proc/72147/fd (opendir: Permission denied) 2024-03-07T07:43:30.755 INFO:tasks.workunit.client.0.smithi160.stdout:journalct 72164 root cwd unknown /proc/72164/cwd (readlink: Permission denied) 2024-03-07T07:43:30.755 INFO:tasks.workunit.client.0.smithi160.stdout:journalct 72164 root rtd unknown /proc/72164/root (readlink: Permission denied) 2024-03-07T07:43:30.755 INFO:tasks.workunit.client.0.smithi160.stdout:journalct 72164 root txt unknown /proc/72164/exe (readlink: Permission denied) 2024-03-07T07:43:30.755 INFO:tasks.workunit.client.0.smithi160.stdout:journalct 72164 root NOFD /proc/72164/fd (opendir: Permission denied) 2024-03-07T07:43:30.755 INFO:tasks.workunit.client.0.smithi160.stdout:sudo 72165 root cwd unknown /proc/72165/cwd (readlink: Permission denied) 2024-03-07T07:43:30.755 INFO:tasks.workunit.client.0.smithi160.stdout:sudo 72165 root rtd unknown /proc/72165/root (readlink: Permission denied) 2024-03-07T07:43:30.755 INFO:tasks.workunit.client.0.smithi160.stdout:sudo 72165 root txt unknown /proc/72165/exe (readlink: Permission denied) 2024-03-07T07:43:30.755 INFO:tasks.workunit.client.0.smithi160.stdout:sudo 72165 root NOFD /proc/72165/fd (opendir: Permission denied) 2024-03-07T07:43:30.756 INFO:tasks.workunit.client.0.smithi160.stdout:journalct 72182 root cwd unknown /proc/72182/cwd (readlink: Permission denied) 2024-03-07T07:43:30.756 INFO:tasks.workunit.client.0.smithi160.stdout:journalct 72182 root rtd unknown /proc/72182/root (readlink: Permission denied) 2024-03-07T07:43:30.756 INFO:tasks.workunit.client.0.smithi160.stdout:journalct 72182 root txt unknown /proc/72182/exe (readlink: Permission denied) 2024-03-07T07:43:30.756 INFO:tasks.workunit.client.0.smithi160.stdout:journalct 72182 root NOFD /proc/72182/fd (opendir: Permission denied) 2024-03-07T07:43:30.756 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/4 72541 root cwd unknown /proc/72541/cwd (readlink: Permission denied) 2024-03-07T07:43:30.756 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/4 72541 root rtd unknown /proc/72541/root (readlink: Permission denied) 2024-03-07T07:43:30.756 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/4 72541 root txt unknown /proc/72541/exe (readlink: Permission denied) 2024-03-07T07:43:30.756 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/4 72541 root NOFD /proc/72541/fd (opendir: Permission denied) 2024-03-07T07:43:30.756 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/u 72547 root cwd unknown /proc/72547/cwd (readlink: Permission denied) 2024-03-07T07:43:30.756 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/u 72547 root rtd unknown /proc/72547/root (readlink: Permission denied) 2024-03-07T07:43:30.756 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/u 72547 root txt unknown /proc/72547/exe (readlink: Permission denied) 2024-03-07T07:43:30.756 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/u 72547 root NOFD /proc/72547/fd (opendir: Permission denied) 2024-03-07T07:43:30.756 INFO:tasks.workunit.client.0.smithi160.stdout:conmon 72721 root cwd unknown /proc/72721/cwd (readlink: Permission denied) 2024-03-07T07:43:30.756 INFO:tasks.workunit.client.0.smithi160.stdout:conmon 72721 root rtd unknown /proc/72721/root (readlink: Permission denied) 2024-03-07T07:43:30.757 INFO:tasks.workunit.client.0.smithi160.stdout:conmon 72721 root txt unknown /proc/72721/exe (readlink: Permission denied) 2024-03-07T07:43:30.757 INFO:tasks.workunit.client.0.smithi160.stdout:conmon 72721 root NOFD /proc/72721/fd (opendir: Permission denied) 2024-03-07T07:43:30.757 INFO:tasks.workunit.client.0.smithi160.stdout:podman-in 72723 root cwd unknown /proc/72723/cwd (readlink: Permission denied) 2024-03-07T07:43:30.757 INFO:tasks.workunit.client.0.smithi160.stdout:podman-in 72723 root rtd unknown /proc/72723/root (readlink: Permission denied) 2024-03-07T07:43:30.757 INFO:tasks.workunit.client.0.smithi160.stdout:podman-in 72723 root txt unknown /proc/72723/exe (readlink: Permission denied) 2024-03-07T07:43:30.757 INFO:tasks.workunit.client.0.smithi160.stdout:podman-in 72723 root NOFD /proc/72723/fd (opendir: Permission denied) 2024-03-07T07:43:30.757 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 ceph cwd unknown /proc/72727/cwd (readlink: Permission denied) 2024-03-07T07:43:30.757 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 ceph rtd unknown /proc/72727/root (readlink: Permission denied) 2024-03-07T07:43:30.757 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 ceph txt unknown /proc/72727/exe (readlink: Permission denied) 2024-03-07T07:43:30.757 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 ceph NOFD /proc/72727/fd (opendir: Permission denied) 2024-03-07T07:43:30.757 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72728 log ceph cwd unknown /proc/72727/task/72728/cwd (readlink: Permission denied) 2024-03-07T07:43:30.757 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72728 log ceph rtd unknown /proc/72727/task/72728/root (readlink: Permission denied) 2024-03-07T07:43:30.757 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72728 log ceph txt unknown /proc/72727/task/72728/exe (readlink: Permission denied) 2024-03-07T07:43:30.757 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72728 log ceph NOFD /proc/72727/task/72728/fd (opendir: Permission denied) 2024-03-07T07:43:30.758 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72730 msgr-work ceph cwd unknown /proc/72727/task/72730/cwd (readlink: Permission denied) 2024-03-07T07:43:30.758 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72730 msgr-work ceph rtd unknown /proc/72727/task/72730/root (readlink: Permission denied) 2024-03-07T07:43:30.758 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72730 msgr-work ceph txt unknown /proc/72727/task/72730/exe (readlink: Permission denied) 2024-03-07T07:43:30.758 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72730 msgr-work ceph NOFD /proc/72727/task/72730/fd (opendir: Permission denied) 2024-03-07T07:43:30.758 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72731 msgr-work ceph cwd unknown /proc/72727/task/72731/cwd (readlink: Permission denied) 2024-03-07T07:43:30.758 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72731 msgr-work ceph rtd unknown /proc/72727/task/72731/root (readlink: Permission denied) 2024-03-07T07:43:30.758 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72731 msgr-work ceph txt unknown /proc/72727/task/72731/exe (readlink: Permission denied) 2024-03-07T07:43:30.758 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72731 msgr-work ceph NOFD /proc/72727/task/72731/fd (opendir: Permission denied) 2024-03-07T07:43:30.758 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72732 msgr-work ceph cwd unknown /proc/72727/task/72732/cwd (readlink: Permission denied) 2024-03-07T07:43:30.758 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72732 msgr-work ceph rtd unknown /proc/72727/task/72732/root (readlink: Permission denied) 2024-03-07T07:43:30.758 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72732 msgr-work ceph txt unknown /proc/72727/task/72732/exe (readlink: Permission denied) 2024-03-07T07:43:30.758 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72732 msgr-work ceph NOFD /proc/72727/task/72732/fd (opendir: Permission denied) 2024-03-07T07:43:30.758 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72736 service ceph cwd unknown /proc/72727/task/72736/cwd (readlink: Permission denied) 2024-03-07T07:43:30.758 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72736 service ceph rtd unknown /proc/72727/task/72736/root (readlink: Permission denied) 2024-03-07T07:43:30.758 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72736 service ceph txt unknown /proc/72727/task/72736/exe (readlink: Permission denied) 2024-03-07T07:43:30.759 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72736 service ceph NOFD /proc/72727/task/72736/fd (opendir: Permission denied) 2024-03-07T07:43:30.759 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72737 admin_soc ceph cwd unknown /proc/72727/task/72737/cwd (readlink: Permission denied) 2024-03-07T07:43:30.759 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72737 admin_soc ceph rtd unknown /proc/72727/task/72737/root (readlink: Permission denied) 2024-03-07T07:43:30.759 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72737 admin_soc ceph txt unknown /proc/72727/task/72737/exe (readlink: Permission denied) 2024-03-07T07:43:30.759 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72737 admin_soc ceph NOFD /proc/72727/task/72737/fd (opendir: Permission denied) 2024-03-07T07:43:30.759 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72738 signal_ha ceph cwd unknown /proc/72727/task/72738/cwd (readlink: Permission denied) 2024-03-07T07:43:30.759 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72738 signal_ha ceph rtd unknown /proc/72727/task/72738/root (readlink: Permission denied) 2024-03-07T07:43:30.759 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72738 signal_ha ceph txt unknown /proc/72727/task/72738/exe (readlink: Permission denied) 2024-03-07T07:43:30.759 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72738 signal_ha ceph NOFD /proc/72727/task/72738/fd (opendir: Permission denied) 2024-03-07T07:43:30.759 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72739 io_contex ceph cwd unknown /proc/72727/task/72739/cwd (readlink: Permission denied) 2024-03-07T07:43:30.759 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72739 io_contex ceph rtd unknown /proc/72727/task/72739/root (readlink: Permission denied) 2024-03-07T07:43:30.759 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72739 io_contex ceph txt unknown /proc/72727/task/72739/exe (readlink: Permission denied) 2024-03-07T07:43:30.759 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72739 io_contex ceph NOFD /proc/72727/task/72739/fd (opendir: Permission denied) 2024-03-07T07:43:30.759 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72740 io_contex ceph cwd unknown /proc/72727/task/72740/cwd (readlink: Permission denied) 2024-03-07T07:43:30.760 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72740 io_contex ceph rtd unknown /proc/72727/task/72740/root (readlink: Permission denied) 2024-03-07T07:43:30.760 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72740 io_contex ceph txt unknown /proc/72727/task/72740/exe (readlink: Permission denied) 2024-03-07T07:43:30.760 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72740 io_contex ceph NOFD /proc/72727/task/72740/fd (opendir: Permission denied) 2024-03-07T07:43:30.760 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72741 ms_dispat ceph cwd unknown /proc/72727/task/72741/cwd (readlink: Permission denied) 2024-03-07T07:43:30.760 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72741 ms_dispat ceph rtd unknown /proc/72727/task/72741/root (readlink: Permission denied) 2024-03-07T07:43:30.760 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72741 ms_dispat ceph txt unknown /proc/72727/task/72741/exe (readlink: Permission denied) 2024-03-07T07:43:30.760 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72741 ms_dispat ceph NOFD /proc/72727/task/72741/fd (opendir: Permission denied) 2024-03-07T07:43:30.760 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72742 ms_local ceph cwd unknown /proc/72727/task/72742/cwd (readlink: Permission denied) 2024-03-07T07:43:30.760 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72742 ms_local ceph rtd unknown /proc/72727/task/72742/root (readlink: Permission denied) 2024-03-07T07:43:30.760 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72742 ms_local ceph txt unknown /proc/72727/task/72742/exe (readlink: Permission denied) 2024-03-07T07:43:30.760 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72742 ms_local ceph NOFD /proc/72727/task/72742/fd (opendir: Permission denied) 2024-03-07T07:43:30.760 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72743 safe_time ceph cwd unknown /proc/72727/task/72743/cwd (readlink: Permission denied) 2024-03-07T07:43:30.760 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72743 safe_time ceph rtd unknown /proc/72727/task/72743/root (readlink: Permission denied) 2024-03-07T07:43:30.760 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72743 safe_time ceph txt unknown /proc/72727/task/72743/exe (readlink: Permission denied) 2024-03-07T07:43:30.761 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72743 safe_time ceph NOFD /proc/72727/task/72743/fd (opendir: Permission denied) 2024-03-07T07:43:30.761 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72744 safe_time ceph cwd unknown /proc/72727/task/72744/cwd (readlink: Permission denied) 2024-03-07T07:43:30.761 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72744 safe_time ceph rtd unknown /proc/72727/task/72744/root (readlink: Permission denied) 2024-03-07T07:43:30.761 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72744 safe_time ceph txt unknown /proc/72727/task/72744/exe (readlink: Permission denied) 2024-03-07T07:43:30.761 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72744 safe_time ceph NOFD /proc/72727/task/72744/fd (opendir: Permission denied) 2024-03-07T07:43:30.761 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72745 ceph-mds ceph cwd unknown /proc/72727/task/72745/cwd (readlink: Permission denied) 2024-03-07T07:43:30.761 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72745 ceph-mds ceph rtd unknown /proc/72727/task/72745/root (readlink: Permission denied) 2024-03-07T07:43:30.761 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72745 ceph-mds ceph txt unknown /proc/72727/task/72745/exe (readlink: Permission denied) 2024-03-07T07:43:30.761 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72745 ceph-mds ceph NOFD /proc/72727/task/72745/fd (opendir: Permission denied) 2024-03-07T07:43:30.761 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72879 safe_time ceph cwd unknown /proc/72727/task/72879/cwd (readlink: Permission denied) 2024-03-07T07:43:30.761 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72879 safe_time ceph rtd unknown /proc/72727/task/72879/root (readlink: Permission denied) 2024-03-07T07:43:30.761 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72879 safe_time ceph txt unknown /proc/72727/task/72879/exe (readlink: Permission denied) 2024-03-07T07:43:30.761 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 72727 72879 safe_time ceph NOFD /proc/72727/task/72879/fd (opendir: Permission denied) 2024-03-07T07:43:30.761 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/5 72970 root cwd unknown /proc/72970/cwd (readlink: Permission denied) 2024-03-07T07:43:30.761 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/5 72970 root rtd unknown /proc/72970/root (readlink: Permission denied) 2024-03-07T07:43:30.762 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/5 72970 root txt unknown /proc/72970/exe (readlink: Permission denied) 2024-03-07T07:43:30.762 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/5 72970 root NOFD /proc/72970/fd (opendir: Permission denied) 2024-03-07T07:43:30.762 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/6 73069 root cwd unknown /proc/73069/cwd (readlink: Permission denied) 2024-03-07T07:43:30.762 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/6 73069 root rtd unknown /proc/73069/root (readlink: Permission denied) 2024-03-07T07:43:30.762 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/6 73069 root txt unknown /proc/73069/exe (readlink: Permission denied) 2024-03-07T07:43:30.762 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/6 73069 root NOFD /proc/73069/fd (opendir: Permission denied) 2024-03-07T07:43:30.762 INFO:tasks.workunit.client.0.smithi160.stdout:conmon 73071 root cwd unknown /proc/73071/cwd (readlink: Permission denied) 2024-03-07T07:43:30.762 INFO:tasks.workunit.client.0.smithi160.stdout:conmon 73071 root rtd unknown /proc/73071/root (readlink: Permission denied) 2024-03-07T07:43:30.762 INFO:tasks.workunit.client.0.smithi160.stdout:conmon 73071 root txt unknown /proc/73071/exe (readlink: Permission denied) 2024-03-07T07:43:30.762 INFO:tasks.workunit.client.0.smithi160.stdout:conmon 73071 root NOFD /proc/73071/fd (opendir: Permission denied) 2024-03-07T07:43:30.762 INFO:tasks.workunit.client.0.smithi160.stdout:podman-in 73073 root cwd unknown /proc/73073/cwd (readlink: Permission denied) 2024-03-07T07:43:30.762 INFO:tasks.workunit.client.0.smithi160.stdout:podman-in 73073 root rtd unknown /proc/73073/root (readlink: Permission denied) 2024-03-07T07:43:30.762 INFO:tasks.workunit.client.0.smithi160.stdout:podman-in 73073 root txt unknown /proc/73073/exe (readlink: Permission denied) 2024-03-07T07:43:30.762 INFO:tasks.workunit.client.0.smithi160.stdout:podman-in 73073 root NOFD /proc/73073/fd (opendir: Permission denied) 2024-03-07T07:43:30.763 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 ceph cwd unknown /proc/73075/cwd (readlink: Permission denied) 2024-03-07T07:43:30.763 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 ceph rtd unknown /proc/73075/root (readlink: Permission denied) 2024-03-07T07:43:30.763 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 ceph txt unknown /proc/73075/exe (readlink: Permission denied) 2024-03-07T07:43:30.763 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 ceph NOFD /proc/73075/fd (opendir: Permission denied) 2024-03-07T07:43:30.763 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73076 log ceph cwd unknown /proc/73075/task/73076/cwd (readlink: Permission denied) 2024-03-07T07:43:30.763 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73076 log ceph rtd unknown /proc/73075/task/73076/root (readlink: Permission denied) 2024-03-07T07:43:30.763 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73076 log ceph txt unknown /proc/73075/task/73076/exe (readlink: Permission denied) 2024-03-07T07:43:30.763 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73076 log ceph NOFD /proc/73075/task/73076/fd (opendir: Permission denied) 2024-03-07T07:43:30.763 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73078 msgr-work ceph cwd unknown /proc/73075/task/73078/cwd (readlink: Permission denied) 2024-03-07T07:43:30.763 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73078 msgr-work ceph rtd unknown /proc/73075/task/73078/root (readlink: Permission denied) 2024-03-07T07:43:30.763 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73078 msgr-work ceph txt unknown /proc/73075/task/73078/exe (readlink: Permission denied) 2024-03-07T07:43:30.763 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73078 msgr-work ceph NOFD /proc/73075/task/73078/fd (opendir: Permission denied) 2024-03-07T07:43:30.763 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73079 msgr-work ceph cwd unknown /proc/73075/task/73079/cwd (readlink: Permission denied) 2024-03-07T07:43:30.764 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73079 msgr-work ceph rtd unknown /proc/73075/task/73079/root (readlink: Permission denied) 2024-03-07T07:43:30.764 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73079 msgr-work ceph txt unknown /proc/73075/task/73079/exe (readlink: Permission denied) 2024-03-07T07:43:30.764 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73079 msgr-work ceph NOFD /proc/73075/task/73079/fd (opendir: Permission denied) 2024-03-07T07:43:30.764 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73080 msgr-work ceph cwd unknown /proc/73075/task/73080/cwd (readlink: Permission denied) 2024-03-07T07:43:30.764 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73080 msgr-work ceph rtd unknown /proc/73075/task/73080/root (readlink: Permission denied) 2024-03-07T07:43:30.764 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73080 msgr-work ceph txt unknown /proc/73075/task/73080/exe (readlink: Permission denied) 2024-03-07T07:43:30.764 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73080 msgr-work ceph NOFD /proc/73075/task/73080/fd (opendir: Permission denied) 2024-03-07T07:43:30.764 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73084 service ceph cwd unknown /proc/73075/task/73084/cwd (readlink: Permission denied) 2024-03-07T07:43:30.764 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73084 service ceph rtd unknown /proc/73075/task/73084/root (readlink: Permission denied) 2024-03-07T07:43:30.764 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73084 service ceph txt unknown /proc/73075/task/73084/exe (readlink: Permission denied) 2024-03-07T07:43:30.764 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73084 service ceph NOFD /proc/73075/task/73084/fd (opendir: Permission denied) 2024-03-07T07:43:30.764 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73085 admin_soc ceph cwd unknown /proc/73075/task/73085/cwd (readlink: Permission denied) 2024-03-07T07:43:30.764 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73085 admin_soc ceph rtd unknown /proc/73075/task/73085/root (readlink: Permission denied) 2024-03-07T07:43:30.764 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73085 admin_soc ceph txt unknown /proc/73075/task/73085/exe (readlink: Permission denied) 2024-03-07T07:43:30.765 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73085 admin_soc ceph NOFD /proc/73075/task/73085/fd (opendir: Permission denied) 2024-03-07T07:43:30.765 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73086 signal_ha ceph cwd unknown /proc/73075/task/73086/cwd (readlink: Permission denied) 2024-03-07T07:43:30.765 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73086 signal_ha ceph rtd unknown /proc/73075/task/73086/root (readlink: Permission denied) 2024-03-07T07:43:30.765 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73086 signal_ha ceph txt unknown /proc/73075/task/73086/exe (readlink: Permission denied) 2024-03-07T07:43:30.765 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73086 signal_ha ceph NOFD /proc/73075/task/73086/fd (opendir: Permission denied) 2024-03-07T07:43:30.765 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73087 io_contex ceph cwd unknown /proc/73075/task/73087/cwd (readlink: Permission denied) 2024-03-07T07:43:30.765 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73087 io_contex ceph rtd unknown /proc/73075/task/73087/root (readlink: Permission denied) 2024-03-07T07:43:30.765 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73087 io_contex ceph txt unknown /proc/73075/task/73087/exe (readlink: Permission denied) 2024-03-07T07:43:30.765 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73087 io_contex ceph NOFD /proc/73075/task/73087/fd (opendir: Permission denied) 2024-03-07T07:43:30.765 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73088 io_contex ceph cwd unknown /proc/73075/task/73088/cwd (readlink: Permission denied) 2024-03-07T07:43:30.765 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73088 io_contex ceph rtd unknown /proc/73075/task/73088/root (readlink: Permission denied) 2024-03-07T07:43:30.765 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73088 io_contex ceph txt unknown /proc/73075/task/73088/exe (readlink: Permission denied) 2024-03-07T07:43:30.765 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73088 io_contex ceph NOFD /proc/73075/task/73088/fd (opendir: Permission denied) 2024-03-07T07:43:30.765 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73089 ms_dispat ceph cwd unknown /proc/73075/task/73089/cwd (readlink: Permission denied) 2024-03-07T07:43:30.766 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73089 ms_dispat ceph rtd unknown /proc/73075/task/73089/root (readlink: Permission denied) 2024-03-07T07:43:30.766 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73089 ms_dispat ceph txt unknown /proc/73075/task/73089/exe (readlink: Permission denied) 2024-03-07T07:43:30.766 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73089 ms_dispat ceph NOFD /proc/73075/task/73089/fd (opendir: Permission denied) 2024-03-07T07:43:30.766 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73090 ms_local ceph cwd unknown /proc/73075/task/73090/cwd (readlink: Permission denied) 2024-03-07T07:43:30.766 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73090 ms_local ceph rtd unknown /proc/73075/task/73090/root (readlink: Permission denied) 2024-03-07T07:43:30.766 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73090 ms_local ceph txt unknown /proc/73075/task/73090/exe (readlink: Permission denied) 2024-03-07T07:43:30.766 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73090 ms_local ceph NOFD /proc/73075/task/73090/fd (opendir: Permission denied) 2024-03-07T07:43:30.766 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73091 safe_time ceph cwd unknown /proc/73075/task/73091/cwd (readlink: Permission denied) 2024-03-07T07:43:30.766 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73091 safe_time ceph rtd unknown /proc/73075/task/73091/root (readlink: Permission denied) 2024-03-07T07:43:30.766 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73091 safe_time ceph txt unknown /proc/73075/task/73091/exe (readlink: Permission denied) 2024-03-07T07:43:30.766 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73091 safe_time ceph NOFD /proc/73075/task/73091/fd (opendir: Permission denied) 2024-03-07T07:43:30.766 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73092 safe_time ceph cwd unknown /proc/73075/task/73092/cwd (readlink: Permission denied) 2024-03-07T07:43:30.766 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73092 safe_time ceph rtd unknown /proc/73075/task/73092/root (readlink: Permission denied) 2024-03-07T07:43:30.767 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73092 safe_time ceph txt unknown /proc/73075/task/73092/exe (readlink: Permission denied) 2024-03-07T07:43:30.767 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73092 safe_time ceph NOFD /proc/73075/task/73092/fd (opendir: Permission denied) 2024-03-07T07:43:30.767 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73093 ceph-mds ceph cwd unknown /proc/73075/task/73093/cwd (readlink: Permission denied) 2024-03-07T07:43:30.767 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73093 ceph-mds ceph rtd unknown /proc/73075/task/73093/root (readlink: Permission denied) 2024-03-07T07:43:30.767 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73093 ceph-mds ceph txt unknown /proc/73075/task/73093/exe (readlink: Permission denied) 2024-03-07T07:43:30.767 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73093 ceph-mds ceph NOFD /proc/73075/task/73093/fd (opendir: Permission denied) 2024-03-07T07:43:30.767 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73096 safe_time ceph cwd unknown /proc/73075/task/73096/cwd (readlink: Permission denied) 2024-03-07T07:43:30.767 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73096 safe_time ceph rtd unknown /proc/73075/task/73096/root (readlink: Permission denied) 2024-03-07T07:43:30.767 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73096 safe_time ceph txt unknown /proc/73075/task/73096/exe (readlink: Permission denied) 2024-03-07T07:43:30.767 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73075 73096 safe_time ceph NOFD /proc/73075/task/73096/fd (opendir: Permission denied) 2024-03-07T07:43:30.767 INFO:tasks.workunit.client.0.smithi160.stdout:conmon 73663 root cwd unknown /proc/73663/cwd (readlink: Permission denied) 2024-03-07T07:43:30.767 INFO:tasks.workunit.client.0.smithi160.stdout:conmon 73663 root rtd unknown /proc/73663/root (readlink: Permission denied) 2024-03-07T07:43:30.767 INFO:tasks.workunit.client.0.smithi160.stdout:conmon 73663 root txt unknown /proc/73663/exe (readlink: Permission denied) 2024-03-07T07:43:30.767 INFO:tasks.workunit.client.0.smithi160.stdout:conmon 73663 root NOFD /proc/73663/fd (opendir: Permission denied) 2024-03-07T07:43:30.768 INFO:tasks.workunit.client.0.smithi160.stdout:podman-in 73665 root cwd unknown /proc/73665/cwd (readlink: Permission denied) 2024-03-07T07:43:30.768 INFO:tasks.workunit.client.0.smithi160.stdout:podman-in 73665 root rtd unknown /proc/73665/root (readlink: Permission denied) 2024-03-07T07:43:30.768 INFO:tasks.workunit.client.0.smithi160.stdout:podman-in 73665 root txt unknown /proc/73665/exe (readlink: Permission denied) 2024-03-07T07:43:30.768 INFO:tasks.workunit.client.0.smithi160.stdout:podman-in 73665 root NOFD /proc/73665/fd (opendir: Permission denied) 2024-03-07T07:43:30.768 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 ceph cwd unknown /proc/73667/cwd (readlink: Permission denied) 2024-03-07T07:43:30.768 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 ceph rtd unknown /proc/73667/root (readlink: Permission denied) 2024-03-07T07:43:30.768 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 ceph txt unknown /proc/73667/exe (readlink: Permission denied) 2024-03-07T07:43:30.768 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 ceph NOFD /proc/73667/fd (opendir: Permission denied) 2024-03-07T07:43:30.768 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73668 log ceph cwd unknown /proc/73667/task/73668/cwd (readlink: Permission denied) 2024-03-07T07:43:30.768 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73668 log ceph rtd unknown /proc/73667/task/73668/root (readlink: Permission denied) 2024-03-07T07:43:30.768 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73668 log ceph txt unknown /proc/73667/task/73668/exe (readlink: Permission denied) 2024-03-07T07:43:30.768 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73668 log ceph NOFD /proc/73667/task/73668/fd (opendir: Permission denied) 2024-03-07T07:43:30.768 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73670 msgr-work ceph cwd unknown /proc/73667/task/73670/cwd (readlink: Permission denied) 2024-03-07T07:43:30.768 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73670 msgr-work ceph rtd unknown /proc/73667/task/73670/root (readlink: Permission denied) 2024-03-07T07:43:30.769 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73670 msgr-work ceph txt unknown /proc/73667/task/73670/exe (readlink: Permission denied) 2024-03-07T07:43:30.769 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73670 msgr-work ceph NOFD /proc/73667/task/73670/fd (opendir: Permission denied) 2024-03-07T07:43:30.769 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73671 msgr-work ceph cwd unknown /proc/73667/task/73671/cwd (readlink: Permission denied) 2024-03-07T07:43:30.769 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73671 msgr-work ceph rtd unknown /proc/73667/task/73671/root (readlink: Permission denied) 2024-03-07T07:43:30.769 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73671 msgr-work ceph txt unknown /proc/73667/task/73671/exe (readlink: Permission denied) 2024-03-07T07:43:30.769 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73671 msgr-work ceph NOFD /proc/73667/task/73671/fd (opendir: Permission denied) 2024-03-07T07:43:30.769 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73672 msgr-work ceph cwd unknown /proc/73667/task/73672/cwd (readlink: Permission denied) 2024-03-07T07:43:30.769 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73672 msgr-work ceph rtd unknown /proc/73667/task/73672/root (readlink: Permission denied) 2024-03-07T07:43:30.769 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73672 msgr-work ceph txt unknown /proc/73667/task/73672/exe (readlink: Permission denied) 2024-03-07T07:43:30.769 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73672 msgr-work ceph NOFD /proc/73667/task/73672/fd (opendir: Permission denied) 2024-03-07T07:43:30.769 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73676 service ceph cwd unknown /proc/73667/task/73676/cwd (readlink: Permission denied) 2024-03-07T07:43:30.769 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73676 service ceph rtd unknown /proc/73667/task/73676/root (readlink: Permission denied) 2024-03-07T07:43:30.769 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73676 service ceph txt unknown /proc/73667/task/73676/exe (readlink: Permission denied) 2024-03-07T07:43:30.769 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73676 service ceph NOFD /proc/73667/task/73676/fd (opendir: Permission denied) 2024-03-07T07:43:30.770 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73677 admin_soc ceph cwd unknown /proc/73667/task/73677/cwd (readlink: Permission denied) 2024-03-07T07:43:30.770 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73677 admin_soc ceph rtd unknown /proc/73667/task/73677/root (readlink: Permission denied) 2024-03-07T07:43:30.770 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73677 admin_soc ceph txt unknown /proc/73667/task/73677/exe (readlink: Permission denied) 2024-03-07T07:43:30.770 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73677 admin_soc ceph NOFD /proc/73667/task/73677/fd (opendir: Permission denied) 2024-03-07T07:43:30.770 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73678 signal_ha ceph cwd unknown /proc/73667/task/73678/cwd (readlink: Permission denied) 2024-03-07T07:43:30.770 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73678 signal_ha ceph rtd unknown /proc/73667/task/73678/root (readlink: Permission denied) 2024-03-07T07:43:30.770 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73678 signal_ha ceph txt unknown /proc/73667/task/73678/exe (readlink: Permission denied) 2024-03-07T07:43:30.770 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73678 signal_ha ceph NOFD /proc/73667/task/73678/fd (opendir: Permission denied) 2024-03-07T07:43:30.770 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73679 io_contex ceph cwd unknown /proc/73667/task/73679/cwd (readlink: Permission denied) 2024-03-07T07:43:30.770 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73679 io_contex ceph rtd unknown /proc/73667/task/73679/root (readlink: Permission denied) 2024-03-07T07:43:30.770 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73679 io_contex ceph txt unknown /proc/73667/task/73679/exe (readlink: Permission denied) 2024-03-07T07:43:30.770 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73679 io_contex ceph NOFD /proc/73667/task/73679/fd (opendir: Permission denied) 2024-03-07T07:43:30.770 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73680 io_contex ceph cwd unknown /proc/73667/task/73680/cwd (readlink: Permission denied) 2024-03-07T07:43:30.770 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73680 io_contex ceph rtd unknown /proc/73667/task/73680/root (readlink: Permission denied) 2024-03-07T07:43:30.771 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73680 io_contex ceph txt unknown /proc/73667/task/73680/exe (readlink: Permission denied) 2024-03-07T07:43:30.771 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73680 io_contex ceph NOFD /proc/73667/task/73680/fd (opendir: Permission denied) 2024-03-07T07:43:30.771 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73681 ms_dispat ceph cwd unknown /proc/73667/task/73681/cwd (readlink: Permission denied) 2024-03-07T07:43:30.771 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73681 ms_dispat ceph rtd unknown /proc/73667/task/73681/root (readlink: Permission denied) 2024-03-07T07:43:30.771 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73681 ms_dispat ceph txt unknown /proc/73667/task/73681/exe (readlink: Permission denied) 2024-03-07T07:43:30.771 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73681 ms_dispat ceph NOFD /proc/73667/task/73681/fd (opendir: Permission denied) 2024-03-07T07:43:30.771 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73682 ms_local ceph cwd unknown /proc/73667/task/73682/cwd (readlink: Permission denied) 2024-03-07T07:43:30.771 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73682 ms_local ceph rtd unknown /proc/73667/task/73682/root (readlink: Permission denied) 2024-03-07T07:43:30.771 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73682 ms_local ceph txt unknown /proc/73667/task/73682/exe (readlink: Permission denied) 2024-03-07T07:43:30.771 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73682 ms_local ceph NOFD /proc/73667/task/73682/fd (opendir: Permission denied) 2024-03-07T07:43:30.771 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73683 safe_time ceph cwd unknown /proc/73667/task/73683/cwd (readlink: Permission denied) 2024-03-07T07:43:30.771 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73683 safe_time ceph rtd unknown /proc/73667/task/73683/root (readlink: Permission denied) 2024-03-07T07:43:30.771 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73683 safe_time ceph txt unknown /proc/73667/task/73683/exe (readlink: Permission denied) 2024-03-07T07:43:30.771 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73683 safe_time ceph NOFD /proc/73667/task/73683/fd (opendir: Permission denied) 2024-03-07T07:43:30.772 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73684 safe_time ceph cwd unknown /proc/73667/task/73684/cwd (readlink: Permission denied) 2024-03-07T07:43:30.772 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73684 safe_time ceph rtd unknown /proc/73667/task/73684/root (readlink: Permission denied) 2024-03-07T07:43:30.772 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73684 safe_time ceph txt unknown /proc/73667/task/73684/exe (readlink: Permission denied) 2024-03-07T07:43:30.772 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73684 safe_time ceph NOFD /proc/73667/task/73684/fd (opendir: Permission denied) 2024-03-07T07:43:30.772 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73685 ceph-mds ceph cwd unknown /proc/73667/task/73685/cwd (readlink: Permission denied) 2024-03-07T07:43:30.772 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73685 ceph-mds ceph rtd unknown /proc/73667/task/73685/root (readlink: Permission denied) 2024-03-07T07:43:30.772 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73685 ceph-mds ceph txt unknown /proc/73667/task/73685/exe (readlink: Permission denied) 2024-03-07T07:43:30.772 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73685 ceph-mds ceph NOFD /proc/73667/task/73685/fd (opendir: Permission denied) 2024-03-07T07:43:30.772 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73756 safe_time ceph cwd unknown /proc/73667/task/73756/cwd (readlink: Permission denied) 2024-03-07T07:43:30.772 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73756 safe_time ceph rtd unknown /proc/73667/task/73756/root (readlink: Permission denied) 2024-03-07T07:43:30.772 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73756 safe_time ceph txt unknown /proc/73667/task/73756/exe (readlink: Permission denied) 2024-03-07T07:43:30.772 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 73667 73756 safe_time ceph NOFD /proc/73667/task/73756/fd (opendir: Permission denied) 2024-03-07T07:43:30.772 INFO:tasks.workunit.client.0.smithi160.stdout:conmon 74023 root cwd unknown /proc/74023/cwd (readlink: Permission denied) 2024-03-07T07:43:30.772 INFO:tasks.workunit.client.0.smithi160.stdout:conmon 74023 root rtd unknown /proc/74023/root (readlink: Permission denied) 2024-03-07T07:43:30.773 INFO:tasks.workunit.client.0.smithi160.stdout:conmon 74023 root txt unknown /proc/74023/exe (readlink: Permission denied) 2024-03-07T07:43:30.773 INFO:tasks.workunit.client.0.smithi160.stdout:conmon 74023 root NOFD /proc/74023/fd (opendir: Permission denied) 2024-03-07T07:43:30.773 INFO:tasks.workunit.client.0.smithi160.stdout:podman-in 74025 root cwd unknown /proc/74025/cwd (readlink: Permission denied) 2024-03-07T07:43:30.773 INFO:tasks.workunit.client.0.smithi160.stdout:podman-in 74025 root rtd unknown /proc/74025/root (readlink: Permission denied) 2024-03-07T07:43:30.773 INFO:tasks.workunit.client.0.smithi160.stdout:podman-in 74025 root txt unknown /proc/74025/exe (readlink: Permission denied) 2024-03-07T07:43:30.773 INFO:tasks.workunit.client.0.smithi160.stdout:podman-in 74025 root NOFD /proc/74025/fd (opendir: Permission denied) 2024-03-07T07:43:30.773 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 ceph cwd unknown /proc/74027/cwd (readlink: Permission denied) 2024-03-07T07:43:30.773 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 ceph rtd unknown /proc/74027/root (readlink: Permission denied) 2024-03-07T07:43:30.773 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 ceph txt unknown /proc/74027/exe (readlink: Permission denied) 2024-03-07T07:43:30.773 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 ceph NOFD /proc/74027/fd (opendir: Permission denied) 2024-03-07T07:43:30.773 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74028 log ceph cwd unknown /proc/74027/task/74028/cwd (readlink: Permission denied) 2024-03-07T07:43:30.773 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74028 log ceph rtd unknown /proc/74027/task/74028/root (readlink: Permission denied) 2024-03-07T07:43:30.773 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74028 log ceph txt unknown /proc/74027/task/74028/exe (readlink: Permission denied) 2024-03-07T07:43:30.773 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74028 log ceph NOFD /proc/74027/task/74028/fd (opendir: Permission denied) 2024-03-07T07:43:30.774 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74030 msgr-work ceph cwd unknown /proc/74027/task/74030/cwd (readlink: Permission denied) 2024-03-07T07:43:30.774 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74030 msgr-work ceph rtd unknown /proc/74027/task/74030/root (readlink: Permission denied) 2024-03-07T07:43:30.774 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74030 msgr-work ceph txt unknown /proc/74027/task/74030/exe (readlink: Permission denied) 2024-03-07T07:43:30.774 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74030 msgr-work ceph NOFD /proc/74027/task/74030/fd (opendir: Permission denied) 2024-03-07T07:43:30.774 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74031 msgr-work ceph cwd unknown /proc/74027/task/74031/cwd (readlink: Permission denied) 2024-03-07T07:43:30.774 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74031 msgr-work ceph rtd unknown /proc/74027/task/74031/root (readlink: Permission denied) 2024-03-07T07:43:30.774 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74031 msgr-work ceph txt unknown /proc/74027/task/74031/exe (readlink: Permission denied) 2024-03-07T07:43:30.774 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74031 msgr-work ceph NOFD /proc/74027/task/74031/fd (opendir: Permission denied) 2024-03-07T07:43:30.774 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74032 msgr-work ceph cwd unknown /proc/74027/task/74032/cwd (readlink: Permission denied) 2024-03-07T07:43:30.774 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74032 msgr-work ceph rtd unknown /proc/74027/task/74032/root (readlink: Permission denied) 2024-03-07T07:43:30.774 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74032 msgr-work ceph txt unknown /proc/74027/task/74032/exe (readlink: Permission denied) 2024-03-07T07:43:30.774 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74032 msgr-work ceph NOFD /proc/74027/task/74032/fd (opendir: Permission denied) 2024-03-07T07:43:30.774 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74036 service ceph cwd unknown /proc/74027/task/74036/cwd (readlink: Permission denied) 2024-03-07T07:43:30.775 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74036 service ceph rtd unknown /proc/74027/task/74036/root (readlink: Permission denied) 2024-03-07T07:43:30.775 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74036 service ceph txt unknown /proc/74027/task/74036/exe (readlink: Permission denied) 2024-03-07T07:43:30.775 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74036 service ceph NOFD /proc/74027/task/74036/fd (opendir: Permission denied) 2024-03-07T07:43:30.775 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74037 admin_soc ceph cwd unknown /proc/74027/task/74037/cwd (readlink: Permission denied) 2024-03-07T07:43:30.775 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74037 admin_soc ceph rtd unknown /proc/74027/task/74037/root (readlink: Permission denied) 2024-03-07T07:43:30.775 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74037 admin_soc ceph txt unknown /proc/74027/task/74037/exe (readlink: Permission denied) 2024-03-07T07:43:30.775 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74037 admin_soc ceph NOFD /proc/74027/task/74037/fd (opendir: Permission denied) 2024-03-07T07:43:30.775 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74038 signal_ha ceph cwd unknown /proc/74027/task/74038/cwd (readlink: Permission denied) 2024-03-07T07:43:30.775 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74038 signal_ha ceph rtd unknown /proc/74027/task/74038/root (readlink: Permission denied) 2024-03-07T07:43:30.775 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74038 signal_ha ceph txt unknown /proc/74027/task/74038/exe (readlink: Permission denied) 2024-03-07T07:43:30.775 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74038 signal_ha ceph NOFD /proc/74027/task/74038/fd (opendir: Permission denied) 2024-03-07T07:43:30.775 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74039 io_contex ceph cwd unknown /proc/74027/task/74039/cwd (readlink: Permission denied) 2024-03-07T07:43:30.775 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74039 io_contex ceph rtd unknown /proc/74027/task/74039/root (readlink: Permission denied) 2024-03-07T07:43:30.775 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74039 io_contex ceph txt unknown /proc/74027/task/74039/exe (readlink: Permission denied) 2024-03-07T07:43:30.775 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74039 io_contex ceph NOFD /proc/74027/task/74039/fd (opendir: Permission denied) 2024-03-07T07:43:30.776 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74040 io_contex ceph cwd unknown /proc/74027/task/74040/cwd (readlink: Permission denied) 2024-03-07T07:43:30.776 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74040 io_contex ceph rtd unknown /proc/74027/task/74040/root (readlink: Permission denied) 2024-03-07T07:43:30.776 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74040 io_contex ceph txt unknown /proc/74027/task/74040/exe (readlink: Permission denied) 2024-03-07T07:43:30.776 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74040 io_contex ceph NOFD /proc/74027/task/74040/fd (opendir: Permission denied) 2024-03-07T07:43:30.776 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74041 ms_dispat ceph cwd unknown /proc/74027/task/74041/cwd (readlink: Permission denied) 2024-03-07T07:43:30.776 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74041 ms_dispat ceph rtd unknown /proc/74027/task/74041/root (readlink: Permission denied) 2024-03-07T07:43:30.776 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74041 ms_dispat ceph txt unknown /proc/74027/task/74041/exe (readlink: Permission denied) 2024-03-07T07:43:30.776 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74041 ms_dispat ceph NOFD /proc/74027/task/74041/fd (opendir: Permission denied) 2024-03-07T07:43:30.776 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74042 ms_local ceph cwd unknown /proc/74027/task/74042/cwd (readlink: Permission denied) 2024-03-07T07:43:30.776 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74042 ms_local ceph rtd unknown /proc/74027/task/74042/root (readlink: Permission denied) 2024-03-07T07:43:30.776 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74042 ms_local ceph txt unknown /proc/74027/task/74042/exe (readlink: Permission denied) 2024-03-07T07:43:30.776 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74042 ms_local ceph NOFD /proc/74027/task/74042/fd (opendir: Permission denied) 2024-03-07T07:43:30.776 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74043 safe_time ceph cwd unknown /proc/74027/task/74043/cwd (readlink: Permission denied) 2024-03-07T07:43:30.776 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74043 safe_time ceph rtd unknown /proc/74027/task/74043/root (readlink: Permission denied) 2024-03-07T07:43:30.777 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74043 safe_time ceph txt unknown /proc/74027/task/74043/exe (readlink: Permission denied) 2024-03-07T07:43:30.777 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74043 safe_time ceph NOFD /proc/74027/task/74043/fd (opendir: Permission denied) 2024-03-07T07:43:30.777 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74044 safe_time ceph cwd unknown /proc/74027/task/74044/cwd (readlink: Permission denied) 2024-03-07T07:43:30.777 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74044 safe_time ceph rtd unknown /proc/74027/task/74044/root (readlink: Permission denied) 2024-03-07T07:43:30.777 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74044 safe_time ceph txt unknown /proc/74027/task/74044/exe (readlink: Permission denied) 2024-03-07T07:43:30.777 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74044 safe_time ceph NOFD /proc/74027/task/74044/fd (opendir: Permission denied) 2024-03-07T07:43:30.777 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74045 ceph-mds ceph cwd unknown /proc/74027/task/74045/cwd (readlink: Permission denied) 2024-03-07T07:43:30.777 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74045 ceph-mds ceph rtd unknown /proc/74027/task/74045/root (readlink: Permission denied) 2024-03-07T07:43:30.777 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74045 ceph-mds ceph txt unknown /proc/74027/task/74045/exe (readlink: Permission denied) 2024-03-07T07:43:30.777 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74045 ceph-mds ceph NOFD /proc/74027/task/74045/fd (opendir: Permission denied) 2024-03-07T07:43:30.777 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74050 safe_time ceph cwd unknown /proc/74027/task/74050/cwd (readlink: Permission denied) 2024-03-07T07:43:30.777 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74050 safe_time ceph rtd unknown /proc/74027/task/74050/root (readlink: Permission denied) 2024-03-07T07:43:30.777 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74050 safe_time ceph txt unknown /proc/74027/task/74050/exe (readlink: Permission denied) 2024-03-07T07:43:30.778 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-mds 74027 74050 safe_time ceph NOFD /proc/74027/task/74050/fd (opendir: Permission denied) 2024-03-07T07:43:30.778 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/2 75151 root cwd unknown /proc/75151/cwd (readlink: Permission denied) 2024-03-07T07:43:30.778 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/2 75151 root rtd unknown /proc/75151/root (readlink: Permission denied) 2024-03-07T07:43:30.778 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/2 75151 root txt unknown /proc/75151/exe (readlink: Permission denied) 2024-03-07T07:43:30.778 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/2 75151 root NOFD /proc/75151/fd (opendir: Permission denied) 2024-03-07T07:43:30.778 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/3 76156 root cwd unknown /proc/76156/cwd (readlink: Permission denied) 2024-03-07T07:43:30.778 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/3 76156 root rtd unknown /proc/76156/root (readlink: Permission denied) 2024-03-07T07:43:30.778 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/3 76156 root txt unknown /proc/76156/exe (readlink: Permission denied) 2024-03-07T07:43:30.778 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/3 76156 root NOFD /proc/76156/fd (opendir: Permission denied) 2024-03-07T07:43:30.778 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/7 77948 root cwd unknown /proc/77948/cwd (readlink: Permission denied) 2024-03-07T07:43:30.778 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/7 77948 root rtd unknown /proc/77948/root (readlink: Permission denied) 2024-03-07T07:43:30.778 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/7 77948 root txt unknown /proc/77948/exe (readlink: Permission denied) 2024-03-07T07:43:30.778 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/7 77948 root NOFD /proc/77948/fd (opendir: Permission denied) 2024-03-07T07:43:30.778 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/u 79834 root cwd unknown /proc/79834/cwd (readlink: Permission denied) 2024-03-07T07:43:30.779 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/u 79834 root rtd unknown /proc/79834/root (readlink: Permission denied) 2024-03-07T07:43:30.779 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/u 79834 root txt unknown /proc/79834/exe (readlink: Permission denied) 2024-03-07T07:43:30.779 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/u 79834 root NOFD /proc/79834/fd (opendir: Permission denied) 2024-03-07T07:43:30.779 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/2 82522 root cwd unknown /proc/82522/cwd (readlink: Permission denied) 2024-03-07T07:43:30.779 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/2 82522 root rtd unknown /proc/82522/root (readlink: Permission denied) 2024-03-07T07:43:30.779 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/2 82522 root txt unknown /proc/82522/exe (readlink: Permission denied) 2024-03-07T07:43:30.779 INFO:tasks.workunit.client.0.smithi160.stdout:kworker/2 82522 root NOFD /proc/82522/fd (opendir: Permission denied) 2024-03-07T07:43:30.779 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-msgr 84088 root cwd unknown /proc/84088/cwd (readlink: Permission denied) 2024-03-07T07:43:30.779 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-msgr 84088 root rtd unknown /proc/84088/root (readlink: Permission denied) 2024-03-07T07:43:30.779 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-msgr 84088 root txt unknown /proc/84088/exe (readlink: Permission denied) 2024-03-07T07:43:30.779 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-msgr 84088 root NOFD /proc/84088/fd (opendir: Permission denied) 2024-03-07T07:43:30.779 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-watc 84089 root cwd unknown /proc/84089/cwd (readlink: Permission denied) 2024-03-07T07:43:30.779 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-watc 84089 root rtd unknown /proc/84089/root (readlink: Permission denied) 2024-03-07T07:43:30.779 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-watc 84089 root txt unknown /proc/84089/exe (readlink: Permission denied) 2024-03-07T07:43:30.780 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-watc 84089 root NOFD /proc/84089/fd (opendir: Permission denied) 2024-03-07T07:43:30.780 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-comp 84090 root cwd unknown /proc/84090/cwd (readlink: Permission denied) 2024-03-07T07:43:30.780 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-comp 84090 root rtd unknown /proc/84090/root (readlink: Permission denied) 2024-03-07T07:43:30.780 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-comp 84090 root txt unknown /proc/84090/exe (readlink: Permission denied) 2024-03-07T07:43:30.780 INFO:tasks.workunit.client.0.smithi160.stdout:ceph-comp 84090 root NOFD /proc/84090/fd (opendir: Permission denied) 2024-03-07T07:43:30.780 INFO:tasks.workunit.client.0.smithi160.stdout:sudo 84288 root cwd unknown /proc/84288/cwd (readlink: Permission denied) 2024-03-07T07:43:30.780 INFO:tasks.workunit.client.0.smithi160.stdout:sudo 84288 root rtd unknown /proc/84288/root (readlink: Permission denied) 2024-03-07T07:43:30.780 INFO:tasks.workunit.client.0.smithi160.stdout:sudo 84288 root txt unknown /proc/84288/exe (readlink: Permission denied) 2024-03-07T07:43:30.780 INFO:tasks.workunit.client.0.smithi160.stdout:sudo 84288 root NOFD /proc/84288/fd (opendir: Permission denied) 2024-03-07T07:43:30.780 INFO:tasks.workunit.client.0.smithi160.stdout:daemon-he 84313 root cwd unknown /proc/84313/cwd (readlink: Permission denied) 2024-03-07T07:43:30.780 INFO:tasks.workunit.client.0.smithi160.stdout:daemon-he 84313 root rtd unknown /proc/84313/root (readlink: Permission denied) 2024-03-07T07:43:30.780 INFO:tasks.workunit.client.0.smithi160.stdout:daemon-he 84313 root txt unknown /proc/84313/exe (readlink: Permission denied) 2024-03-07T07:43:30.780 INFO:tasks.workunit.client.0.smithi160.stdout:daemon-he 84313 root NOFD /proc/84313/fd (opendir: Permission denied) 2024-03-07T07:43:30.781 INFO:tasks.workunit.client.0.smithi160.stdout:bash 84322 root cwd unknown /proc/84322/cwd (readlink: Permission denied) 2024-03-07T07:43:30.781 INFO:tasks.workunit.client.0.smithi160.stdout:bash 84322 root rtd unknown /proc/84322/root (readlink: Permission denied) 2024-03-07T07:43:30.781 INFO:tasks.workunit.client.0.smithi160.stdout:bash 84322 root txt unknown /proc/84322/exe (readlink: Permission denied) 2024-03-07T07:43:30.781 INFO:tasks.workunit.client.0.smithi160.stdout:bash 84322 root NOFD /proc/84322/fd (opendir: Permission denied) 2024-03-07T07:43:30.781 INFO:tasks.workunit.client.0.smithi160.stdout:timeout 85187 ubuntu cwd DIR 0,119 4 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-07T07:43:30.781 INFO:tasks.workunit.client.0.smithi160.stdout:timeout 85187 ubuntu rtd DIR 8,1 4096 2 / 2024-03-07T07:43:30.781 INFO:tasks.workunit.client.0.smithi160.stdout:timeout 85187 ubuntu txt REG 8,1 40944 10098 /usr/bin/timeout 2024-03-07T07:43:30.781 INFO:tasks.workunit.client.0.smithi160.stdout:timeout 85187 ubuntu mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-03-07T07:43:30.781 INFO:tasks.workunit.client.0.smithi160.stdout:timeout 85187 ubuntu mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2024-03-07T07:43:30.781 INFO:tasks.workunit.client.0.smithi160.stdout:timeout 85187 ubuntu mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2024-03-07T07:43:30.781 INFO:tasks.workunit.client.0.smithi160.stdout:timeout 85187 ubuntu mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-03-07T07:43:30.781 INFO:tasks.workunit.client.0.smithi160.stdout:timeout 85187 ubuntu mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2024-03-07T07:43:30.781 INFO:tasks.workunit.client.0.smithi160.stdout:timeout 85187 ubuntu mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-03-07T07:43:30.781 INFO:tasks.workunit.client.0.smithi160.stdout:timeout 85187 ubuntu mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-03-07T07:43:30.782 INFO:tasks.workunit.client.0.smithi160.stdout:timeout 85187 ubuntu mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-03-07T07:43:30.782 INFO:tasks.workunit.client.0.smithi160.stdout:timeout 85187 ubuntu mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2024-03-07T07:43:30.782 INFO:tasks.workunit.client.0.smithi160.stdout:timeout 85187 ubuntu mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-03-07T07:43:30.782 INFO:tasks.workunit.client.0.smithi160.stdout:timeout 85187 ubuntu mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-03-07T07:43:30.782 INFO:tasks.workunit.client.0.smithi160.stdout:timeout 85187 ubuntu mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-03-07T07:43:30.782 INFO:tasks.workunit.client.0.smithi160.stdout:timeout 85187 ubuntu mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2024-03-07T07:43:30.782 INFO:tasks.workunit.client.0.smithi160.stdout:timeout 85187 ubuntu mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2024-03-07T07:43:30.782 INFO:tasks.workunit.client.0.smithi160.stdout:timeout 85187 ubuntu mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-03-07T07:43:30.782 INFO:tasks.workunit.client.0.smithi160.stdout:timeout 85187 ubuntu 0r FIFO 0,13 0t0 231601 pipe 2024-03-07T07:43:30.782 INFO:tasks.workunit.client.0.smithi160.stdout:timeout 85187 ubuntu 1w FIFO 0,13 0t0 231602 pipe 2024-03-07T07:43:30.782 INFO:tasks.workunit.client.0.smithi160.stdout:timeout 85187 ubuntu 2w FIFO 0,13 0t0 231603 pipe 2024-03-07T07:43:30.782 INFO:tasks.workunit.client.0.smithi160.stdout:fsync-tes 85205 ubuntu cwd DIR 0,119 4 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-07T07:43:30.782 INFO:tasks.workunit.client.0.smithi160.stdout:fsync-tes 85205 ubuntu rtd DIR 8,1 4096 2 / 2024-03-07T07:43:30.783 INFO:tasks.workunit.client.0.smithi160.stdout:fsync-tes 85205 ubuntu txt REG 8,1 1388920 9667 /usr/bin/bash 2024-03-07T07:43:30.783 INFO:tasks.workunit.client.0.smithi160.stdout:fsync-tes 85205 ubuntu mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-03-07T07:43:30.783 INFO:tasks.workunit.client.0.smithi160.stdout:fsync-tes 85205 ubuntu mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2024-03-07T07:43:30.783 INFO:tasks.workunit.client.0.smithi160.stdout:fsync-tes 85205 ubuntu mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2024-03-07T07:43:30.783 INFO:tasks.workunit.client.0.smithi160.stdout:fsync-tes 85205 ubuntu mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-03-07T07:43:30.783 INFO:tasks.workunit.client.0.smithi160.stdout:fsync-tes 85205 ubuntu mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2024-03-07T07:43:30.783 INFO:tasks.workunit.client.0.smithi160.stdout:fsync-tes 85205 ubuntu mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-03-07T07:43:30.783 INFO:tasks.workunit.client.0.smithi160.stdout:fsync-tes 85205 ubuntu mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-03-07T07:43:30.783 INFO:tasks.workunit.client.0.smithi160.stdout:fsync-tes 85205 ubuntu mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-03-07T07:43:30.783 INFO:tasks.workunit.client.0.smithi160.stdout:fsync-tes 85205 ubuntu mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2024-03-07T07:43:30.783 INFO:tasks.workunit.client.0.smithi160.stdout:fsync-tes 85205 ubuntu mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-03-07T07:43:30.783 INFO:tasks.workunit.client.0.smithi160.stdout:fsync-tes 85205 ubuntu mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-03-07T07:43:30.783 INFO:tasks.workunit.client.0.smithi160.stdout:fsync-tes 85205 ubuntu mem REG 8,1 195080 9664 /usr/lib64/libtinfo.so.6.2 2024-03-07T07:43:30.783 INFO:tasks.workunit.client.0.smithi160.stdout:fsync-tes 85205 ubuntu mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-03-07T07:43:30.784 INFO:tasks.workunit.client.0.smithi160.stdout:fsync-tes 85205 ubuntu mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2024-03-07T07:43:30.784 INFO:tasks.workunit.client.0.smithi160.stdout:fsync-tes 85205 ubuntu mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2024-03-07T07:43:30.784 INFO:tasks.workunit.client.0.smithi160.stdout:fsync-tes 85205 ubuntu mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-03-07T07:43:30.784 INFO:tasks.workunit.client.0.smithi160.stdout:fsync-tes 85205 ubuntu 0r FIFO 0,13 0t0 231601 pipe 2024-03-07T07:43:30.784 INFO:tasks.workunit.client.0.smithi160.stdout:fsync-tes 85205 ubuntu 1w FIFO 0,13 0t0 231602 pipe 2024-03-07T07:43:30.784 INFO:tasks.workunit.client.0.smithi160.stdout:fsync-tes 85205 ubuntu 2w FIFO 0,13 0t0 231603 pipe 2024-03-07T07:43:30.784 INFO:tasks.workunit.client.0.smithi160.stdout:fsync-tes 85205 ubuntu 255r REG 8,1 207 661291 /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsync-tester.sh 2024-03-07T07:43:30.784 INFO:tasks.workunit.client.0.smithi160.stdout:sleep 85676 root cwd unknown /proc/85676/cwd (readlink: Permission denied) 2024-03-07T07:43:30.784 INFO:tasks.workunit.client.0.smithi160.stdout:sleep 85676 root rtd unknown /proc/85676/root (readlink: Permission denied) 2024-03-07T07:43:30.784 INFO:tasks.workunit.client.0.smithi160.stdout:sleep 85676 root txt unknown /proc/85676/exe (readlink: Permission denied) 2024-03-07T07:43:30.784 INFO:tasks.workunit.client.0.smithi160.stdout:sleep 85676 root NOFD /proc/85676/fd (opendir: Permission denied) 2024-03-07T07:43:30.784 INFO:tasks.workunit.client.0.smithi160.stdout:lsof 85678 ubuntu cwd DIR 0,119 4 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-07T07:43:30.784 INFO:tasks.workunit.client.0.smithi160.stdout:lsof 85678 ubuntu rtd DIR 8,1 4096 2 / 2024-03-07T07:43:30.785 INFO:tasks.workunit.client.0.smithi160.stdout:lsof 85678 ubuntu txt REG 8,1 179232 12343 /usr/bin/lsof 2024-03-07T07:43:30.785 INFO:tasks.workunit.client.0.smithi160.stdout:lsof 85678 ubuntu mem REG 8,1 4487176 2250 /usr/lib64/libcrypto.so.3.0.7 2024-03-07T07:43:30.785 INFO:tasks.workunit.client.0.smithi160.stdout:lsof 85678 ubuntu mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2024-03-07T07:43:30.785 INFO:tasks.workunit.client.0.smithi160.stdout:lsof 85678 ubuntu mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2024-03-07T07:43:30.785 INFO:tasks.workunit.client.0.smithi160.stdout:lsof 85678 ubuntu mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2024-03-07T07:43:30.785 INFO:tasks.workunit.client.0.smithi160.stdout:lsof 85678 ubuntu mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2024-03-07T07:43:30.785 INFO:tasks.workunit.client.0.smithi160.stdout:lsof 85678 ubuntu mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2024-03-07T07:43:30.785 INFO:tasks.workunit.client.0.smithi160.stdout:lsof 85678 ubuntu mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2024-03-07T07:43:30.785 INFO:tasks.workunit.client.0.smithi160.stdout:lsof 85678 ubuntu mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2024-03-07T07:43:30.785 INFO:tasks.workunit.client.0.smithi160.stdout:lsof 85678 ubuntu mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2024-03-07T07:43:30.785 INFO:tasks.workunit.client.0.smithi160.stdout:lsof 85678 ubuntu mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2024-03-07T07:43:30.785 INFO:tasks.workunit.client.0.smithi160.stdout:lsof 85678 ubuntu mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2024-03-07T07:43:30.785 INFO:tasks.workunit.client.0.smithi160.stdout:lsof 85678 ubuntu mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2024-03-07T07:43:30.785 INFO:tasks.workunit.client.0.smithi160.stdout:lsof 85678 ubuntu mem REG 8,1 196112 11431 /usr/lib64/libtirpc.so.3.0.0 2024-03-07T07:43:30.786 INFO:tasks.workunit.client.0.smithi160.stdout:lsof 85678 ubuntu mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2024-03-07T07:43:30.786 INFO:tasks.workunit.client.0.smithi160.stdout:lsof 85678 ubuntu mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2024-03-07T07:43:30.786 INFO:tasks.workunit.client.0.smithi160.stdout:lsof 85678 ubuntu mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2024-03-07T07:43:30.786 INFO:tasks.workunit.client.0.smithi160.stdout:lsof 85678 ubuntu 0r FIFO 0,13 0t0 231601 pipe 2024-03-07T07:43:30.786 INFO:tasks.workunit.client.0.smithi160.stdout:lsof 85678 ubuntu 1w FIFO 0,13 0t0 231602 pipe 2024-03-07T07:43:30.786 INFO:tasks.workunit.client.0.smithi160.stdout:lsof 85678 ubuntu 2w FIFO 0,13 0t0 231603 pipe 2024-03-07T07:43:30.786 INFO:tasks.workunit.client.0.smithi160.stdout:lsof 85678 ubuntu 3r DIR 0,20 0 1 /proc 2024-03-07T07:43:30.786 INFO:tasks.workunit.client.0.smithi160.stdout:lsof 85678 ubuntu 4r DIR 0,20 0 230664 /proc/85678/fd 2024-03-07T07:43:30.786 INFO:tasks.workunit.client.0.smithi160.stdout:lsof 85678 ubuntu 5w FIFO 0,13 0t0 230669 pipe 2024-03-07T07:43:30.786 INFO:tasks.workunit.client.0.smithi160.stdout:lsof 85678 ubuntu 6r FIFO 0,13 0t0 230670 pipe 2024-03-07T07:43:30.786 INFO:tasks.workunit.client.0.smithi160.stdout:lsof 85679 ubuntu cwd DIR 0,119 4 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-07T07:43:30.786 INFO:tasks.workunit.client.0.smithi160.stdout:lsof 85679 ubuntu rtd DIR 8,1 4096 2 / 2024-03-07T07:43:30.786 INFO:tasks.workunit.client.0.smithi160.stdout:lsof 85679 ubuntu txt REG 8,1 179232 12343 /usr/bin/lsof 2024-03-07T07:43:30.787 INFO:tasks.workunit.client.0.smithi160.stdout:lsof 85679 ubuntu mem REG 8,1 4487176 2250 /usr/lib64/libcrypto.so.3.0.7 2024-03-07T07:43:30.787 INFO:tasks.workunit.client.0.smithi160.stdout:lsof 85679 ubuntu mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2024-03-07T07:43:30.787 INFO:tasks.workunit.client.0.smithi160.stdout:lsof 85679 ubuntu mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2024-03-07T07:43:30.787 INFO:tasks.workunit.client.0.smithi160.stdout:lsof 85679 ubuntu mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2024-03-07T07:43:30.787 INFO:tasks.workunit.client.0.smithi160.stdout:lsof 85679 ubuntu mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2024-03-07T07:43:30.787 INFO:tasks.workunit.client.0.smithi160.stdout:lsof 85679 ubuntu mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2024-03-07T07:43:30.787 INFO:tasks.workunit.client.0.smithi160.stdout:lsof 85679 ubuntu mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2024-03-07T07:43:30.787 INFO:tasks.workunit.client.0.smithi160.stdout:lsof 85679 ubuntu mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2024-03-07T07:43:30.787 INFO:tasks.workunit.client.0.smithi160.stdout:lsof 85679 ubuntu mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2024-03-07T07:43:30.787 INFO:tasks.workunit.client.0.smithi160.stdout:lsof 85679 ubuntu mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2024-03-07T07:43:30.787 INFO:tasks.workunit.client.0.smithi160.stdout:lsof 85679 ubuntu mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2024-03-07T07:43:30.787 INFO:tasks.workunit.client.0.smithi160.stdout:lsof 85679 ubuntu mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2024-03-07T07:43:30.787 INFO:tasks.workunit.client.0.smithi160.stdout:lsof 85679 ubuntu mem REG 8,1 196112 11431 /usr/lib64/libtirpc.so.3.0.0 2024-03-07T07:43:30.787 INFO:tasks.workunit.client.0.smithi160.stdout:lsof 85679 ubuntu mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2024-03-07T07:43:30.787 INFO:tasks.workunit.client.0.smithi160.stdout:lsof 85679 ubuntu mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2024-03-07T07:43:30.788 INFO:tasks.workunit.client.0.smithi160.stdout:lsof 85679 ubuntu mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2024-03-07T07:43:30.788 INFO:tasks.workunit.client.0.smithi160.stdout:lsof 85679 ubuntu 4r FIFO 0,13 0t0 230669 pipe 2024-03-07T07:43:30.788 INFO:tasks.workunit.client.0.smithi160.stdout:lsof 85679 ubuntu 7w FIFO 0,13 0t0 230670 pipe 2024-03-07T07:43:30.788 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-07T07:43:30.788 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-07T07:43:30.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:43:30 smithi178 ceph-mon[36650]: pgmap v418: 97 pgs: 97 active+clean; 258 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.3 MiB/s wr, 14 op/s 2024-03-07T07:43:30.814 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:43:30 smithi160 ceph-mon[31963]: pgmap v418: 97 pgs: 97 active+clean; 258 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.3 MiB/s wr, 14 op/s 2024-03-07T07:43:30.837 INFO:tasks.workunit:Stopping ['suites/fsync-tester.sh'] on client.0... 2024-03-07T07:43:30.837 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-07T07:43:30.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:43:30 smithi177 ceph-mon[36578]: pgmap v418: 97 pgs: 97 active+clean; 258 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.3 MiB/s wr, 14 op/s 2024-03-07T07:43:31.146 DEBUG:teuthology.parallel:result is None 2024-03-07T07:43:31.146 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-07T07:43:31.182 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-07T07:43:31.182 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-03-07T07:43:31.193 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi160.front.sepia.ceph.com 2024-03-07T07:43:31.230 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-03-07T07:43:31.242 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T07:43:31.376 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.374+0000 7f78bf483640 1 -- 172.21.15.160:0/21984755 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f78b81028a0 msgr2=0x7f78b8102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:43:31.376 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.374+0000 7f78bf483640 1 --2- 172.21.15.160:0/21984755 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f78b81028a0 0x7f78b8102d20 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f78a8009900 tx=0x7f78a802bc30 comp rx=0 tx=0).stop 2024-03-07T07:43:31.376 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.374+0000 7f78bf483640 1 -- 172.21.15.160:0/21984755 shutdown_connections 2024-03-07T07:43:31.376 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.374+0000 7f78bf483640 1 --2- 172.21.15.160:0/21984755 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78b8103260 0x7f78b8109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:31.377 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.374+0000 7f78bf483640 1 --2- 172.21.15.160:0/21984755 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f78b81028a0 0x7f78b8102d20 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:31.377 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.374+0000 7f78bf483640 1 --2- 172.21.15.160:0/21984755 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f78b81016b0 0x7f78b8101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:31.377 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.374+0000 7f78bf483640 1 -- 172.21.15.160:0/21984755 >> 172.21.15.160:0/21984755 conn(0x7f78b80fcc00 msgr2=0x7f78b80ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:43:31.377 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.374+0000 7f78bf483640 1 -- 172.21.15.160:0/21984755 shutdown_connections 2024-03-07T07:43:31.377 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.374+0000 7f78bf483640 1 -- 172.21.15.160:0/21984755 wait complete. 2024-03-07T07:43:31.377 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.374+0000 7f78bf483640 1 Processor -- start 2024-03-07T07:43:31.377 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.374+0000 7f78bf483640 1 -- start start 2024-03-07T07:43:31.377 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.375+0000 7f78bf483640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f78b81016b0 0x7f78b8075410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:31.377 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.375+0000 7f78bf483640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f78b81028a0 0x7f78b8073a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:31.377 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.375+0000 7f78bf483640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78b8103260 0x7f78b8073f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:31.377 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.375+0000 7f78bf483640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b810d310 con 0x7f78b81028a0 2024-03-07T07:43:31.377 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.375+0000 7f78bf483640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b810d190 con 0x7f78b81016b0 2024-03-07T07:43:31.377 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.375+0000 7f78bf483640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b810d490 con 0x7f78b8103260 2024-03-07T07:43:31.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.375+0000 7f78bca1e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f78b81028a0 0x7f78b8073a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:31.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.375+0000 7f78bd21f640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f78b81016b0 0x7f78b8075410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:31.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.375+0000 7f78bca1e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f78b81028a0 0x7f78b8073a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34170/0 (socket says 172.21.15.160:34170) 2024-03-07T07:43:31.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.375+0000 7f78bca1e640 1 -- 172.21.15.160:0/181904682 learned_addr learned my addr 172.21.15.160:0/181904682 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:43:31.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.375+0000 7f78bda20640 1 --2- 172.21.15.160:0/181904682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78b8103260 0x7f78b8073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:31.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.375+0000 7f78bca1e640 1 -- 172.21.15.160:0/181904682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78b8103260 msgr2=0x7f78b8073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:43:31.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.375+0000 7f78bca1e640 1 --2- 172.21.15.160:0/181904682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78b8103260 0x7f78b8073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:31.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.375+0000 7f78bca1e640 1 -- 172.21.15.160:0/181904682 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f78b81016b0 msgr2=0x7f78b8075410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:43:31.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.375+0000 7f78bca1e640 1 --2- 172.21.15.160:0/181904682 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f78b81016b0 0x7f78b8075410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:31.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.375+0000 7f78bca1e640 1 -- 172.21.15.160:0/181904682 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b8074820 con 0x7f78b81028a0 2024-03-07T07:43:31.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.375+0000 7f78bca1e640 1 --2- 172.21.15.160:0/181904682 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f78b81028a0 0x7f78b8073a50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f78a8009230 tx=0x7f78a802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:43:31.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.375+0000 7f78ae7fc640 1 -- 172.21.15.160:0/181904682 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a8041070 con 0x7f78b81028a0 2024-03-07T07:43:31.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.375+0000 7f78bf483640 1 -- 172.21.15.160:0/181904682 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78b81a1a90 con 0x7f78b81028a0 2024-03-07T07:43:31.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.375+0000 7f78ae7fc640 1 -- 172.21.15.160:0/181904682 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f78a8004450 con 0x7f78b81028a0 2024-03-07T07:43:31.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.375+0000 7f78ae7fc640 1 -- 172.21.15.160:0/181904682 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a8003db0 con 0x7f78b81028a0 2024-03-07T07:43:31.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.375+0000 7f78bf483640 1 -- 172.21.15.160:0/181904682 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78b81a1fe0 con 0x7f78b81028a0 2024-03-07T07:43:31.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.376+0000 7f78bf483640 1 -- 172.21.15.160:0/181904682 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78b81089d0 con 0x7f78b81028a0 2024-03-07T07:43:31.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.377+0000 7f78ae7fc640 1 -- 172.21.15.160:0/181904682 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 20) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f78a8044070 con 0x7f78b81028a0 2024-03-07T07:43:31.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.378+0000 7f78ae7fc640 1 --2- 172.21.15.160:0/181904682 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f78a40a1d80 0x7f78a40a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:31.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.378+0000 7f78bd21f640 1 --2- 172.21.15.160:0/181904682 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f78a40a1d80 0x7f78a40a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:31.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.378+0000 7f78ae7fc640 1 -- 172.21.15.160:0/181904682 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f78a80ef980 con 0x7f78b81028a0 2024-03-07T07:43:31.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.378+0000 7f78bd21f640 1 --2- 172.21.15.160:0/181904682 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f78a40a1d80 0x7f78a40a4240 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f78b8102700 tx=0x7f78b0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:43:31.383 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.382+0000 7f78ae7fc640 1 -- 172.21.15.160:0/181904682 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f78a80bcdb0 con 0x7f78b81028a0 2024-03-07T07:43:31.558 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.557+0000 7f78bf483640 1 -- 172.21.15.160:0/181904682 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f78b8062260 con 0x7f78b81028a0 2024-03-07T07:43:31.559 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.559+0000 7f78ae7fc640 1 -- 172.21.15.160:0/181904682 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22721 (secure 0 0 0) 0x7f78a80dae20 con 0x7f78b81028a0 2024-03-07T07:43:31.559 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:43:31.559 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":87,"fsid":"826f6582-dc54-11ee-95c6-87774f69a715","created":"2024-03-07T07:31:14.229288+0000","modified":"2024-03-07T07:40:09.354265+0000","last_up_change":"2024-03-07T07:38:20.776225+0000","last_in_change":"2024-03-07T07:38:00.508917+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T07:34:34.529424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T07:39:12.075432+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T07:39:13.367820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bee66c67-fb6e-4717-9685-ca62587d5f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6803","nonce":1368482926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6805","nonce":1368482926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6809","nonce":1368482926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6807","nonce":1368482926}]},"public_addr":"172.21.15.160:6803/1368482926","cluster_addr":"172.21.15.160:6805/1368482926","heartbeat_back_addr":"172.21.15.160:6809/1368482926","heartbeat_front_addr":"172.21.15.160:6807/1368482926","state":["exists","up"]},{"osd":1,"uuid":"49236251-f750-424e-992b-bddf55762556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6801","nonce":2922391884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6803","nonce":2922391884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6807","nonce":2922391884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6805","nonce":2922391884}]},"public_addr":"172.21.15.177:6801/2922391884","cluster_addr":"172.21.15.177:6803/2922391884","heartbeat_back_addr":"172.21.15.177:6807/2922391884","heartbeat_front_addr":"172.21.15.177:6805/2922391884","state":["exists","up"]},{"osd":2,"uuid":"7a432b28-f7ee-43fe-9196-7aaeccfecf77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6801","nonce":1217901551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6803","nonce":1217901551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6807","nonce":1217901551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6805","nonce":1217901551}]},"public_addr":"172.21.15.178:6801/1217901551","cluster_addr":"172.21.15.178:6803/1217901551","heartbeat_back_addr":"172.21.15.178:6807/1217901551","heartbeat_front_addr":"172.21.15.178:6805/1217901551","state":["exists","up"]},{"osd":3,"uuid":"4ad22a20-f1be-4002-a48d-ecd3953e63f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6811","nonce":42543989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6813","nonce":42543989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6817","nonce":42543989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6815","nonce":42543989}]},"public_addr":"172.21.15.160:6811/42543989","cluster_addr":"172.21.15.160:6813/42543989","heartbeat_back_addr":"172.21.15.160:6817/42543989","heartbeat_front_addr":"172.21.15.160:6815/42543989","state":["exists","up"]},{"osd":4,"uuid":"2b65b104-ac82-4ce3-b6c5-11a399103045","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6809","nonce":88178553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6811","nonce":88178553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6815","nonce":88178553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6813","nonce":88178553}]},"public_addr":"172.21.15.177:6809/88178553","cluster_addr":"172.21.15.177:6811/88178553","heartbeat_back_addr":"172.21.15.177:6815/88178553","heartbeat_front_addr":"172.21.15.177:6813/88178553","state":["exists","up"]},{"osd":5,"uuid":"30a1e714-08ff-44e8-9264-9ef164604b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6809","nonce":964732891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6811","nonce":964732891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6815","nonce":964732891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6813","nonce":964732891}]},"public_addr":"172.21.15.178:6809/964732891","cluster_addr":"172.21.15.178:6811/964732891","heartbeat_back_addr":"172.21.15.178:6815/964732891","heartbeat_front_addr":"172.21.15.178:6813/964732891","state":["exists","up"]},{"osd":6,"uuid":"b1e1646e-f8a9-42b8-b62d-9d7b5d6a1ff4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6819","nonce":3251327001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6821","nonce":3251327001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6825","nonce":3251327001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6823","nonce":3251327001}]},"public_addr":"172.21.15.160:6819/3251327001","cluster_addr":"172.21.15.160:6821/3251327001","heartbeat_back_addr":"172.21.15.160:6825/3251327001","heartbeat_front_addr":"172.21.15.160:6823/3251327001","state":["exists","up"]},{"osd":7,"uuid":"e66b400b-9747-4f6d-b05b-db22468e2750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6817","nonce":1175500850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6819","nonce":1175500850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6823","nonce":1175500850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6821","nonce":1175500850}]},"public_addr":"172.21.15.177:6817/1175500850","cluster_addr":"172.21.15.177:6819/1175500850","heartbeat_back_addr":"172.21.15.177:6823/1175500850","heartbeat_front_addr":"172.21.15.177:6821/1175500850","state":["exists","up"]},{"osd":8,"uuid":"3ee0f2c4-fca4-49fc-a98b-07aa0fcba0e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6817","nonce":688015531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6819","nonce":688015531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6823","nonce":688015531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6821","nonce":688015531}]},"public_addr":"172.21.15.178:6817/688015531","cluster_addr":"172.21.15.178:6819/688015531","heartbeat_back_addr":"172.21.15.178:6823/688015531","heartbeat_front_addr":"172.21.15.178:6821/688015531","state":["exists","up"]},{"osd":9,"uuid":"856039d7-f7ee-4dbd-b019-0073e705ce0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6827","nonce":3265797851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6829","nonce":3265797851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6833","nonce":3265797851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6831","nonce":3265797851}]},"public_addr":"172.21.15.160:6827/3265797851","cluster_addr":"172.21.15.160:6829/3265797851","heartbeat_back_addr":"172.21.15.160:6833/3265797851","heartbeat_front_addr":"172.21.15.160:6831/3265797851","state":["exists","up"]},{"osd":10,"uuid":"bdc9899d-d720-4429-ad53-4c388a4dbe75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6825","nonce":2867473412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6827","nonce":2867473412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6831","nonce":2867473412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6829","nonce":2867473412}]},"public_addr":"172.21.15.177:6825/2867473412","cluster_addr":"172.21.15.177:6827/2867473412","heartbeat_back_addr":"172.21.15.177:6831/2867473412","heartbeat_front_addr":"172.21.15.177:6829/2867473412","state":["exists","up"]},{"osd":11,"uuid":"cc9adb54-ef49-47c6-90e3-fc03bc0f5cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6825","nonce":4068724731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6827","nonce":4068724731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6831","nonce":4068724731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6829","nonce":4068724731}]},"public_addr":"172.21.15.178:6825/4068724731","cluster_addr":"172.21.15.178:6827/4068724731","heartbeat_back_addr":"172.21.15.178:6831/4068724731","heartbeat_front_addr":"172.21.15.178:6829/4068724731","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:33:44.141585+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:07.685585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:31.044386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:56.308925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:35:21.582097+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:35:45.983540+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:36:11.530242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:36:36.844681+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:02.303437+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:27.629867+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:53.289495+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:38:18.536169+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/3027450612":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/874709722":"2024-03-08T07:32:08.190837+0000","172.21.15.160:6800/1130675225":"2024-03-08T07:31:41.166858+0000","172.21.15.160:0/3857193980":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/3861968297":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6800/3859174315":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/3013534743":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6801/3859174315":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/1310937667":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6801/1130675225":"2024-03-08T07:31:41.166858+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T07:43:31.561 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.561+0000 7f78bf483640 1 -- 172.21.15.160:0/181904682 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f78a40a1d80 msgr2=0x7f78a40a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:43:31.561 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.561+0000 7f78bf483640 1 --2- 172.21.15.160:0/181904682 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f78a40a1d80 0x7f78a40a4240 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f78b8102700 tx=0x7f78b0009500 comp rx=0 tx=0).stop 2024-03-07T07:43:31.561 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.561+0000 7f78bf483640 1 -- 172.21.15.160:0/181904682 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f78b81028a0 msgr2=0x7f78b8073a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:43:31.562 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.561+0000 7f78bf483640 1 --2- 172.21.15.160:0/181904682 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f78b81028a0 0x7f78b8073a50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f78a8009230 tx=0x7f78a802c650 comp rx=0 tx=0).stop 2024-03-07T07:43:31.562 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.561+0000 7f78bf483640 1 -- 172.21.15.160:0/181904682 shutdown_connections 2024-03-07T07:43:31.562 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.561+0000 7f78bf483640 1 --2- 172.21.15.160:0/181904682 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f78a40a1d80 0x7f78a40a4240 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:31.562 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.561+0000 7f78bf483640 1 --2- 172.21.15.160:0/181904682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78b8103260 0x7f78b8073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:31.562 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.561+0000 7f78bf483640 1 --2- 172.21.15.160:0/181904682 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f78b81028a0 0x7f78b8073a50 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:31.562 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.561+0000 7f78bf483640 1 --2- 172.21.15.160:0/181904682 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f78b81016b0 0x7f78b8075410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:31.562 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.561+0000 7f78bf483640 1 -- 172.21.15.160:0/181904682 >> 172.21.15.160:0/181904682 conn(0x7f78b80fcc00 msgr2=0x7f78b8105ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:43:31.562 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.561+0000 7f78bf483640 1 -- 172.21.15.160:0/181904682 shutdown_connections 2024-03-07T07:43:31.562 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.561+0000 7f78bf483640 1 -- 172.21.15.160:0/181904682 wait complete. 2024-03-07T07:43:31.573 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-07T07:34:34.529424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-07T07:39:12.075432+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-07T07:39:13.367820+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-07T07:43:31.573 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-07T07:43:31.679 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.678+0000 7f3b8cc34640 1 -- 172.21.15.160:0/4284051237 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b88103230 msgr2=0x7f3b88109ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:43:31.679 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.678+0000 7f3b8cc34640 1 --2- 172.21.15.160:0/4284051237 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b88103230 0x7f3b88109ac0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3b80009900 tx=0x7f3b8002bc30 comp rx=0 tx=0).stop 2024-03-07T07:43:31.679 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.678+0000 7f3b8cc34640 1 -- 172.21.15.160:0/4284051237 shutdown_connections 2024-03-07T07:43:31.679 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.678+0000 7f3b8cc34640 1 --2- 172.21.15.160:0/4284051237 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b88103230 0x7f3b88109ac0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:31.679 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.678+0000 7f3b8cc34640 1 --2- 172.21.15.160:0/4284051237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b88102870 0x7f3b88102cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:31.679 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.678+0000 7f3b8cc34640 1 --2- 172.21.15.160:0/4284051237 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b88101680 0x7f3b88101aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:31.679 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.678+0000 7f3b8cc34640 1 -- 172.21.15.160:0/4284051237 >> 172.21.15.160:0/4284051237 conn(0x7f3b880fcbf0 msgr2=0x7f3b880ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:43:31.679 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.678+0000 7f3b8cc34640 1 -- 172.21.15.160:0/4284051237 shutdown_connections 2024-03-07T07:43:31.679 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.678+0000 7f3b8cc34640 1 -- 172.21.15.160:0/4284051237 wait complete. 2024-03-07T07:43:31.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.679+0000 7f3b8cc34640 1 Processor -- start 2024-03-07T07:43:31.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.679+0000 7f3b8cc34640 1 -- start start 2024-03-07T07:43:31.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.679+0000 7f3b8cc34640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b88101680 0x7f3b881978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:31.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.679+0000 7f3b8cc34640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b88102870 0x7f3b88197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:31.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.679+0000 7f3b8cc34640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b88103230 0x7f3b8819eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:31.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.679+0000 7f3b8cc34640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8810d2e0 con 0x7f3b88101680 2024-03-07T07:43:31.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.679+0000 7f3b8cc34640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8810d160 con 0x7f3b88103230 2024-03-07T07:43:31.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.679+0000 7f3b8cc34640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8810d460 con 0x7f3b88102870 2024-03-07T07:43:31.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.680+0000 7f3b8659c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b88101680 0x7f3b881978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:31.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.680+0000 7f3b8659c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b88101680 0x7f3b881978f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34192/0 (socket says 172.21.15.160:34192) 2024-03-07T07:43:31.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.680+0000 7f3b8659c640 1 -- 172.21.15.160:0/2600590607 learned_addr learned my addr 172.21.15.160:0/2600590607 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:43:31.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.680+0000 7f3b85d9b640 1 --2- 172.21.15.160:0/2600590607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b88102870 0x7f3b88197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:31.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.680+0000 7f3b86d9d640 1 --2- 172.21.15.160:0/2600590607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b88103230 0x7f3b8819eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:31.683 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.680+0000 7f3b8659c640 1 -- 172.21.15.160:0/2600590607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b88102870 msgr2=0x7f3b88197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:43:31.683 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.680+0000 7f3b8659c640 1 --2- 172.21.15.160:0/2600590607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b88102870 0x7f3b88197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:31.683 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.680+0000 7f3b8659c640 1 -- 172.21.15.160:0/2600590607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b88103230 msgr2=0x7f3b8819eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:43:31.683 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.680+0000 7f3b8659c640 1 --2- 172.21.15.160:0/2600590607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b88103230 0x7f3b8819eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:31.683 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.680+0000 7f3b8659c640 1 -- 172.21.15.160:0/2600590607 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b8819f510 con 0x7f3b88101680 2024-03-07T07:43:31.683 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.680+0000 7f3b85d9b640 1 --2- 172.21.15.160:0/2600590607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b88102870 0x7f3b88197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:43:31.683 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.680+0000 7f3b8659c640 1 --2- 172.21.15.160:0/2600590607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b88101680 0x7f3b881978f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3b7800ebc0 tx=0x7f3b7800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:43:31.683 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.680+0000 7f3b777fe640 1 -- 172.21.15.160:0/2600590607 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b78003b70 con 0x7f3b88101680 2024-03-07T07:43:31.683 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.680+0000 7f3b8cc34640 1 -- 172.21.15.160:0/2600590607 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b8819f800 con 0x7f3b88101680 2024-03-07T07:43:31.683 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.680+0000 7f3b777fe640 1 -- 172.21.15.160:0/2600590607 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b78003d10 con 0x7f3b88101680 2024-03-07T07:43:31.683 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.680+0000 7f3b777fe640 1 -- 172.21.15.160:0/2600590607 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b78019980 con 0x7f3b88101680 2024-03-07T07:43:31.683 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.680+0000 7f3b8cc34640 1 -- 172.21.15.160:0/2600590607 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b8819fd50 con 0x7f3b88101680 2024-03-07T07:43:31.683 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.681+0000 7f3b8cc34640 1 -- 172.21.15.160:0/2600590607 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b881089a0 con 0x7f3b88101680 2024-03-07T07:43:31.684 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.683+0000 7f3b777fe640 1 -- 172.21.15.160:0/2600590607 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 20) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f3b78019b20 con 0x7f3b88101680 2024-03-07T07:43:31.684 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.683+0000 7f3b777fe640 1 --2- 172.21.15.160:0/2600590607 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f3b700a1d80 0x7f3b700a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:31.684 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.684+0000 7f3b85d9b640 1 --2- 172.21.15.160:0/2600590607 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f3b700a1d80 0x7f3b700a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:31.684 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.684+0000 7f3b777fe640 1 -- 172.21.15.160:0/2600590607 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3b780ce690 con 0x7f3b88101680 2024-03-07T07:43:31.684 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.684+0000 7f3b85d9b640 1 --2- 172.21.15.160:0/2600590607 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f3b700a1d80 0x7f3b700a4240 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c0095f0 tx=0x7f3b7c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:43:31.687 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.687+0000 7f3b777fe640 1 -- 172.21.15.160:0/2600590607 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3b7809bb50 con 0x7f3b88101680 2024-03-07T07:43:31.861 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.861+0000 7f3b8cc34640 1 -- 172.21.15.160:0/2600590607 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3b88062260 con 0x7f3b88101680 2024-03-07T07:43:31.861 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.861+0000 7f3b777fe640 1 -- 172.21.15.160:0/2600590607 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3b7809b450 con 0x7f3b88101680 2024-03-07T07:43:31.861 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 1 2024-03-07T07:43:31.864 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.863+0000 7f3b8cc34640 1 -- 172.21.15.160:0/2600590607 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f3b700a1d80 msgr2=0x7f3b700a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:43:31.864 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.863+0000 7f3b8cc34640 1 --2- 172.21.15.160:0/2600590607 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f3b700a1d80 0x7f3b700a4240 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c0095f0 tx=0x7f3b7c005f90 comp rx=0 tx=0).stop 2024-03-07T07:43:31.864 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.863+0000 7f3b8cc34640 1 -- 172.21.15.160:0/2600590607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b88101680 msgr2=0x7f3b881978f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:43:31.864 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.863+0000 7f3b8cc34640 1 --2- 172.21.15.160:0/2600590607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b88101680 0x7f3b881978f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3b7800ebc0 tx=0x7f3b7800c6a0 comp rx=0 tx=0).stop 2024-03-07T07:43:31.864 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.863+0000 7f3b8cc34640 1 -- 172.21.15.160:0/2600590607 shutdown_connections 2024-03-07T07:43:31.864 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.863+0000 7f3b8cc34640 1 --2- 172.21.15.160:0/2600590607 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f3b700a1d80 0x7f3b700a4240 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:31.864 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.863+0000 7f3b8cc34640 1 --2- 172.21.15.160:0/2600590607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b88103230 0x7f3b8819eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:31.864 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.863+0000 7f3b8cc34640 1 --2- 172.21.15.160:0/2600590607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b88102870 0x7f3b88197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:31.864 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.863+0000 7f3b8cc34640 1 --2- 172.21.15.160:0/2600590607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b88101680 0x7f3b881978f0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:31.864 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.863+0000 7f3b8cc34640 1 -- 172.21.15.160:0/2600590607 >> 172.21.15.160:0/2600590607 conn(0x7f3b880fcbf0 msgr2=0x7f3b88105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:43:31.864 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.863+0000 7f3b8cc34640 1 -- 172.21.15.160:0/2600590607 shutdown_connections 2024-03-07T07:43:31.865 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.863+0000 7f3b8cc34640 1 -- 172.21.15.160:0/2600590607 wait complete. 2024-03-07T07:43:31.874 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:43:31 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/181904682' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:43:31.875 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-07T07:43:31.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.979+0000 7f1837a34640 1 -- 172.21.15.160:0/1887806084 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1830101770 msgr2=0x7f1830101bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:43:31.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.979+0000 7f1837a34640 1 --2- 172.21.15.160:0/1887806084 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1830101770 0x7f1830101bf0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1820009900 tx=0x7f182002bc30 comp rx=0 tx=0).stop 2024-03-07T07:43:31.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.980+0000 7f1837a34640 1 -- 172.21.15.160:0/1887806084 shutdown_connections 2024-03-07T07:43:31.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.980+0000 7f1837a34640 1 --2- 172.21.15.160:0/1887806084 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1830102130 0x7f18301089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:31.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.980+0000 7f1837a34640 1 --2- 172.21.15.160:0/1887806084 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1830101770 0x7f1830101bf0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:31.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.980+0000 7f1837a34640 1 --2- 172.21.15.160:0/1887806084 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1830100580 0x7f18301009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:31.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.980+0000 7f1837a34640 1 -- 172.21.15.160:0/1887806084 >> 172.21.15.160:0/1887806084 conn(0x7f18300fbb10 msgr2=0x7f18300fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:43:31.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.980+0000 7f1837a34640 1 -- 172.21.15.160:0/1887806084 shutdown_connections 2024-03-07T07:43:31.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.980+0000 7f1837a34640 1 -- 172.21.15.160:0/1887806084 wait complete. 2024-03-07T07:43:31.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.980+0000 7f1837a34640 1 Processor -- start 2024-03-07T07:43:31.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.980+0000 7f1837a34640 1 -- start start 2024-03-07T07:43:31.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.980+0000 7f1837a34640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1830100580 0x7f183006a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:31.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.980+0000 7f1837a34640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1830101770 0x7f183019c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:31.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.980+0000 7f1837a34640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1830102130 0x7f183019ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:31.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.980+0000 7f1837a34640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f183010c210 con 0x7f1830100580 2024-03-07T07:43:31.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.980+0000 7f1837a34640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f183010c090 con 0x7f1830102130 2024-03-07T07:43:31.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.980+0000 7f1837a34640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f183010c390 con 0x7f1830101770 2024-03-07T07:43:31.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.980+0000 7f18357d0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1830100580 0x7f183006a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:31.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.980+0000 7f18357d0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1830100580 0x7f183006a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34212/0 (socket says 172.21.15.160:34212) 2024-03-07T07:43:31.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.980+0000 7f18357d0640 1 -- 172.21.15.160:0/1689018960 learned_addr learned my addr 172.21.15.160:0/1689018960 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:43:31.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.980+0000 7f1834fcf640 1 --2- 172.21.15.160:0/1689018960 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1830101770 0x7f183019c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:31.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.981+0000 7f1835fd1640 1 --2- 172.21.15.160:0/1689018960 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1830102130 0x7f183019ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:31.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.981+0000 7f18357d0640 1 -- 172.21.15.160:0/1689018960 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1830101770 msgr2=0x7f183019c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:43:31.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.981+0000 7f18357d0640 1 --2- 172.21.15.160:0/1689018960 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1830101770 0x7f183019c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:31.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.981+0000 7f18357d0640 1 -- 172.21.15.160:0/1689018960 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1830102130 msgr2=0x7f183019ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:43:31.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.981+0000 7f18357d0640 1 --2- 172.21.15.160:0/1689018960 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1830102130 0x7f183019ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:31.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.981+0000 7f18357d0640 1 -- 172.21.15.160:0/1689018960 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f183019f5b0 con 0x7f1830100580 2024-03-07T07:43:31.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.981+0000 7f18357d0640 1 --2- 172.21.15.160:0/1689018960 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1830100580 0x7f183006a7c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f182c00e9c0 tx=0x7f182c00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:43:31.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.981+0000 7f18267fc640 1 -- 172.21.15.160:0/1689018960 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c00cd80 con 0x7f1830100580 2024-03-07T07:43:31.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.981+0000 7f18267fc640 1 -- 172.21.15.160:0/1689018960 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f182c010910 con 0x7f1830100580 2024-03-07T07:43:31.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.981+0000 7f1837a34640 1 -- 172.21.15.160:0/1689018960 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f183019f8a0 con 0x7f1830100580 2024-03-07T07:43:31.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.981+0000 7f18267fc640 1 -- 172.21.15.160:0/1689018960 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c018950 con 0x7f1830100580 2024-03-07T07:43:31.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.981+0000 7f1837a34640 1 -- 172.21.15.160:0/1689018960 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f183019fdc0 con 0x7f1830100580 2024-03-07T07:43:31.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.982+0000 7f1837a34640 1 -- 172.21.15.160:0/1689018960 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1830107920 con 0x7f1830100580 2024-03-07T07:43:31.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.983+0000 7f18267fc640 1 -- 172.21.15.160:0/1689018960 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 20) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f182c01f020 con 0x7f1830100580 2024-03-07T07:43:31.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.983+0000 7f18267fc640 1 --2- 172.21.15.160:0/1689018960 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f181c0a1d80 0x7f181c0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:31.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.984+0000 7f1834fcf640 1 --2- 172.21.15.160:0/1689018960 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f181c0a1d80 0x7f181c0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:31.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.984+0000 7f18267fc640 1 -- 172.21.15.160:0/1689018960 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f182c0cde20 con 0x7f1830100580 2024-03-07T07:43:31.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.984+0000 7f1834fcf640 1 --2- 172.21.15.160:0/1689018960 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f181c0a1d80 0x7f181c0a4240 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f182002c7b0 tx=0x7f1820005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:43:31.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:31.988+0000 7f18267fc640 1 -- 172.21.15.160:0/1689018960 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f182c09b250 con 0x7f1830100580 2024-03-07T07:43:32.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:43:31 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/181904682' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:43:32.106 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:43:31 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/181904682' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:43:32.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.164+0000 7f1837a34640 1 -- 172.21.15.160:0/1689018960 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1830062260 con 0x7f1830100580 2024-03-07T07:43:32.165 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.165+0000 7f18267fc640 1 -- 172.21.15.160:0/1689018960 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f182c09ab50 con 0x7f1830100580 2024-03-07T07:43:32.165 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 32 2024-03-07T07:43:32.167 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.167+0000 7f1837a34640 1 -- 172.21.15.160:0/1689018960 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f181c0a1d80 msgr2=0x7f181c0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:43:32.167 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.167+0000 7f1837a34640 1 --2- 172.21.15.160:0/1689018960 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f181c0a1d80 0x7f181c0a4240 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f182002c7b0 tx=0x7f1820005fd0 comp rx=0 tx=0).stop 2024-03-07T07:43:32.167 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.167+0000 7f1837a34640 1 -- 172.21.15.160:0/1689018960 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1830100580 msgr2=0x7f183006a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:43:32.167 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.167+0000 7f1837a34640 1 --2- 172.21.15.160:0/1689018960 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1830100580 0x7f183006a7c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f182c00e9c0 tx=0x7f182c00edc0 comp rx=0 tx=0).stop 2024-03-07T07:43:32.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.167+0000 7f1837a34640 1 -- 172.21.15.160:0/1689018960 shutdown_connections 2024-03-07T07:43:32.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.167+0000 7f1837a34640 1 --2- 172.21.15.160:0/1689018960 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f181c0a1d80 0x7f181c0a4240 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:32.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.167+0000 7f1837a34640 1 --2- 172.21.15.160:0/1689018960 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1830102130 0x7f183019ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:32.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.167+0000 7f1837a34640 1 --2- 172.21.15.160:0/1689018960 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1830101770 0x7f183019c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:32.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.167+0000 7f1837a34640 1 --2- 172.21.15.160:0/1689018960 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1830100580 0x7f183006a7c0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:32.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.167+0000 7f1837a34640 1 -- 172.21.15.160:0/1689018960 >> 172.21.15.160:0/1689018960 conn(0x7f18300fbb10 msgr2=0x7f1830104af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:43:32.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.167+0000 7f1837a34640 1 -- 172.21.15.160:0/1689018960 shutdown_connections 2024-03-07T07:43:32.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.167+0000 7f1837a34640 1 -- 172.21.15.160:0/1689018960 wait complete. 2024-03-07T07:43:32.178 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-07T07:43:32.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.284+0000 7f4786fc9640 1 -- 172.21.15.160:0/3889985590 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4780100570 msgr2=0x7f4780100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:43:32.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.284+0000 7f4786fc9640 1 --2- 172.21.15.160:0/3889985590 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4780100570 0x7f4780100990 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4774009900 tx=0x7f477402bc30 comp rx=0 tx=0).stop 2024-03-07T07:43:32.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.284+0000 7f4786fc9640 1 -- 172.21.15.160:0/3889985590 shutdown_connections 2024-03-07T07:43:32.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.284+0000 7f4786fc9640 1 --2- 172.21.15.160:0/3889985590 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4780102120 0x7f47801089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:32.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.284+0000 7f4786fc9640 1 --2- 172.21.15.160:0/3889985590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4780101760 0x7f4780101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:32.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.284+0000 7f4786fc9640 1 --2- 172.21.15.160:0/3889985590 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4780100570 0x7f4780100990 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:32.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.284+0000 7f4786fc9640 1 -- 172.21.15.160:0/3889985590 >> 172.21.15.160:0/3889985590 conn(0x7f47800fbb00 msgr2=0x7f47800fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:43:32.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.284+0000 7f4786fc9640 1 -- 172.21.15.160:0/3889985590 shutdown_connections 2024-03-07T07:43:32.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.284+0000 7f4786fc9640 1 -- 172.21.15.160:0/3889985590 wait complete. 2024-03-07T07:43:32.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.285+0000 7f4786fc9640 1 Processor -- start 2024-03-07T07:43:32.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.285+0000 7f4786fc9640 1 -- start start 2024-03-07T07:43:32.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.285+0000 7f4786fc9640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4780100570 0x7f4780197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:32.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.285+0000 7f4786fc9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4780101760 0x7f4780197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:32.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.285+0000 7f4786fc9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4780102120 0x7f478019efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:32.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.285+0000 7f4786fc9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478010c200 con 0x7f4780101760 2024-03-07T07:43:32.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.285+0000 7f4786fc9640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478010c080 con 0x7f4780100570 2024-03-07T07:43:32.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.285+0000 7f4786fc9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478010c380 con 0x7f4780102120 2024-03-07T07:43:32.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.285+0000 7f477ffff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4780101760 0x7f4780197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:32.288 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.285+0000 7f477ffff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4780101760 0x7f4780197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34230/0 (socket says 172.21.15.160:34230) 2024-03-07T07:43:32.288 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.285+0000 7f477ffff640 1 -- 172.21.15.160:0/2096591023 learned_addr learned my addr 172.21.15.160:0/2096591023 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:43:32.288 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.285+0000 7f4785566640 1 --2- 172.21.15.160:0/2096591023 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4780102120 0x7f478019efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:32.288 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.285+0000 7f4784d65640 1 --2- 172.21.15.160:0/2096591023 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4780100570 0x7f4780197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:32.288 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.285+0000 7f477ffff640 1 -- 172.21.15.160:0/2096591023 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4780102120 msgr2=0x7f478019efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:43:32.288 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.285+0000 7f477ffff640 1 --2- 172.21.15.160:0/2096591023 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4780102120 0x7f478019efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:32.288 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.285+0000 7f477ffff640 1 -- 172.21.15.160:0/2096591023 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4780100570 msgr2=0x7f4780197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:43:32.288 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.285+0000 7f477ffff640 1 --2- 172.21.15.160:0/2096591023 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4780100570 0x7f4780197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:32.288 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.285+0000 7f477ffff640 1 -- 172.21.15.160:0/2096591023 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f478019f640 con 0x7f4780101760 2024-03-07T07:43:32.288 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.285+0000 7f4784d65640 1 --2- 172.21.15.160:0/2096591023 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4780100570 0x7f4780197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:43:32.288 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.285+0000 7f477ffff640 1 --2- 172.21.15.160:0/2096591023 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4780101760 0x7f4780197fb0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f476c00e9f0 tx=0x7f476c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:43:32.288 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.285+0000 7f477dffb640 1 -- 172.21.15.160:0/2096591023 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476c00ccc0 con 0x7f4780101760 2024-03-07T07:43:32.288 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.286+0000 7f4786fc9640 1 -- 172.21.15.160:0/2096591023 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f478019f930 con 0x7f4780101760 2024-03-07T07:43:32.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.286+0000 7f477dffb640 1 -- 172.21.15.160:0/2096591023 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f476c00ce60 con 0x7f4780101760 2024-03-07T07:43:32.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.286+0000 7f477dffb640 1 -- 172.21.15.160:0/2096591023 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476c010690 con 0x7f4780101760 2024-03-07T07:43:32.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.286+0000 7f4786fc9640 1 -- 172.21.15.160:0/2096591023 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f478019fe80 con 0x7f4780101760 2024-03-07T07:43:32.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.287+0000 7f4786fc9640 1 -- 172.21.15.160:0/2096591023 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47801a0110 con 0x7f4780101760 2024-03-07T07:43:32.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.288+0000 7f477dffb640 1 -- 172.21.15.160:0/2096591023 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 20) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f476c0046c0 con 0x7f4780101760 2024-03-07T07:43:32.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.288+0000 7f477dffb640 1 --2- 172.21.15.160:0/2096591023 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f47700a1d80 0x7f47700a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:32.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.288+0000 7f477dffb640 1 -- 172.21.15.160:0/2096591023 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f476c0ce1a0 con 0x7f4780101760 2024-03-07T07:43:32.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.288+0000 7f4784d65640 1 --2- 172.21.15.160:0/2096591023 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f47700a1d80 0x7f47700a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:32.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.289+0000 7f4784d65640 1 --2- 172.21.15.160:0/2096591023 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f47700a1d80 0x7f47700a4240 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4774000c00 tx=0x7f4774005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:43:32.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.293+0000 7f477dffb640 1 -- 172.21.15.160:0/2096591023 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f476c09b5d0 con 0x7f4780101760 2024-03-07T07:43:32.467 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.467+0000 7f4786fc9640 1 -- 172.21.15.160:0/2096591023 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f47801a02e0 con 0x7f4780101760 2024-03-07T07:43:32.468 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.468+0000 7f477dffb640 1 -- 172.21.15.160:0/2096591023 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f476c09aed0 con 0x7f4780101760 2024-03-07T07:43:32.468 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 64 2024-03-07T07:43:32.470 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.470+0000 7f4786fc9640 1 -- 172.21.15.160:0/2096591023 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f47700a1d80 msgr2=0x7f47700a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:43:32.470 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.470+0000 7f4786fc9640 1 --2- 172.21.15.160:0/2096591023 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f47700a1d80 0x7f47700a4240 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4774000c00 tx=0x7f4774005fd0 comp rx=0 tx=0).stop 2024-03-07T07:43:32.470 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.470+0000 7f4786fc9640 1 -- 172.21.15.160:0/2096591023 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4780101760 msgr2=0x7f4780197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:43:32.470 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.470+0000 7f4786fc9640 1 --2- 172.21.15.160:0/2096591023 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4780101760 0x7f4780197fb0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f476c00e9f0 tx=0x7f476c00edf0 comp rx=0 tx=0).stop 2024-03-07T07:43:32.470 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.470+0000 7f4786fc9640 1 -- 172.21.15.160:0/2096591023 shutdown_connections 2024-03-07T07:43:32.470 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.470+0000 7f4786fc9640 1 --2- 172.21.15.160:0/2096591023 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f47700a1d80 0x7f47700a4240 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:32.470 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.470+0000 7f4786fc9640 1 --2- 172.21.15.160:0/2096591023 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4780102120 0x7f478019efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:32.470 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.470+0000 7f4786fc9640 1 --2- 172.21.15.160:0/2096591023 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4780101760 0x7f4780197fb0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:32.470 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.470+0000 7f4786fc9640 1 --2- 172.21.15.160:0/2096591023 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4780100570 0x7f4780197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:32.471 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.470+0000 7f4786fc9640 1 -- 172.21.15.160:0/2096591023 >> 172.21.15.160:0/2096591023 conn(0x7f47800fbb00 msgr2=0x7f47800fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:43:32.471 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.470+0000 7f4786fc9640 1 -- 172.21.15.160:0/2096591023 shutdown_connections 2024-03-07T07:43:32.471 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.470+0000 7f4786fc9640 1 -- 172.21.15.160:0/2096591023 wait complete. 2024-03-07T07:43:32.481 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-03-07T07:43:32.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.584+0000 7f3240467640 1 -- 172.21.15.160:0/474921493 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3238101680 msgr2=0x7f3238101aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:43:32.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.584+0000 7f3240467640 1 --2- 172.21.15.160:0/474921493 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3238101680 0x7f3238101aa0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f3230009900 tx=0x7f323002bc30 comp rx=0 tx=0).stop 2024-03-07T07:43:32.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.584+0000 7f3240467640 1 -- 172.21.15.160:0/474921493 shutdown_connections 2024-03-07T07:43:32.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.584+0000 7f3240467640 1 --2- 172.21.15.160:0/474921493 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3238103230 0x7f3238109ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:32.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.584+0000 7f3240467640 1 --2- 172.21.15.160:0/474921493 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3238102870 0x7f3238102cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:32.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.584+0000 7f3240467640 1 --2- 172.21.15.160:0/474921493 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3238101680 0x7f3238101aa0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:32.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.584+0000 7f3240467640 1 -- 172.21.15.160:0/474921493 >> 172.21.15.160:0/474921493 conn(0x7f32380fcc10 msgr2=0x7f32380ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:43:32.587 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.584+0000 7f3240467640 1 -- 172.21.15.160:0/474921493 shutdown_connections 2024-03-07T07:43:32.587 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.584+0000 7f3240467640 1 -- 172.21.15.160:0/474921493 wait complete. 2024-03-07T07:43:32.587 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.584+0000 7f3240467640 1 Processor -- start 2024-03-07T07:43:32.587 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.584+0000 7f3240467640 1 -- start start 2024-03-07T07:43:32.587 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.585+0000 7f3240467640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3238101680 0x7f32381979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:32.587 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.585+0000 7f3240467640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3238102870 0x7f3238197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:32.587 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.585+0000 7f3240467640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3238103230 0x7f323819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:32.587 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.585+0000 7f3240467640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323810d340 con 0x7f3238102870 2024-03-07T07:43:32.587 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.585+0000 7f3240467640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323810d1c0 con 0x7f3238103230 2024-03-07T07:43:32.587 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.585+0000 7f3240467640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323810d4c0 con 0x7f3238101680 2024-03-07T07:43:32.587 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.585+0000 7f323da02640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3238102870 0x7f3238197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:32.587 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.585+0000 7f323da02640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3238102870 0x7f3238197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34250/0 (socket says 172.21.15.160:34250) 2024-03-07T07:43:32.587 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.585+0000 7f323da02640 1 -- 172.21.15.160:0/4252030292 learned_addr learned my addr 172.21.15.160:0/4252030292 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:43:32.588 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.585+0000 7f323e203640 1 --2- 172.21.15.160:0/4252030292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3238101680 0x7f32381979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:32.588 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.585+0000 7f323ea04640 1 --2- 172.21.15.160:0/4252030292 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3238103230 0x7f323819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:32.588 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.585+0000 7f323da02640 1 -- 172.21.15.160:0/4252030292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3238101680 msgr2=0x7f32381979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:43:32.588 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.585+0000 7f323da02640 1 --2- 172.21.15.160:0/4252030292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3238101680 0x7f32381979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:32.588 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.585+0000 7f323da02640 1 -- 172.21.15.160:0/4252030292 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3238103230 msgr2=0x7f323819ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:43:32.588 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.585+0000 7f323da02640 1 --2- 172.21.15.160:0/4252030292 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3238103230 0x7f323819ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:32.588 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.585+0000 7f323da02640 1 -- 172.21.15.160:0/4252030292 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f323819f5d0 con 0x7f3238102870 2024-03-07T07:43:32.588 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.585+0000 7f323ea04640 1 --2- 172.21.15.160:0/4252030292 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3238103230 0x7f323819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:43:32.588 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.585+0000 7f323e203640 1 --2- 172.21.15.160:0/4252030292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3238101680 0x7f32381979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:43:32.588 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.585+0000 7f323da02640 1 --2- 172.21.15.160:0/4252030292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3238102870 0x7f3238197ef0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f322c00eb90 tx=0x7f322c00ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:43:32.588 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.585+0000 7f32377fe640 1 -- 172.21.15.160:0/4252030292 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322c004b20 con 0x7f3238102870 2024-03-07T07:43:32.588 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.585+0000 7f32377fe640 1 -- 172.21.15.160:0/4252030292 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f322c004cc0 con 0x7f3238102870 2024-03-07T07:43:32.588 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.585+0000 7f3240467640 1 -- 172.21.15.160:0/4252030292 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f323819f8c0 con 0x7f3238102870 2024-03-07T07:43:32.589 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.585+0000 7f32377fe640 1 -- 172.21.15.160:0/4252030292 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322c010720 con 0x7f3238102870 2024-03-07T07:43:32.589 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.585+0000 7f3240467640 1 -- 172.21.15.160:0/4252030292 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f323819fdc0 con 0x7f3238102870 2024-03-07T07:43:32.589 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.586+0000 7f3240467640 1 -- 172.21.15.160:0/4252030292 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3238108a20 con 0x7f3238102870 2024-03-07T07:43:32.589 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.587+0000 7f32377fe640 1 -- 172.21.15.160:0/4252030292 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 20) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f322c005020 con 0x7f3238102870 2024-03-07T07:43:32.590 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.588+0000 7f32377fe640 1 --2- 172.21.15.160:0/4252030292 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f32200a1d80 0x7f32200a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:43:32.590 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.588+0000 7f32377fe640 1 -- 172.21.15.160:0/4252030292 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f322c0ce3b0 con 0x7f3238102870 2024-03-07T07:43:32.590 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.588+0000 7f323e203640 1 --2- 172.21.15.160:0/4252030292 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f32200a1d80 0x7f32200a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:43:32.590 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.588+0000 7f323e203640 1 --2- 172.21.15.160:0/4252030292 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f32200a1d80 0x7f32200a4240 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f323002c170 tx=0x7f323002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:43:32.592 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.592+0000 7f32377fe640 1 -- 172.21.15.160:0/4252030292 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f322c09b7e0 con 0x7f3238102870 2024-03-07T07:43:32.814 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.814+0000 7f3240467640 1 -- 172.21.15.160:0/4252030292 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f3238062260 con 0x7f3238102870 2024-03-07T07:43:32.822 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.822+0000 7f32377fe640 1 -- 172.21.15.160:0/4252030292 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) v1 ==== 81+0+736399 (secure 0 0 0) 0x7f322c09b0e0 con 0x7f3238102870 2024-03-07T07:43:32.831 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:43:32.831 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-03-07T07:43:32.831 INFO:teuthology.orchestra.run.smithi160.stdout: "epoch": 20, 2024-03-07T07:43:32.832 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.832 INFO:teuthology.orchestra.run.smithi160.stdout: "active_gid": 14152, 2024-03-07T07:43:32.832 INFO:teuthology.orchestra.run.smithi160.stdout: "active_name": "x", 2024-03-07T07:43:32.832 INFO:teuthology.orchestra.run.smithi160.stdout: "active_addrs": { 2024-03-07T07:43:32.832 INFO:teuthology.orchestra.run.smithi160.stdout: "addrvec": [ 2024-03-07T07:43:32.832 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:32.832 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "v2", 2024-03-07T07:43:32.832 INFO:teuthology.orchestra.run.smithi160.stdout: "addr": "172.21.15.160:6800", 2024-03-07T07:43:32.832 INFO:teuthology.orchestra.run.smithi160.stdout: "nonce": 419630619 2024-03-07T07:43:32.832 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.832 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:32.832 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "v1", 2024-03-07T07:43:32.832 INFO:teuthology.orchestra.run.smithi160.stdout: "addr": "172.21.15.160:6801", 2024-03-07T07:43:32.833 INFO:teuthology.orchestra.run.smithi160.stdout: "nonce": 419630619 2024-03-07T07:43:32.833 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:32.833 INFO:teuthology.orchestra.run.smithi160.stdout: ] 2024-03-07T07:43:32.833 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.833 INFO:teuthology.orchestra.run.smithi160.stdout: "active_addr": "172.21.15.160:6801/419630619", 2024-03-07T07:43:32.833 INFO:teuthology.orchestra.run.smithi160.stdout: "active_change": "2024-03-07T07:32:08.190949+0000", 2024-03-07T07:43:32.833 INFO:teuthology.orchestra.run.smithi160.stdout: "active_mgr_features": 4540701547738038271, 2024-03-07T07:43:32.833 INFO:teuthology.orchestra.run.smithi160.stdout: "available": true, 2024-03-07T07:43:32.833 INFO:teuthology.orchestra.run.smithi160.stdout: "standbys": [ 2024-03-07T07:43:32.833 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:32.833 INFO:teuthology.orchestra.run.smithi160.stdout: "gid": 24109, 2024-03-07T07:43:32.833 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "z", 2024-03-07T07:43:32.833 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr_features": 4540701547738038271, 2024-03-07T07:43:32.834 INFO:teuthology.orchestra.run.smithi160.stdout: "available_modules": [ 2024-03-07T07:43:32.834 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:32.834 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "alerts", 2024-03-07T07:43:32.834 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:32.834 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:32.834 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:32.834 INFO:teuthology.orchestra.run.smithi160.stdout: "interval": { 2024-03-07T07:43:32.834 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "interval", 2024-03-07T07:43:32.834 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:32.834 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.834 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:32.834 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "60", 2024-03-07T07:43:32.834 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.834 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.834 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.835 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "How frequently to reexamine health status", 2024-03-07T07:43:32.835 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.835 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.835 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.835 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.835 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:32.835 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:32.835 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.835 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.835 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:32.835 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:32.835 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.835 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.835 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:32.835 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:32.836 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:32.836 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:32.836 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:32.836 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:32.836 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:32.836 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:32.836 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.836 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.836 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.836 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.836 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.836 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:32.837 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:32.837 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.837 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.837 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:32.837 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:32.837 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.837 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.837 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.837 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.837 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.837 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.837 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.837 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.837 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:32.837 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:32.838 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.838 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.838 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:32.838 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:32.838 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.838 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.838 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:32.838 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:32.838 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:32.838 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:32.838 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:32.838 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:32.838 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:32.838 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:32.839 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.839 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.839 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.839 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.839 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.839 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:32.839 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:32.839 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.839 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.839 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:32.839 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:32.839 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.839 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.839 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.839 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.840 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.840 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.840 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.840 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.840 INFO:teuthology.orchestra.run.smithi160.stdout: "smtp_destination": { 2024-03-07T07:43:32.840 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "smtp_destination", 2024-03-07T07:43:32.840 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.840 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.840 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:32.840 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:32.840 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.840 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.841 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.841 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Email address to send alerts to", 2024-03-07T07:43:32.841 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.841 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.841 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.841 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.841 INFO:teuthology.orchestra.run.smithi160.stdout: "smtp_from_name": { 2024-03-07T07:43:32.841 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "smtp_from_name", 2024-03-07T07:43:32.841 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.841 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.841 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:32.841 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "Ceph", 2024-03-07T07:43:32.841 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.841 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.841 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.842 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Email From: name", 2024-03-07T07:43:32.842 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.842 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.842 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.842 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.842 INFO:teuthology.orchestra.run.smithi160.stdout: "smtp_host": { 2024-03-07T07:43:32.842 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "smtp_host", 2024-03-07T07:43:32.842 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.842 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.842 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:32.842 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:32.842 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.842 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.842 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.842 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "SMTP server", 2024-03-07T07:43:32.843 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.843 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.843 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.843 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.843 INFO:teuthology.orchestra.run.smithi160.stdout: "smtp_password": { 2024-03-07T07:43:32.843 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "smtp_password", 2024-03-07T07:43:32.843 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.843 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.843 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:32.843 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:32.843 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.843 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.843 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.843 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Password to authenticate with", 2024-03-07T07:43:32.844 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.844 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.844 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.844 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.844 INFO:teuthology.orchestra.run.smithi160.stdout: "smtp_port": { 2024-03-07T07:43:32.844 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "smtp_port", 2024-03-07T07:43:32.844 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:32.844 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.844 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:32.844 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "465", 2024-03-07T07:43:32.844 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.844 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.844 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.845 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "SMTP port", 2024-03-07T07:43:32.845 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.845 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.845 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.845 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.845 INFO:teuthology.orchestra.run.smithi160.stdout: "smtp_sender": { 2024-03-07T07:43:32.845 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "smtp_sender", 2024-03-07T07:43:32.845 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.845 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.845 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:32.845 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:32.845 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.845 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.845 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.846 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "SMTP envelope sender", 2024-03-07T07:43:32.846 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.846 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.846 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.846 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.846 INFO:teuthology.orchestra.run.smithi160.stdout: "smtp_ssl": { 2024-03-07T07:43:32.846 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "smtp_ssl", 2024-03-07T07:43:32.846 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.846 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.846 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:32.846 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:32.846 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.846 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.847 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.847 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-07T07:43:32.847 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.847 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.847 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.847 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.847 INFO:teuthology.orchestra.run.smithi160.stdout: "smtp_user": { 2024-03-07T07:43:32.847 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "smtp_user", 2024-03-07T07:43:32.847 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.847 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.847 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:32.847 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:32.847 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.848 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.848 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.848 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "User to authenticate as", 2024-03-07T07:43:32.848 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.848 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.848 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.848 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:32.848 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:32.848 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.848 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:32.848 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "balancer", 2024-03-07T07:43:32.848 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:32.848 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:32.849 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:32.849 INFO:teuthology.orchestra.run.smithi160.stdout: "active": { 2024-03-07T07:43:32.849 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "active", 2024-03-07T07:43:32.849 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.849 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.849 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:32.849 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:32.849 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.849 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.849 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.849 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "automatically balance PGs across cluster", 2024-03-07T07:43:32.849 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.849 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.849 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.850 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.850 INFO:teuthology.orchestra.run.smithi160.stdout: "begin_time": { 2024-03-07T07:43:32.850 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "begin_time", 2024-03-07T07:43:32.850 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.850 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.850 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:32.850 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "0000", 2024-03-07T07:43:32.850 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.850 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.850 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.850 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "beginning time of day to automatically balance", 2024-03-07T07:43:32.850 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-07T07:43:32.850 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.851 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.851 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.851 INFO:teuthology.orchestra.run.smithi160.stdout: "begin_weekday": { 2024-03-07T07:43:32.851 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "begin_weekday", 2024-03-07T07:43:32.851 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "uint", 2024-03-07T07:43:32.851 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.851 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:32.851 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "0", 2024-03-07T07:43:32.851 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "0", 2024-03-07T07:43:32.851 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "6", 2024-03-07T07:43:32.851 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.851 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-07T07:43:32.851 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-07T07:43:32.852 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.852 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.852 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.852 INFO:teuthology.orchestra.run.smithi160.stdout: "crush_compat_max_iterations": { 2024-03-07T07:43:32.852 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "crush_compat_max_iterations", 2024-03-07T07:43:32.852 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "uint", 2024-03-07T07:43:32.852 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.852 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:32.852 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "25", 2024-03-07T07:43:32.852 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "1", 2024-03-07T07:43:32.852 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "250", 2024-03-07T07:43:32.852 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.852 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-07T07:43:32.852 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.853 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.853 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.853 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.853 INFO:teuthology.orchestra.run.smithi160.stdout: "crush_compat_metrics": { 2024-03-07T07:43:32.853 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "crush_compat_metrics", 2024-03-07T07:43:32.853 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.853 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.853 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:32.853 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "pgs,objects,bytes", 2024-03-07T07:43:32.853 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.853 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.853 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.853 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-07T07:43:32.854 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-07T07:43:32.854 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.854 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.854 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.854 INFO:teuthology.orchestra.run.smithi160.stdout: "crush_compat_step": { 2024-03-07T07:43:32.854 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "crush_compat_step", 2024-03-07T07:43:32.854 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "float", 2024-03-07T07:43:32.854 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.854 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:32.854 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "0.5", 2024-03-07T07:43:32.854 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "0.001", 2024-03-07T07:43:32.854 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "0.999", 2024-03-07T07:43:32.854 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.855 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "aggressiveness of optimization", 2024-03-07T07:43:32.855 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-07T07:43:32.855 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.855 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.855 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.855 INFO:teuthology.orchestra.run.smithi160.stdout: "end_time": { 2024-03-07T07:43:32.855 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "end_time", 2024-03-07T07:43:32.855 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.855 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.855 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:32.855 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "2359", 2024-03-07T07:43:32.855 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.855 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.855 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.855 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "ending time of day to automatically balance", 2024-03-07T07:43:32.856 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-07T07:43:32.856 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.856 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.856 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.856 INFO:teuthology.orchestra.run.smithi160.stdout: "end_weekday": { 2024-03-07T07:43:32.856 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "end_weekday", 2024-03-07T07:43:32.856 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "uint", 2024-03-07T07:43:32.856 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.856 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:32.856 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "0", 2024-03-07T07:43:32.856 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "0", 2024-03-07T07:43:32.856 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "6", 2024-03-07T07:43:32.856 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.857 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-07T07:43:32.857 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-07T07:43:32.857 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.857 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.857 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.857 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:32.857 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:32.857 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.857 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.857 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:32.857 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:32.857 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.857 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.857 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:32.857 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:32.858 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:32.858 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:32.858 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:32.858 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:32.858 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:32.858 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:32.858 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.858 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.858 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.858 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.858 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.858 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:32.858 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:32.859 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.859 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.859 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:32.859 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:32.859 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.859 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.859 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.859 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.859 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.859 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.859 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.859 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.860 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:32.860 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:32.860 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.860 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.860 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:32.860 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:32.860 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.860 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.860 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:32.860 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:32.860 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:32.860 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:32.860 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:32.860 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:32.861 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:32.861 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:32.861 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.861 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.861 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.861 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.861 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.861 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:32.861 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:32.861 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.861 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.861 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:32.861 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:32.861 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.862 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.862 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.862 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.862 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.862 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.862 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.862 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.862 INFO:teuthology.orchestra.run.smithi160.stdout: "min_score": { 2024-03-07T07:43:32.862 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "min_score", 2024-03-07T07:43:32.862 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "float", 2024-03-07T07:43:32.862 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.862 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:32.862 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "0", 2024-03-07T07:43:32.863 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.863 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.863 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.863 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-07T07:43:32.863 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.863 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.863 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.863 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.863 INFO:teuthology.orchestra.run.smithi160.stdout: "mode": { 2024-03-07T07:43:32.863 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "mode", 2024-03-07T07:43:32.863 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.863 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.863 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:32.864 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "upmap", 2024-03-07T07:43:32.864 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.864 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.864 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:32.864 INFO:teuthology.orchestra.run.smithi160.stdout: "crush-compat", 2024-03-07T07:43:32.864 INFO:teuthology.orchestra.run.smithi160.stdout: "none", 2024-03-07T07:43:32.864 INFO:teuthology.orchestra.run.smithi160.stdout: "read", 2024-03-07T07:43:32.864 INFO:teuthology.orchestra.run.smithi160.stdout: "upmap", 2024-03-07T07:43:32.864 INFO:teuthology.orchestra.run.smithi160.stdout: "upmap-read" 2024-03-07T07:43:32.864 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:32.864 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Balancer mode", 2024-03-07T07:43:32.864 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.864 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.865 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.865 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.865 INFO:teuthology.orchestra.run.smithi160.stdout: "pool_ids": { 2024-03-07T07:43:32.865 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "pool_ids", 2024-03-07T07:43:32.865 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.865 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.865 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:32.865 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:32.865 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.865 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.865 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.865 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-07T07:43:32.865 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.865 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.866 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.866 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.866 INFO:teuthology.orchestra.run.smithi160.stdout: "sleep_interval": { 2024-03-07T07:43:32.866 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "sleep_interval", 2024-03-07T07:43:32.866 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:32.866 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.866 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:32.866 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "60", 2024-03-07T07:43:32.866 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.866 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.866 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.866 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-07T07:43:32.866 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.866 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.867 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.867 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.867 INFO:teuthology.orchestra.run.smithi160.stdout: "upmap_max_deviation": { 2024-03-07T07:43:32.867 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "upmap_max_deviation", 2024-03-07T07:43:32.867 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:32.867 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.867 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:32.867 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "5", 2024-03-07T07:43:32.867 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "1", 2024-03-07T07:43:32.867 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.867 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.867 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-07T07:43:32.867 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-07T07:43:32.868 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.868 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.868 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.868 INFO:teuthology.orchestra.run.smithi160.stdout: "upmap_max_optimizations": { 2024-03-07T07:43:32.868 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "upmap_max_optimizations", 2024-03-07T07:43:32.868 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "uint", 2024-03-07T07:43:32.868 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.868 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:32.868 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "10", 2024-03-07T07:43:32.868 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.868 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.868 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.868 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-07T07:43:32.868 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.869 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.869 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.869 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:32.869 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:32.869 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.869 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:32.869 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "cephadm", 2024-03-07T07:43:32.869 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:32.869 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:32.869 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:32.869 INFO:teuthology.orchestra.run.smithi160.stdout: "agent_down_multiplier": { 2024-03-07T07:43:32.869 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "agent_down_multiplier", 2024-03-07T07:43:32.869 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "float", 2024-03-07T07:43:32.869 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.870 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.870 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "3.0", 2024-03-07T07:43:32.870 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.870 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.870 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.870 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-07T07:43:32.870 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.870 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.870 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.870 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.870 INFO:teuthology.orchestra.run.smithi160.stdout: "agent_refresh_rate": { 2024-03-07T07:43:32.870 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "agent_refresh_rate", 2024-03-07T07:43:32.870 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:32.871 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.871 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.871 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "20", 2024-03-07T07:43:32.871 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.871 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.871 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.871 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-07T07:43:32.871 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.871 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.871 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.871 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.871 INFO:teuthology.orchestra.run.smithi160.stdout: "agent_starting_port": { 2024-03-07T07:43:32.871 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "agent_starting_port", 2024-03-07T07:43:32.871 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:32.872 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.872 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.872 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "4721", 2024-03-07T07:43:32.872 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.872 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.872 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.872 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-07T07:43:32.872 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.872 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.872 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.872 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.872 INFO:teuthology.orchestra.run.smithi160.stdout: "allow_ptrace": { 2024-03-07T07:43:32.873 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "allow_ptrace", 2024-03-07T07:43:32.873 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.873 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.873 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.873 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:32.873 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.873 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.873 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.873 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-07T07:43:32.873 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-07T07:43:32.873 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.873 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.873 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.873 INFO:teuthology.orchestra.run.smithi160.stdout: "autotune_interval": { 2024-03-07T07:43:32.874 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "autotune_interval", 2024-03-07T07:43:32.874 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:32.874 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.874 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.874 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "600", 2024-03-07T07:43:32.874 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.874 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.874 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.874 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-07T07:43:32.874 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.874 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.874 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.874 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.874 INFO:teuthology.orchestra.run.smithi160.stdout: "autotune_memory_target_ratio": { 2024-03-07T07:43:32.875 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "autotune_memory_target_ratio", 2024-03-07T07:43:32.875 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "float", 2024-03-07T07:43:32.875 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.875 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.875 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "0.7", 2024-03-07T07:43:32.875 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.875 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.875 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.875 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-07T07:43:32.875 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.875 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.875 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.875 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.875 INFO:teuthology.orchestra.run.smithi160.stdout: "cephadm_log_destination": { 2024-03-07T07:43:32.876 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "cephadm_log_destination", 2024-03-07T07:43:32.876 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.876 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.876 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.876 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:32.876 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.876 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.876 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:32.876 INFO:teuthology.orchestra.run.smithi160.stdout: "file", 2024-03-07T07:43:32.876 INFO:teuthology.orchestra.run.smithi160.stdout: "file,syslog", 2024-03-07T07:43:32.876 INFO:teuthology.orchestra.run.smithi160.stdout: "syslog" 2024-03-07T07:43:32.876 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:32.876 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-07T07:43:32.877 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.877 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.877 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.877 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.877 INFO:teuthology.orchestra.run.smithi160.stdout: "cgroups_split": { 2024-03-07T07:43:32.877 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "cgroups_split", 2024-03-07T07:43:32.877 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.877 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.877 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.877 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:32.877 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.877 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.877 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.878 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-07T07:43:32.878 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.878 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.878 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.878 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.878 INFO:teuthology.orchestra.run.smithi160.stdout: "config_checks_enabled": { 2024-03-07T07:43:32.878 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "config_checks_enabled", 2024-03-07T07:43:32.878 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.878 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.878 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.878 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:32.878 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.878 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.879 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.879 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-07T07:43:32.879 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.879 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.879 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.879 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.879 INFO:teuthology.orchestra.run.smithi160.stdout: "config_dashboard": { 2024-03-07T07:43:32.879 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "config_dashboard", 2024-03-07T07:43:32.879 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.879 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.879 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.879 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:32.879 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.880 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.880 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.880 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-07T07:43:32.880 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.880 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.880 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.880 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.880 INFO:teuthology.orchestra.run.smithi160.stdout: "container_image_alertmanager": { 2024-03-07T07:43:32.880 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "container_image_alertmanager", 2024-03-07T07:43:32.880 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.880 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.880 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.880 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-07T07:43:32.881 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.881 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.881 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.881 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Prometheus container image", 2024-03-07T07:43:32.881 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.881 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.881 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.881 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.881 INFO:teuthology.orchestra.run.smithi160.stdout: "container_image_base": { 2024-03-07T07:43:32.881 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "container_image_base", 2024-03-07T07:43:32.881 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.881 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.881 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:32.881 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-07T07:43:32.881 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.881 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.882 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.882 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Container image name, without the tag", 2024-03-07T07:43:32.882 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.882 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.882 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.882 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.882 INFO:teuthology.orchestra.run.smithi160.stdout: "container_image_elasticsearch": { 2024-03-07T07:43:32.882 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "container_image_elasticsearch", 2024-03-07T07:43:32.882 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.882 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.882 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.882 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-07T07:43:32.882 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.882 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.882 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.883 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "elasticsearch container image", 2024-03-07T07:43:32.883 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.883 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.883 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.883 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.883 INFO:teuthology.orchestra.run.smithi160.stdout: "container_image_grafana": { 2024-03-07T07:43:32.883 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "container_image_grafana", 2024-03-07T07:43:32.883 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.883 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.883 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.883 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-07T07:43:32.883 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.883 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.883 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.884 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Prometheus container image", 2024-03-07T07:43:32.884 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.884 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.884 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.884 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.884 INFO:teuthology.orchestra.run.smithi160.stdout: "container_image_haproxy": { 2024-03-07T07:43:32.884 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "container_image_haproxy", 2024-03-07T07:43:32.884 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.884 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.884 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.884 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-07T07:43:32.884 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.885 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.885 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.885 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "HAproxy container image", 2024-03-07T07:43:32.885 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.885 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.885 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.885 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.885 INFO:teuthology.orchestra.run.smithi160.stdout: "container_image_jaeger_agent": { 2024-03-07T07:43:32.885 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "container_image_jaeger_agent", 2024-03-07T07:43:32.885 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.885 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.885 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.885 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-07T07:43:32.885 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.886 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.886 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.886 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Jaeger agent container image", 2024-03-07T07:43:32.886 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.886 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.886 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.886 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.886 INFO:teuthology.orchestra.run.smithi160.stdout: "container_image_jaeger_collector": { 2024-03-07T07:43:32.886 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "container_image_jaeger_collector", 2024-03-07T07:43:32.886 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.886 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.886 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.886 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-07T07:43:32.886 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.887 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.887 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.887 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Jaeger collector container image", 2024-03-07T07:43:32.887 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.887 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.887 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.887 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.887 INFO:teuthology.orchestra.run.smithi160.stdout: "container_image_jaeger_query": { 2024-03-07T07:43:32.887 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "container_image_jaeger_query", 2024-03-07T07:43:32.887 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.887 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.887 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.887 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-07T07:43:32.887 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.888 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.888 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.888 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Jaeger query container image", 2024-03-07T07:43:32.888 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.888 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.888 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.888 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.888 INFO:teuthology.orchestra.run.smithi160.stdout: "container_image_keepalived": { 2024-03-07T07:43:32.888 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "container_image_keepalived", 2024-03-07T07:43:32.888 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.888 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.888 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.888 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-07T07:43:32.888 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.889 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.889 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.889 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Keepalived container image", 2024-03-07T07:43:32.889 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.889 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.889 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.889 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.889 INFO:teuthology.orchestra.run.smithi160.stdout: "container_image_loki": { 2024-03-07T07:43:32.889 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "container_image_loki", 2024-03-07T07:43:32.889 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.889 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.889 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.889 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-07T07:43:32.889 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.889 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.890 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.890 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Loki container image", 2024-03-07T07:43:32.890 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.890 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.890 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.890 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.890 INFO:teuthology.orchestra.run.smithi160.stdout: "container_image_node_exporter": { 2024-03-07T07:43:32.890 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "container_image_node_exporter", 2024-03-07T07:43:32.890 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.890 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.890 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.890 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-07T07:43:32.890 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.890 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.890 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.891 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Prometheus container image", 2024-03-07T07:43:32.891 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.891 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.891 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.891 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.891 INFO:teuthology.orchestra.run.smithi160.stdout: "container_image_nvmeof": { 2024-03-07T07:43:32.891 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "container_image_nvmeof", 2024-03-07T07:43:32.891 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.891 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.891 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.891 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-07T07:43:32.891 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.891 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.891 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.891 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Nvme-of container image", 2024-03-07T07:43:32.892 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.892 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.892 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.892 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.892 INFO:teuthology.orchestra.run.smithi160.stdout: "container_image_prometheus": { 2024-03-07T07:43:32.892 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "container_image_prometheus", 2024-03-07T07:43:32.892 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.892 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.892 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.892 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-07T07:43:32.892 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.892 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.892 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.892 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Prometheus container image", 2024-03-07T07:43:32.893 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.893 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.893 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.893 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.893 INFO:teuthology.orchestra.run.smithi160.stdout: "container_image_promtail": { 2024-03-07T07:43:32.893 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "container_image_promtail", 2024-03-07T07:43:32.893 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.893 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.893 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.893 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-07T07:43:32.893 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.893 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.893 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.893 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Promtail container image", 2024-03-07T07:43:32.894 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.894 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.894 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.894 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.894 INFO:teuthology.orchestra.run.smithi160.stdout: "container_image_snmp_gateway": { 2024-03-07T07:43:32.894 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "container_image_snmp_gateway", 2024-03-07T07:43:32.894 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.894 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.894 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.894 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-07T07:43:32.894 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.894 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.894 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.895 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "SNMP Gateway container image", 2024-03-07T07:43:32.895 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.895 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.895 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.895 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.895 INFO:teuthology.orchestra.run.smithi160.stdout: "container_init": { 2024-03-07T07:43:32.895 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "container_init", 2024-03-07T07:43:32.895 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.895 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.895 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.895 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:32.895 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.895 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.895 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.896 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Run podman/docker with `--init`", 2024-03-07T07:43:32.896 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.896 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.896 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.896 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.896 INFO:teuthology.orchestra.run.smithi160.stdout: "daemon_cache_timeout": { 2024-03-07T07:43:32.896 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "daemon_cache_timeout", 2024-03-07T07:43:32.896 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:32.896 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.896 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.896 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "600", 2024-03-07T07:43:32.896 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.896 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.897 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.897 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-07T07:43:32.897 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.897 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.897 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.897 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.897 INFO:teuthology.orchestra.run.smithi160.stdout: "default_cephadm_command_timeout": { 2024-03-07T07:43:32.897 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "default_cephadm_command_timeout", 2024-03-07T07:43:32.897 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:32.897 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.897 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.897 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "900", 2024-03-07T07:43:32.897 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.897 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.898 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.898 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-07T07:43:32.898 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.898 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.898 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.898 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.898 INFO:teuthology.orchestra.run.smithi160.stdout: "default_registry": { 2024-03-07T07:43:32.898 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "default_registry", 2024-03-07T07:43:32.898 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.898 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.898 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.898 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "docker.io", 2024-03-07T07:43:32.898 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.899 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.899 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.899 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-07T07:43:32.899 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.899 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.899 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.899 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.899 INFO:teuthology.orchestra.run.smithi160.stdout: "device_cache_timeout": { 2024-03-07T07:43:32.899 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "device_cache_timeout", 2024-03-07T07:43:32.899 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:32.899 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.899 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.899 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "1800", 2024-03-07T07:43:32.899 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.900 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.900 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.900 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "seconds to cache device inventory", 2024-03-07T07:43:32.900 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.900 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.900 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.900 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.900 INFO:teuthology.orchestra.run.smithi160.stdout: "device_enhanced_scan": { 2024-03-07T07:43:32.900 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "device_enhanced_scan", 2024-03-07T07:43:32.900 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.900 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.900 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.900 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:32.900 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.901 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.901 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.901 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-07T07:43:32.901 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.901 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.901 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.901 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.901 INFO:teuthology.orchestra.run.smithi160.stdout: "facts_cache_timeout": { 2024-03-07T07:43:32.901 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "facts_cache_timeout", 2024-03-07T07:43:32.901 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:32.901 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.901 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.901 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "60", 2024-03-07T07:43:32.901 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.901 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.902 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.902 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "seconds to cache host facts data", 2024-03-07T07:43:32.902 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.902 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.902 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.902 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.902 INFO:teuthology.orchestra.run.smithi160.stdout: "host_check_interval": { 2024-03-07T07:43:32.902 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "host_check_interval", 2024-03-07T07:43:32.902 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:32.902 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.902 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.902 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "600", 2024-03-07T07:43:32.902 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.902 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.902 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.903 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "how frequently to perform a host check", 2024-03-07T07:43:32.903 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.903 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.903 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.903 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.903 INFO:teuthology.orchestra.run.smithi160.stdout: "hw_monitoring": { 2024-03-07T07:43:32.903 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "hw_monitoring", 2024-03-07T07:43:32.903 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.903 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.903 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.903 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:32.903 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.903 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.903 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.903 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-07T07:43:32.903 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.904 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.904 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.904 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.904 INFO:teuthology.orchestra.run.smithi160.stdout: "inventory_list_all": { 2024-03-07T07:43:32.904 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "inventory_list_all", 2024-03-07T07:43:32.904 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.904 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.904 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.904 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:32.904 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.904 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.904 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.904 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-07T07:43:32.904 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.905 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.905 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.905 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.905 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:32.905 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:32.905 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.905 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.905 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:32.905 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:32.905 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.905 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.905 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:32.905 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:32.905 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:32.905 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:32.905 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:32.906 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:32.906 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:32.906 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:32.906 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.906 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.906 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.906 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.906 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.906 INFO:teuthology.orchestra.run.smithi160.stdout: "log_refresh_metadata": { 2024-03-07T07:43:32.906 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_refresh_metadata", 2024-03-07T07:43:32.906 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.906 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.906 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.906 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:32.906 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.907 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.907 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.907 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-07T07:43:32.907 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.907 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.907 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.907 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.907 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:32.907 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:32.907 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.907 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.907 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.907 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:32.907 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.908 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.908 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.908 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-07T07:43:32.908 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.908 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.908 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.908 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.908 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:32.908 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:32.908 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.908 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.908 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:32.908 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:32.909 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.909 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.909 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:32.909 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:32.909 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:32.909 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:32.909 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:32.909 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:32.909 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:32.909 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:32.909 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.909 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.909 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.909 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.909 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.910 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:32.910 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:32.910 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.910 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.910 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:32.910 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:32.910 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.910 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.910 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.910 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.910 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.910 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.910 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.910 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.911 INFO:teuthology.orchestra.run.smithi160.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-07T07:43:32.911 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-07T07:43:32.911 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.911 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.911 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.911 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:32.911 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.911 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.911 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.911 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-07T07:43:32.911 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.911 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.911 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.911 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.912 INFO:teuthology.orchestra.run.smithi160.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-07T07:43:32.912 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-07T07:43:32.912 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.912 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.912 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.912 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "*", 2024-03-07T07:43:32.912 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.912 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.912 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.912 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-07T07:43:32.912 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.912 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.912 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.912 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.913 INFO:teuthology.orchestra.run.smithi160.stdout: "max_count_per_host": { 2024-03-07T07:43:32.913 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "max_count_per_host", 2024-03-07T07:43:32.913 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:32.913 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.913 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.913 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "10", 2024-03-07T07:43:32.913 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.913 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.913 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.913 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "max number of daemons per service per host", 2024-03-07T07:43:32.913 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.913 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.913 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.913 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.913 INFO:teuthology.orchestra.run.smithi160.stdout: "max_osd_draining_count": { 2024-03-07T07:43:32.914 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "max_osd_draining_count", 2024-03-07T07:43:32.914 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:32.914 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.914 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.914 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "10", 2024-03-07T07:43:32.914 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.914 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.914 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.914 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-07T07:43:32.914 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.914 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.914 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.914 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.914 INFO:teuthology.orchestra.run.smithi160.stdout: "migration_current": { 2024-03-07T07:43:32.914 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "migration_current", 2024-03-07T07:43:32.915 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:32.915 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.915 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.915 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:32.915 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.915 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.915 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.915 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "internal - do not modify", 2024-03-07T07:43:32.915 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.915 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.915 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.915 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.915 INFO:teuthology.orchestra.run.smithi160.stdout: "mode": { 2024-03-07T07:43:32.915 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "mode", 2024-03-07T07:43:32.916 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.916 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.916 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.916 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "root", 2024-03-07T07:43:32.916 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.916 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.916 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:32.916 INFO:teuthology.orchestra.run.smithi160.stdout: "cephadm-package", 2024-03-07T07:43:32.916 INFO:teuthology.orchestra.run.smithi160.stdout: "root" 2024-03-07T07:43:32.916 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:32.916 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "mode for remote execution of cephadm", 2024-03-07T07:43:32.916 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.916 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.916 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.917 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.917 INFO:teuthology.orchestra.run.smithi160.stdout: "oob_default_addr": { 2024-03-07T07:43:32.917 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "oob_default_addr", 2024-03-07T07:43:32.917 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.917 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.917 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.917 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "169.254.1.1", 2024-03-07T07:43:32.917 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.917 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.917 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.917 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-07T07:43:32.917 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.917 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.917 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.917 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.918 INFO:teuthology.orchestra.run.smithi160.stdout: "prometheus_alerts_path": { 2024-03-07T07:43:32.918 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "prometheus_alerts_path", 2024-03-07T07:43:32.918 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.918 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.918 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.918 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-07T07:43:32.918 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.918 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.918 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.918 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-07T07:43:32.918 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.918 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.918 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.919 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.919 INFO:teuthology.orchestra.run.smithi160.stdout: "registry_insecure": { 2024-03-07T07:43:32.919 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "registry_insecure", 2024-03-07T07:43:32.919 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.919 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.919 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.919 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:32.919 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.919 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.919 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.919 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-07T07:43:32.919 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.919 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.919 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.920 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.920 INFO:teuthology.orchestra.run.smithi160.stdout: "registry_password": { 2024-03-07T07:43:32.920 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "registry_password", 2024-03-07T07:43:32.920 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.920 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.920 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.920 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:32.920 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.920 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.920 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.920 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-07T07:43:32.920 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.920 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.920 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.921 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.921 INFO:teuthology.orchestra.run.smithi160.stdout: "registry_url": { 2024-03-07T07:43:32.921 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "registry_url", 2024-03-07T07:43:32.921 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.921 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.921 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.921 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:32.921 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.921 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.921 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.921 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-07T07:43:32.921 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.921 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.922 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.922 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.922 INFO:teuthology.orchestra.run.smithi160.stdout: "registry_username": { 2024-03-07T07:43:32.922 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "registry_username", 2024-03-07T07:43:32.922 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.922 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.922 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.922 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:32.922 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.922 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.922 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.922 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-07T07:43:32.922 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.922 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.922 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.922 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.923 INFO:teuthology.orchestra.run.smithi160.stdout: "secure_monitoring_stack": { 2024-03-07T07:43:32.923 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "secure_monitoring_stack", 2024-03-07T07:43:32.923 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.923 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.923 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.923 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:32.923 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.923 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.923 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.923 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-07T07:43:32.923 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.923 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.923 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.923 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.924 INFO:teuthology.orchestra.run.smithi160.stdout: "service_discovery_port": { 2024-03-07T07:43:32.924 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "service_discovery_port", 2024-03-07T07:43:32.924 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:32.924 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.924 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.924 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "8765", 2024-03-07T07:43:32.924 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.924 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.924 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.924 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "cephadm service discovery port", 2024-03-07T07:43:32.924 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.924 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.924 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.925 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.925 INFO:teuthology.orchestra.run.smithi160.stdout: "ssh_config_file": { 2024-03-07T07:43:32.925 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "ssh_config_file", 2024-03-07T07:43:32.925 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.925 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.925 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.925 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:32.925 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.925 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.925 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.925 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-07T07:43:32.925 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.925 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.925 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.925 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.926 INFO:teuthology.orchestra.run.smithi160.stdout: "use_agent": { 2024-03-07T07:43:32.926 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "use_agent", 2024-03-07T07:43:32.926 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.926 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.926 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.926 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:32.926 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.926 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.926 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.926 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-07T07:43:32.926 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.926 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.926 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.926 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.926 INFO:teuthology.orchestra.run.smithi160.stdout: "use_repo_digest": { 2024-03-07T07:43:32.927 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "use_repo_digest", 2024-03-07T07:43:32.927 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.927 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.927 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.927 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:32.927 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.927 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.927 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.927 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-07T07:43:32.927 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.927 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.927 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.927 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.927 INFO:teuthology.orchestra.run.smithi160.stdout: "warn_on_failed_host_check": { 2024-03-07T07:43:32.927 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "warn_on_failed_host_check", 2024-03-07T07:43:32.928 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.928 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.928 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.928 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:32.928 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.928 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.928 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.928 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "raise a health warning if the host check fails", 2024-03-07T07:43:32.928 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.928 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.928 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.928 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.928 INFO:teuthology.orchestra.run.smithi160.stdout: "warn_on_stray_daemons": { 2024-03-07T07:43:32.928 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "warn_on_stray_daemons", 2024-03-07T07:43:32.929 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.929 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.929 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.929 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:32.929 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.929 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.929 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.929 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-07T07:43:32.929 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.929 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.929 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.929 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.929 INFO:teuthology.orchestra.run.smithi160.stdout: "warn_on_stray_hosts": { 2024-03-07T07:43:32.929 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "warn_on_stray_hosts", 2024-03-07T07:43:32.929 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.930 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.930 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.930 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:32.930 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.930 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.930 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.930 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-07T07:43:32.930 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.930 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.930 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.930 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:32.930 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:32.930 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.930 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:32.930 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "crash", 2024-03-07T07:43:32.931 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:32.931 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:32.931 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:32.931 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:32.931 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:32.931 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.931 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.931 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:32.931 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:32.931 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.931 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.931 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:32.931 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:32.931 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:32.931 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:32.932 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:32.932 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:32.932 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:32.932 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:32.932 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.932 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.932 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.932 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.932 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.932 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:32.932 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:32.932 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.932 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.932 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:32.933 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:32.933 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.933 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.933 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.933 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.933 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.933 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.933 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.933 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.933 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:32.933 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:32.933 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.933 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.933 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:32.934 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:32.934 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.934 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.934 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:32.934 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:32.934 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:32.934 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:32.934 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:32.934 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:32.934 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:32.934 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:32.934 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.934 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.934 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.934 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.935 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.935 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:32.935 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:32.935 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.935 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.935 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:32.935 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:32.935 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.935 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.935 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.935 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.935 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.935 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.935 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.935 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.936 INFO:teuthology.orchestra.run.smithi160.stdout: "retain_interval": { 2024-03-07T07:43:32.936 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "retain_interval", 2024-03-07T07:43:32.936 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:32.936 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.936 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:32.936 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "31536000", 2024-03-07T07:43:32.936 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.936 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.936 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.936 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-07T07:43:32.936 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.936 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.936 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.936 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.936 INFO:teuthology.orchestra.run.smithi160.stdout: "warn_recent_interval": { 2024-03-07T07:43:32.937 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "warn_recent_interval", 2024-03-07T07:43:32.937 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:32.937 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.937 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:32.937 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "1209600", 2024-03-07T07:43:32.937 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.937 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.937 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.937 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-07T07:43:32.937 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.937 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.937 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.937 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:32.937 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:32.937 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.937 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:32.938 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "dashboard", 2024-03-07T07:43:32.938 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:32.938 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:32.938 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:32.938 INFO:teuthology.orchestra.run.smithi160.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-07T07:43:32.938 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-07T07:43:32.938 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:32.938 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.938 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.938 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "10", 2024-03-07T07:43:32.938 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.938 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.938 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.938 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.938 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.939 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.939 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.939 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.939 INFO:teuthology.orchestra.run.smithi160.stdout: "ALERTMANAGER_API_HOST": { 2024-03-07T07:43:32.939 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-07T07:43:32.939 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.939 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.939 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.939 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:32.939 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.939 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.939 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.939 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.939 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.939 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.939 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.940 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.940 INFO:teuthology.orchestra.run.smithi160.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-07T07:43:32.940 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-07T07:43:32.940 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.940 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.940 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.940 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:32.940 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.940 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.940 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.940 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.940 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.940 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.941 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.941 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.941 INFO:teuthology.orchestra.run.smithi160.stdout: "AUDIT_API_ENABLED": { 2024-03-07T07:43:32.941 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "AUDIT_API_ENABLED", 2024-03-07T07:43:32.941 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.941 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.941 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.941 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:32.941 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.941 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.941 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.941 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.941 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.941 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.941 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.941 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.942 INFO:teuthology.orchestra.run.smithi160.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-07T07:43:32.942 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-07T07:43:32.942 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.942 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.942 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.942 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:32.942 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.942 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.942 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.942 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.942 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.942 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.942 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.942 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.943 INFO:teuthology.orchestra.run.smithi160.stdout: "ENABLE_BROWSABLE_API": { 2024-03-07T07:43:32.943 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-07T07:43:32.943 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.943 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.943 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.943 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:32.943 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.943 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.943 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.943 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.943 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.943 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.943 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.943 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.943 INFO:teuthology.orchestra.run.smithi160.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-07T07:43:32.943 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-07T07:43:32.944 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.944 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.944 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.944 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:32.944 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.944 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.944 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.944 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.944 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.944 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.944 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.944 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.944 INFO:teuthology.orchestra.run.smithi160.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-07T07:43:32.944 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-07T07:43:32.945 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.945 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.945 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.945 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:32.945 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.945 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.945 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.945 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.945 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.945 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.945 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.945 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.945 INFO:teuthology.orchestra.run.smithi160.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-07T07:43:32.945 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-07T07:43:32.946 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.946 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.946 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.946 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:32.946 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.946 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.946 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.946 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.946 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.946 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.946 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.946 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.946 INFO:teuthology.orchestra.run.smithi160.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-07T07:43:32.946 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-07T07:43:32.947 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.947 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.947 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.947 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:32.947 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.947 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.947 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.947 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.947 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.947 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.947 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.947 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.947 INFO:teuthology.orchestra.run.smithi160.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-07T07:43:32.947 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-07T07:43:32.948 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.948 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.948 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.948 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:32.948 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.948 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.948 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.948 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.948 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.948 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.948 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.948 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.948 INFO:teuthology.orchestra.run.smithi160.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-07T07:43:32.949 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-07T07:43:32.949 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.949 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.949 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.949 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:32.949 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.949 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.949 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.949 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.949 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.949 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.949 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.949 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.949 INFO:teuthology.orchestra.run.smithi160.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-07T07:43:32.950 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-07T07:43:32.950 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.950 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.950 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.950 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:32.950 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.950 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.950 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.950 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.950 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.950 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.950 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.950 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.950 INFO:teuthology.orchestra.run.smithi160.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-07T07:43:32.951 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-07T07:43:32.951 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.951 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.951 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.951 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:32.951 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.951 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.951 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.951 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.951 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.951 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.951 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.951 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.951 INFO:teuthology.orchestra.run.smithi160.stdout: "GRAFANA_API_PASSWORD": { 2024-03-07T07:43:32.952 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-07T07:43:32.952 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.952 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.952 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.952 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "admin", 2024-03-07T07:43:32.952 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.952 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.952 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.952 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.952 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.952 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.952 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.952 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.953 INFO:teuthology.orchestra.run.smithi160.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-07T07:43:32.953 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-07T07:43:32.953 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.953 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.953 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.953 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:32.953 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.953 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.953 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.953 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.953 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.953 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.953 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.953 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.953 INFO:teuthology.orchestra.run.smithi160.stdout: "GRAFANA_API_URL": { 2024-03-07T07:43:32.954 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "GRAFANA_API_URL", 2024-03-07T07:43:32.954 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.954 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.954 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.954 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:32.954 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.954 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.954 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.954 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.954 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.954 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.954 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.954 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.954 INFO:teuthology.orchestra.run.smithi160.stdout: "GRAFANA_API_USERNAME": { 2024-03-07T07:43:32.955 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-07T07:43:32.955 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.955 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.955 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.955 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "admin", 2024-03-07T07:43:32.955 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.955 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.955 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.955 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.955 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.955 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.955 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.955 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.956 INFO:teuthology.orchestra.run.smithi160.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-07T07:43:32.956 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-07T07:43:32.956 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.956 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.956 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.956 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:32.956 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.956 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.956 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.956 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.956 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.956 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.956 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.956 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.957 INFO:teuthology.orchestra.run.smithi160.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-07T07:43:32.957 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-07T07:43:32.957 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.957 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.957 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.957 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:32.957 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.957 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.957 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.957 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.957 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.957 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.957 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.958 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.958 INFO:teuthology.orchestra.run.smithi160.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-07T07:43:32.958 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-07T07:43:32.958 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.958 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.958 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.958 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:32.958 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.958 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.958 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.958 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.958 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.958 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.958 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.958 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.959 INFO:teuthology.orchestra.run.smithi160.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-07T07:43:32.959 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-07T07:43:32.959 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.959 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.959 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.959 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:32.959 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.959 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.959 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.959 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.959 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.959 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.960 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.960 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.960 INFO:teuthology.orchestra.run.smithi160.stdout: "PROMETHEUS_API_HOST": { 2024-03-07T07:43:32.960 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-07T07:43:32.960 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.960 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.960 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.960 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:32.960 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.960 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.960 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.960 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.960 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.960 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.961 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.961 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.961 INFO:teuthology.orchestra.run.smithi160.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-07T07:43:32.961 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-07T07:43:32.961 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.961 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.961 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.961 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:32.961 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.961 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.961 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.961 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.961 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.961 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.961 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.962 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.962 INFO:teuthology.orchestra.run.smithi160.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-07T07:43:32.962 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-07T07:43:32.962 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.962 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.962 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.962 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:32.962 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.962 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.962 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.962 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.962 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.963 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.963 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.963 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.963 INFO:teuthology.orchestra.run.smithi160.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-07T07:43:32.963 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-07T07:43:32.963 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.963 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.963 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.963 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:32.963 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.963 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.963 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.963 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.963 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.963 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.964 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.964 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.964 INFO:teuthology.orchestra.run.smithi160.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-07T07:43:32.964 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-07T07:43:32.964 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.964 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.964 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.964 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:32.964 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.964 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.964 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.964 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.964 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.965 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.965 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.965 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.965 INFO:teuthology.orchestra.run.smithi160.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-07T07:43:32.965 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-07T07:43:32.965 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.965 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.965 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.965 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:32.965 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.965 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.965 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.965 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.965 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.966 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.966 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.966 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.966 INFO:teuthology.orchestra.run.smithi160.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-07T07:43:32.966 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-07T07:43:32.966 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.966 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.966 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.966 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:32.966 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.966 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.966 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.966 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.966 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.966 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.967 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.967 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.967 INFO:teuthology.orchestra.run.smithi160.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-07T07:43:32.967 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-07T07:43:32.967 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.967 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.967 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.967 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:32.967 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.967 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.967 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.967 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.967 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.968 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.968 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.968 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.968 INFO:teuthology.orchestra.run.smithi160.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-07T07:43:32.968 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-07T07:43:32.968 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.968 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.968 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.968 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:32.968 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.968 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.968 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.968 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.968 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.969 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.969 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.969 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.969 INFO:teuthology.orchestra.run.smithi160.stdout: "PWD_POLICY_ENABLED": { 2024-03-07T07:43:32.969 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-07T07:43:32.969 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.969 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.969 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.969 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:32.969 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.969 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.969 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.969 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.969 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.969 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.970 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.970 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.970 INFO:teuthology.orchestra.run.smithi160.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-07T07:43:32.970 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-07T07:43:32.970 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.970 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.970 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.970 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-07T07:43:32.970 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.970 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.970 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.970 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.970 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.970 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.971 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.971 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.971 INFO:teuthology.orchestra.run.smithi160.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-07T07:43:32.971 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-07T07:43:32.971 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:32.971 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.971 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.971 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "10", 2024-03-07T07:43:32.971 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.971 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.971 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.971 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.971 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.971 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.971 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.972 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.972 INFO:teuthology.orchestra.run.smithi160.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-07T07:43:32.972 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-07T07:43:32.972 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:32.972 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.972 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.972 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "8", 2024-03-07T07:43:32.972 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.972 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.972 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.972 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.972 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.972 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.973 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.973 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.973 INFO:teuthology.orchestra.run.smithi160.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-07T07:43:32.973 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-07T07:43:32.973 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:32.973 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.973 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.973 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "45", 2024-03-07T07:43:32.973 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.973 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.973 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.973 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.973 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.973 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.974 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.974 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.974 INFO:teuthology.orchestra.run.smithi160.stdout: "RGW_API_ACCESS_KEY": { 2024-03-07T07:43:32.974 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-07T07:43:32.974 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.974 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.974 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.974 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:32.974 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.974 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.974 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.974 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.974 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.975 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.975 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.975 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.975 INFO:teuthology.orchestra.run.smithi160.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-07T07:43:32.975 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-07T07:43:32.975 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.975 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.975 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.975 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "admin", 2024-03-07T07:43:32.975 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.975 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.975 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.975 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.975 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.975 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.976 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.976 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.976 INFO:teuthology.orchestra.run.smithi160.stdout: "RGW_API_SECRET_KEY": { 2024-03-07T07:43:32.976 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-07T07:43:32.976 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.976 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.976 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.976 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:32.976 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.976 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.976 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.976 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.976 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.977 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.977 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.977 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.977 INFO:teuthology.orchestra.run.smithi160.stdout: "RGW_API_SSL_VERIFY": { 2024-03-07T07:43:32.977 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-07T07:43:32.977 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.977 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.977 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.977 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:32.977 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.977 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.977 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.977 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.978 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.978 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.978 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.978 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.978 INFO:teuthology.orchestra.run.smithi160.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-07T07:43:32.978 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-07T07:43:32.978 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:32.978 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.978 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.978 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "0", 2024-03-07T07:43:32.978 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.978 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.978 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.978 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.979 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.979 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.979 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.979 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.979 INFO:teuthology.orchestra.run.smithi160.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-07T07:43:32.979 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-07T07:43:32.979 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:32.979 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.979 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.979 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "10", 2024-03-07T07:43:32.979 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.979 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.979 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.980 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.980 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.980 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.980 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.980 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.980 INFO:teuthology.orchestra.run.smithi160.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-07T07:43:32.980 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-07T07:43:32.980 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:32.980 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.980 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.980 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "5", 2024-03-07T07:43:32.980 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.980 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.981 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.981 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.981 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.981 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.981 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.981 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.981 INFO:teuthology.orchestra.run.smithi160.stdout: "cross_origin_url": { 2024-03-07T07:43:32.981 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "cross_origin_url", 2024-03-07T07:43:32.981 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.981 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.981 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.981 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:32.981 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.982 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.982 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.982 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.982 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.982 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.982 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.982 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.982 INFO:teuthology.orchestra.run.smithi160.stdout: "crt_file": { 2024-03-07T07:43:32.982 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "crt_file", 2024-03-07T07:43:32.982 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.982 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.982 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.982 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:32.982 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.982 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.983 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.983 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.983 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.983 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.983 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.983 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.983 INFO:teuthology.orchestra.run.smithi160.stdout: "debug": { 2024-03-07T07:43:32.983 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "debug", 2024-03-07T07:43:32.983 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.983 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.983 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.983 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:32.983 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.983 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.984 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.984 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Enable/disable debug options", 2024-03-07T07:43:32.984 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.984 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.984 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.984 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.984 INFO:teuthology.orchestra.run.smithi160.stdout: "jwt_token_ttl": { 2024-03-07T07:43:32.984 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "jwt_token_ttl", 2024-03-07T07:43:32.984 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:32.984 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.984 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.984 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "28800", 2024-03-07T07:43:32.984 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.985 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.985 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.985 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.985 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.985 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.985 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.985 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.985 INFO:teuthology.orchestra.run.smithi160.stdout: "key_file": { 2024-03-07T07:43:32.985 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "key_file", 2024-03-07T07:43:32.985 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.985 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.985 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.985 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:32.986 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.986 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.986 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.986 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.986 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.986 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.986 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.986 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.986 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:32.986 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:32.986 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.986 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.987 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:32.987 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:32.987 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.987 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.987 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:32.987 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:32.987 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:32.987 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:32.987 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:32.987 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:32.987 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:32.987 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:32.987 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.987 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.988 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.988 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.988 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.988 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:32.988 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:32.988 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.988 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.988 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:32.988 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:32.988 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.988 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.988 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.988 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.989 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.989 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.989 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.989 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.989 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:32.989 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:32.989 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.989 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.989 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:32.989 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:32.989 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.989 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.989 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:32.990 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:32.990 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:32.990 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:32.990 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:32.990 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:32.990 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:32.990 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:32.990 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.990 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.990 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.990 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.990 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.990 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:32.990 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:32.991 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.991 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.991 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:32.991 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:32.991 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.991 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.991 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.991 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.991 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.991 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.991 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.991 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.991 INFO:teuthology.orchestra.run.smithi160.stdout: "motd": { 2024-03-07T07:43:32.991 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "motd", 2024-03-07T07:43:32.992 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.992 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.992 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.992 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:32.992 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.992 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.992 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.992 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "The message of the day", 2024-03-07T07:43:32.992 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.992 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.992 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.992 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.992 INFO:teuthology.orchestra.run.smithi160.stdout: "redirect_resolve_ip_addr": { 2024-03-07T07:43:32.992 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "redirect_resolve_ip_addr", 2024-03-07T07:43:32.992 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.993 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.993 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.993 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:32.993 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.993 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.993 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.993 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.993 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.993 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.993 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.993 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.993 INFO:teuthology.orchestra.run.smithi160.stdout: "server_addr": { 2024-03-07T07:43:32.993 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "server_addr", 2024-03-07T07:43:32.993 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.994 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.994 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.994 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "::", 2024-03-07T07:43:32.994 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.994 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.994 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.994 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.994 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.994 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.994 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.994 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.994 INFO:teuthology.orchestra.run.smithi160.stdout: "server_port": { 2024-03-07T07:43:32.994 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "server_port", 2024-03-07T07:43:32.994 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:32.994 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.995 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.995 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "8080", 2024-03-07T07:43:32.995 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.995 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.995 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.995 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.995 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.995 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.995 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.995 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.995 INFO:teuthology.orchestra.run.smithi160.stdout: "ssl": { 2024-03-07T07:43:32.995 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "ssl", 2024-03-07T07:43:32.995 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:32.996 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.996 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.996 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:32.996 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.996 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.996 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.996 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.996 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.996 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.996 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.996 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.996 INFO:teuthology.orchestra.run.smithi160.stdout: "ssl_server_port": { 2024-03-07T07:43:32.996 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "ssl_server_port", 2024-03-07T07:43:32.996 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:32.997 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.997 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.997 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "8443", 2024-03-07T07:43:32.997 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.997 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.997 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.997 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.997 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.997 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.997 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.997 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.997 INFO:teuthology.orchestra.run.smithi160.stdout: "standby_behaviour": { 2024-03-07T07:43:32.997 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "standby_behaviour", 2024-03-07T07:43:32.997 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:32.998 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.998 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.998 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "redirect", 2024-03-07T07:43:32.998 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:32.998 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:32.998 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:32.998 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:32.998 INFO:teuthology.orchestra.run.smithi160.stdout: "redirect" 2024-03-07T07:43:32.998 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:32.998 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.998 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.998 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.998 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:32.998 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:32.998 INFO:teuthology.orchestra.run.smithi160.stdout: "standby_error_status_code": { 2024-03-07T07:43:32.999 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "standby_error_status_code", 2024-03-07T07:43:32.999 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:32.999 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:32.999 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:32.999 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "500", 2024-03-07T07:43:32.999 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "400", 2024-03-07T07:43:32.999 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "599", 2024-03-07T07:43:32.999 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:32.999 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:32.999 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:32.999 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:32.999 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.000 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.000 INFO:teuthology.orchestra.run.smithi160.stdout: "url_prefix": { 2024-03-07T07:43:33.000 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "url_prefix", 2024-03-07T07:43:33.000 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.000 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.000 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.000 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.000 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.000 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.000 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.000 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.000 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.000 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.000 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.001 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.001 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.001 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.001 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.001 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "devicehealth", 2024-03-07T07:43:33.001 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.001 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.001 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.001 INFO:teuthology.orchestra.run.smithi160.stdout: "enable_monitoring": { 2024-03-07T07:43:33.001 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "enable_monitoring", 2024-03-07T07:43:33.001 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.001 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.001 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.001 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.001 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.002 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.002 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.002 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "monitor device health metrics", 2024-03-07T07:43:33.002 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.002 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.002 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.002 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.002 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.002 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.002 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.002 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.002 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.002 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.002 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.003 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.003 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.003 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.003 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.003 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.003 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.003 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.003 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.003 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.003 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.003 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.003 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.003 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.003 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.003 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.004 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.004 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.004 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.004 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.004 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.004 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.004 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.004 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.004 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.004 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.004 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.004 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.004 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.004 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.005 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.005 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.005 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.005 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.005 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.005 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.005 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.005 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.005 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.005 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.005 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.005 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.005 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.006 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.006 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.006 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.006 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.006 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.006 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.006 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.006 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.006 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.006 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.006 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.006 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.006 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.006 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.007 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.007 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.007 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.007 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.007 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.007 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.007 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.007 INFO:teuthology.orchestra.run.smithi160.stdout: "mark_out_threshold": { 2024-03-07T07:43:33.007 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "mark_out_threshold", 2024-03-07T07:43:33.007 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:33.007 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.007 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.007 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "2419200", 2024-03-07T07:43:33.007 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.007 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.008 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.008 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-07T07:43:33.008 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.008 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.008 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.008 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.008 INFO:teuthology.orchestra.run.smithi160.stdout: "pool_name": { 2024-03-07T07:43:33.008 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "pool_name", 2024-03-07T07:43:33.008 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.008 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.008 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.008 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "device_health_metrics", 2024-03-07T07:43:33.008 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.009 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.009 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.009 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-07T07:43:33.009 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.009 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.009 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.009 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.009 INFO:teuthology.orchestra.run.smithi160.stdout: "retention_period": { 2024-03-07T07:43:33.009 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "retention_period", 2024-03-07T07:43:33.009 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:33.009 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.009 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.009 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "15552000", 2024-03-07T07:43:33.010 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.010 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.010 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.010 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "how long to retain device health metrics", 2024-03-07T07:43:33.010 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.010 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.010 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.010 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.010 INFO:teuthology.orchestra.run.smithi160.stdout: "scrape_frequency": { 2024-03-07T07:43:33.010 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "scrape_frequency", 2024-03-07T07:43:33.010 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:33.010 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.010 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.010 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "86400", 2024-03-07T07:43:33.011 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.011 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.011 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.011 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-07T07:43:33.011 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.011 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.011 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.011 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.011 INFO:teuthology.orchestra.run.smithi160.stdout: "self_heal": { 2024-03-07T07:43:33.011 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "self_heal", 2024-03-07T07:43:33.011 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.011 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.011 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.011 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.011 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.012 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.012 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.012 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-07T07:43:33.012 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.012 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.012 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.012 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.012 INFO:teuthology.orchestra.run.smithi160.stdout: "sleep_interval": { 2024-03-07T07:43:33.012 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "sleep_interval", 2024-03-07T07:43:33.012 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:33.012 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.012 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.012 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "600", 2024-03-07T07:43:33.013 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.013 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.013 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.013 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "how frequently to wake up and check device health", 2024-03-07T07:43:33.013 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.013 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.013 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.013 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.013 INFO:teuthology.orchestra.run.smithi160.stdout: "warn_threshold": { 2024-03-07T07:43:33.013 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "warn_threshold", 2024-03-07T07:43:33.013 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:33.013 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.013 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.014 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "7257600", 2024-03-07T07:43:33.014 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.014 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.014 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.014 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-07T07:43:33.014 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.014 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.014 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.014 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.014 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.014 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.014 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.014 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "diskprediction_local", 2024-03-07T07:43:33.014 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.015 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.015 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.015 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.015 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.015 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.015 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.015 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.015 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.015 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.015 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.015 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.015 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.015 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.015 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.016 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.016 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.016 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.016 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.016 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.016 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.016 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.016 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.016 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.016 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.016 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.016 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.016 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.016 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.017 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.017 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.017 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.017 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.017 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.017 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.017 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.017 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.017 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.017 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.017 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.017 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.017 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.017 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.018 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.018 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.018 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.018 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.018 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.018 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.018 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.018 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.018 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.018 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.018 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.018 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.018 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.018 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.019 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.019 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.019 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.019 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.019 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.019 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.019 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.019 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.019 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.019 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.019 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.019 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.019 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.019 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.020 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.020 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.020 INFO:teuthology.orchestra.run.smithi160.stdout: "predict_interval": { 2024-03-07T07:43:33.020 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "predict_interval", 2024-03-07T07:43:33.020 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.020 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.020 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.020 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "86400", 2024-03-07T07:43:33.020 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.020 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.020 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.020 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.020 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.020 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.021 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.021 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.021 INFO:teuthology.orchestra.run.smithi160.stdout: "predictor_model": { 2024-03-07T07:43:33.021 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "predictor_model", 2024-03-07T07:43:33.021 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.021 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.021 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.021 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "prophetstor", 2024-03-07T07:43:33.021 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.021 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.021 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.021 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.021 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.021 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.022 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.022 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.022 INFO:teuthology.orchestra.run.smithi160.stdout: "sleep_interval": { 2024-03-07T07:43:33.022 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "sleep_interval", 2024-03-07T07:43:33.022 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.022 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.022 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.022 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "600", 2024-03-07T07:43:33.022 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.022 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.022 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.022 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.022 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.022 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.023 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.023 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.023 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.023 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.023 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.023 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "influx", 2024-03-07T07:43:33.023 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": false, 2024-03-07T07:43:33.023 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "influxdb python module not found", 2024-03-07T07:43:33.023 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.023 INFO:teuthology.orchestra.run.smithi160.stdout: "batch_size": { 2024-03-07T07:43:33.023 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "batch_size", 2024-03-07T07:43:33.023 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.023 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.023 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.023 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "5000", 2024-03-07T07:43:33.023 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.024 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.024 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.024 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-07T07:43:33.024 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.024 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.024 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.024 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.024 INFO:teuthology.orchestra.run.smithi160.stdout: "database": { 2024-03-07T07:43:33.024 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "database", 2024-03-07T07:43:33.024 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.024 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.024 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.024 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "ceph", 2024-03-07T07:43:33.025 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.025 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.025 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.025 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-07T07:43:33.025 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.025 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.025 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.025 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.025 INFO:teuthology.orchestra.run.smithi160.stdout: "hostname": { 2024-03-07T07:43:33.025 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "hostname", 2024-03-07T07:43:33.025 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.025 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.025 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.026 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.026 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.026 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.026 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.026 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "InfluxDB server hostname", 2024-03-07T07:43:33.026 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.026 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.026 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.026 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.026 INFO:teuthology.orchestra.run.smithi160.stdout: "interval": { 2024-03-07T07:43:33.026 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "interval", 2024-03-07T07:43:33.026 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:33.027 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.027 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.027 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "30", 2024-03-07T07:43:33.027 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "5", 2024-03-07T07:43:33.027 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.027 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.027 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-07T07:43:33.027 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.027 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.027 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.027 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.027 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.027 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.027 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.028 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.028 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.028 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.028 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.028 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.028 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.028 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.028 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.028 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.028 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.028 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.028 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.028 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.028 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.029 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.029 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.029 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.029 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.029 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.029 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.029 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.029 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.029 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.029 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.029 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.029 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.029 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.029 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.030 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.030 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.030 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.030 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.030 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.030 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.030 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.030 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.030 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.030 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.030 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.030 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.030 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.031 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.031 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.031 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.031 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.031 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.031 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.031 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.031 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.031 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.031 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.031 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.031 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.031 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.031 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.032 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.032 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.032 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.032 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.032 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.032 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.032 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.032 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.032 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.032 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.032 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.032 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.032 INFO:teuthology.orchestra.run.smithi160.stdout: "password": { 2024-03-07T07:43:33.032 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "password", 2024-03-07T07:43:33.033 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.033 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.033 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.033 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.033 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.033 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.033 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.033 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "password of InfluxDB server user", 2024-03-07T07:43:33.033 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.033 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.033 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.033 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.033 INFO:teuthology.orchestra.run.smithi160.stdout: "port": { 2024-03-07T07:43:33.033 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "port", 2024-03-07T07:43:33.034 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.034 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.034 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.034 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "8086", 2024-03-07T07:43:33.034 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.034 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.034 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.034 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "InfluxDB server port", 2024-03-07T07:43:33.034 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.034 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.034 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.034 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.034 INFO:teuthology.orchestra.run.smithi160.stdout: "ssl": { 2024-03-07T07:43:33.034 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "ssl", 2024-03-07T07:43:33.035 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.035 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.035 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.035 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "false", 2024-03-07T07:43:33.035 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.035 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.035 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.035 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-07T07:43:33.035 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.035 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.035 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.035 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.035 INFO:teuthology.orchestra.run.smithi160.stdout: "threads": { 2024-03-07T07:43:33.035 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "threads", 2024-03-07T07:43:33.035 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.036 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.036 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.036 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "5", 2024-03-07T07:43:33.036 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "1", 2024-03-07T07:43:33.036 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "32", 2024-03-07T07:43:33.036 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.036 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-07T07:43:33.036 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.036 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.036 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.036 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.036 INFO:teuthology.orchestra.run.smithi160.stdout: "username": { 2024-03-07T07:43:33.036 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "username", 2024-03-07T07:43:33.036 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.037 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.037 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.037 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.037 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.037 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.037 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.037 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "username of InfluxDB server user", 2024-03-07T07:43:33.037 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.037 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.037 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.037 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.037 INFO:teuthology.orchestra.run.smithi160.stdout: "verify_ssl": { 2024-03-07T07:43:33.037 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "verify_ssl", 2024-03-07T07:43:33.037 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.038 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.038 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.038 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "true", 2024-03-07T07:43:33.038 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.038 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.038 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.038 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-07T07:43:33.038 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.038 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.038 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.038 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.038 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.038 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.038 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.038 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "insights", 2024-03-07T07:43:33.039 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.039 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.039 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.039 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.039 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.039 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.039 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.039 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.039 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.039 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.039 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.039 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.039 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.039 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.039 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.040 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.040 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.040 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.040 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.040 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.040 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.040 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.040 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.040 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.040 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.040 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.040 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.040 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.041 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.041 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.041 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.041 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.041 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.041 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.041 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.041 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.041 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.041 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.041 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.041 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.041 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.041 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.041 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.041 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.042 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.042 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.042 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.042 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.042 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.042 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.042 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.042 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.042 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.042 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.042 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.042 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.042 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.042 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.042 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.042 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.043 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.043 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.043 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.043 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.043 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.043 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.043 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.043 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.043 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.043 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.043 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.043 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.043 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.043 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.043 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.044 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.044 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "iostat", 2024-03-07T07:43:33.044 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.044 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.044 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.044 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.044 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.044 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.044 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.044 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.044 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.044 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.044 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.044 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.045 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.045 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.045 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.045 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.045 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.045 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.045 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.045 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.045 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.045 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.045 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.045 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.045 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.045 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.045 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.046 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.046 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.046 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.046 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.046 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.046 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.046 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.046 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.046 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.046 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.046 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.046 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.046 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.046 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.046 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.046 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.047 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.047 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.047 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.047 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.047 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.047 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.047 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.047 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.047 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.047 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.047 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.047 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.047 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.047 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.047 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.048 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.048 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.048 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.048 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.048 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.048 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.048 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.048 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.048 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.048 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.048 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.048 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.049 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.049 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.049 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.049 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.049 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.049 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.049 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "k8sevents", 2024-03-07T07:43:33.049 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.049 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.049 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.049 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_event_retention_days": { 2024-03-07T07:43:33.049 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "ceph_event_retention_days", 2024-03-07T07:43:33.049 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.049 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.050 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.050 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "7", 2024-03-07T07:43:33.050 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.050 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.050 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.050 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-07T07:43:33.050 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.050 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.050 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.050 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.050 INFO:teuthology.orchestra.run.smithi160.stdout: "config_check_secs": { 2024-03-07T07:43:33.050 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "config_check_secs", 2024-03-07T07:43:33.050 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.051 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.051 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.051 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "10", 2024-03-07T07:43:33.051 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "10", 2024-03-07T07:43:33.051 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.051 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.051 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-07T07:43:33.051 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.051 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.051 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.051 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.051 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.051 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.051 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.052 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.052 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.052 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.052 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.052 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.052 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.052 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.052 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.052 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.052 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.052 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.052 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.052 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.052 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.053 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.053 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.053 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.053 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.053 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.053 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.053 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.053 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.053 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.053 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.053 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.053 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.053 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.054 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.054 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.054 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.054 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.054 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.054 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.054 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.054 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.054 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.054 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.054 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.054 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.054 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.054 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.055 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.055 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.055 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.055 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.055 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.055 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.055 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.055 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.055 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.055 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.055 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.055 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.055 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.055 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.056 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.056 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.056 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.056 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.056 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.056 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.056 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.056 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.056 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.056 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.056 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.056 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.056 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.057 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.057 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.057 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "localpool", 2024-03-07T07:43:33.057 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.057 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.057 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.057 INFO:teuthology.orchestra.run.smithi160.stdout: "failure_domain": { 2024-03-07T07:43:33.057 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "failure_domain", 2024-03-07T07:43:33.057 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.057 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.057 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.057 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "host", 2024-03-07T07:43:33.057 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.057 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.058 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.058 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "failure domain for any created local pool", 2024-03-07T07:43:33.058 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-07T07:43:33.058 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.058 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.058 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.058 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.058 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.058 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.058 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.058 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.058 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.058 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.059 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.059 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.059 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.059 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.059 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.059 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.059 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.059 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.059 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.059 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.059 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.059 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.059 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.060 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.060 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.060 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.060 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.060 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.060 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.060 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.060 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.060 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.060 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.060 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.060 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.060 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.060 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.061 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.061 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.061 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.061 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.061 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.061 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.061 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.061 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.061 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.061 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.061 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.061 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.061 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.062 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.062 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.062 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.062 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.062 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.062 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.062 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.062 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.062 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.062 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.062 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.062 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.062 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.062 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.063 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.063 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.063 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.063 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.063 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.063 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.063 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.063 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.065 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.065 INFO:teuthology.orchestra.run.smithi160.stdout: "min_size": { 2024-03-07T07:43:33.065 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "min_size", 2024-03-07T07:43:33.066 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.066 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.066 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.066 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.066 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.066 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.066 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.066 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "default min_size for any created local pool", 2024-03-07T07:43:33.066 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-07T07:43:33.066 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.066 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.066 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.067 INFO:teuthology.orchestra.run.smithi160.stdout: "num_rep": { 2024-03-07T07:43:33.067 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "num_rep", 2024-03-07T07:43:33.067 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.067 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.067 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.067 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "3", 2024-03-07T07:43:33.067 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.067 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.067 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.067 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "default replica count for any created local pool", 2024-03-07T07:43:33.067 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.067 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.067 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.067 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.068 INFO:teuthology.orchestra.run.smithi160.stdout: "pg_num": { 2024-03-07T07:43:33.068 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "pg_num", 2024-03-07T07:43:33.068 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.068 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.068 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.068 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "128", 2024-03-07T07:43:33.068 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.068 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.068 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.068 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "default pg_num for any created local pool", 2024-03-07T07:43:33.068 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.068 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.068 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.068 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.069 INFO:teuthology.orchestra.run.smithi160.stdout: "prefix": { 2024-03-07T07:43:33.069 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "prefix", 2024-03-07T07:43:33.069 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.069 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.069 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.069 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.069 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.069 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.069 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.069 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "name prefix for any created local pool", 2024-03-07T07:43:33.069 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.069 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.070 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.070 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.070 INFO:teuthology.orchestra.run.smithi160.stdout: "subtree": { 2024-03-07T07:43:33.070 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "subtree", 2024-03-07T07:43:33.070 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.070 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.070 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.070 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "rack", 2024-03-07T07:43:33.070 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.070 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.070 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.070 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-07T07:43:33.070 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-07T07:43:33.071 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.071 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.071 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.071 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.071 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.071 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.071 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "mds_autoscaler", 2024-03-07T07:43:33.071 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.071 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.071 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.071 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.071 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.071 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.072 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.072 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.072 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.072 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.072 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.072 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.072 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.072 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.072 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.072 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.072 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.072 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.072 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.073 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.073 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.073 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.073 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.073 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.073 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.073 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.073 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.073 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.073 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.073 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.073 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.073 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.073 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.074 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.074 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.074 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.074 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.074 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.074 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.074 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.074 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.074 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.074 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.074 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.074 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.074 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.075 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.075 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.075 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.075 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.075 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.075 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.075 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.075 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.075 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.075 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.075 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.075 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.075 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.075 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.076 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.076 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.076 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.076 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.076 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.076 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.076 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.076 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.076 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.076 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.076 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.076 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.076 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.077 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.077 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.077 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.077 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "mirroring", 2024-03-07T07:43:33.077 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.077 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.077 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.077 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.077 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.077 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.077 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.077 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.077 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.077 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.077 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.078 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.078 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.078 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.078 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.078 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.078 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.078 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.078 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.078 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.078 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.078 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.078 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.078 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.078 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.079 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.079 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.079 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.079 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.079 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.079 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.079 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.079 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.079 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.079 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.079 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.079 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.079 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.079 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.080 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.080 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.080 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.080 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.080 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.080 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.080 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.080 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.080 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.080 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.080 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.080 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.080 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.080 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.081 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.081 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.081 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.081 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.081 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.081 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.081 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.081 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.081 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.081 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.081 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.081 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.081 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.081 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.082 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.082 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.082 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.082 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.082 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.082 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.082 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.082 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.082 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.082 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "nfs", 2024-03-07T07:43:33.082 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.082 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.082 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.082 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.082 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.083 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.083 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.083 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.083 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.083 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.083 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.083 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.083 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.083 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.083 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.083 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.083 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.083 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.083 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.083 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.083 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.084 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.084 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.084 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.084 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.084 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.084 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.084 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.084 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.084 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.084 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.084 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.084 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.085 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.085 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.085 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.085 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.085 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.085 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.085 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.085 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.085 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.085 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.085 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.085 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.085 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.085 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.085 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.086 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.086 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.086 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.086 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.086 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.086 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.086 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.086 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.086 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.086 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.086 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.086 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.086 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.087 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.087 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.087 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.087 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.087 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.087 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.087 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.087 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.087 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.087 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.087 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.087 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.087 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.087 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.088 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.088 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "orchestrator", 2024-03-07T07:43:33.088 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.088 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.088 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.088 INFO:teuthology.orchestra.run.smithi160.stdout: "fail_fs": { 2024-03-07T07:43:33.088 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "fail_fs", 2024-03-07T07:43:33.088 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.088 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.088 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.088 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.088 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.088 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.089 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.089 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-07T07:43:33.089 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.089 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.089 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.089 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.089 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.089 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.089 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.089 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.089 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.089 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.089 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.089 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.089 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.090 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.090 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.090 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.090 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.090 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.090 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.090 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.090 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.090 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.090 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.090 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.090 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.090 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.090 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.091 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.091 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.091 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.091 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.091 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.091 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.091 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.091 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.091 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.091 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.091 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.091 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.091 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.091 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.092 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.092 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.092 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.092 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.092 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.092 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.092 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.092 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.092 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.092 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.092 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.092 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.092 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.092 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.093 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.093 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.093 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.093 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.093 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.093 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.093 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.093 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.093 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.093 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.093 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.093 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.093 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.093 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.093 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.094 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.094 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.094 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.094 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.094 INFO:teuthology.orchestra.run.smithi160.stdout: "orchestrator": { 2024-03-07T07:43:33.094 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "orchestrator", 2024-03-07T07:43:33.094 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.094 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.094 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.094 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.094 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.094 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.094 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.094 INFO:teuthology.orchestra.run.smithi160.stdout: "cephadm", 2024-03-07T07:43:33.095 INFO:teuthology.orchestra.run.smithi160.stdout: "rook", 2024-03-07T07:43:33.095 INFO:teuthology.orchestra.run.smithi160.stdout: "test_orchestrator" 2024-03-07T07:43:33.095 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.095 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Orchestrator backend", 2024-03-07T07:43:33.095 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.095 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.095 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.095 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.095 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.095 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.095 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.095 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "osd_perf_query", 2024-03-07T07:43:33.095 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.095 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.096 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.096 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.096 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.096 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.096 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.096 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.096 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.096 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.096 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.096 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.096 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.096 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.096 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.096 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.097 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.097 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.097 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.097 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.097 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.097 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.097 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.097 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.097 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.097 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.097 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.097 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.097 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.097 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.098 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.098 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.098 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.098 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.098 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.098 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.098 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.098 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.098 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.098 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.098 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.098 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.099 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.099 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.099 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.099 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.099 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.099 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.099 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.099 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.099 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.099 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.099 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.099 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.099 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.100 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.100 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.100 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.100 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.100 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.100 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.100 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.100 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.100 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.100 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.100 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.100 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.100 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.101 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.101 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.101 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.101 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.101 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.101 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.101 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.101 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.101 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "osd_support", 2024-03-07T07:43:33.101 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.101 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.101 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.101 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.101 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.102 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.102 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.102 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.102 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.102 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.102 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.102 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.102 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.102 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.102 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.102 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.102 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.102 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.102 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.103 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.103 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.103 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.103 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.103 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.103 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.103 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.103 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.103 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.103 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.103 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.103 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.103 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.103 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.104 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.104 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.104 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.104 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.104 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.104 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.104 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.104 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.104 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.104 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.104 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.104 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.104 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.104 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.105 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.105 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.105 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.105 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.105 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.105 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.105 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.105 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.105 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.105 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.105 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.105 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.105 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.105 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.106 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.106 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.106 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.106 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.106 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.106 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.106 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.106 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.106 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.106 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.106 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.106 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.106 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.106 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.107 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.107 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "pg_autoscaler", 2024-03-07T07:43:33.107 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.107 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.107 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.107 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.107 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.107 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.107 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.107 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.107 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.107 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.107 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.107 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.108 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.108 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.108 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.108 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.108 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.108 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.108 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.108 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.108 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.108 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.108 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.108 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.108 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.108 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.109 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.109 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.109 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.109 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.109 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.109 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.109 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.109 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.109 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.109 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.109 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.109 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.109 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.109 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.109 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.110 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.110 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.110 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.110 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.110 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.110 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.110 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.110 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.110 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.110 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.110 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.110 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.110 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.110 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.110 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.111 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.111 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.111 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.111 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.111 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.111 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.111 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.111 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.111 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.111 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.111 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.111 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.111 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.111 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.111 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.112 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.112 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.112 INFO:teuthology.orchestra.run.smithi160.stdout: "sleep_interval": { 2024-03-07T07:43:33.112 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "sleep_interval", 2024-03-07T07:43:33.112 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:33.112 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.112 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.112 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "60", 2024-03-07T07:43:33.112 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.112 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.112 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.112 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.112 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.113 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.113 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.113 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.113 INFO:teuthology.orchestra.run.smithi160.stdout: "threshold": { 2024-03-07T07:43:33.113 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "threshold", 2024-03-07T07:43:33.113 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "float", 2024-03-07T07:43:33.113 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.113 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.113 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "3.0", 2024-03-07T07:43:33.113 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "1.0", 2024-03-07T07:43:33.113 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.113 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.113 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "scaling threshold", 2024-03-07T07:43:33.113 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-07T07:43:33.113 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.114 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.114 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.114 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.114 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.114 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.114 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "progress", 2024-03-07T07:43:33.114 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.114 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.114 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.114 INFO:teuthology.orchestra.run.smithi160.stdout: "allow_pg_recovery_event": { 2024-03-07T07:43:33.114 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "allow_pg_recovery_event", 2024-03-07T07:43:33.114 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.114 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.114 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.114 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.114 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.115 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.115 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.115 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-07T07:43:33.115 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.115 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.115 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.115 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.115 INFO:teuthology.orchestra.run.smithi160.stdout: "enabled": { 2024-03-07T07:43:33.115 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "enabled", 2024-03-07T07:43:33.115 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.115 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.115 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.115 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.115 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.115 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.116 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.116 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.116 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.116 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.116 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.116 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.116 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.116 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.116 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.116 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.116 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.116 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.116 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.116 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.117 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.117 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.117 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.117 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.117 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.117 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.117 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.117 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.117 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.117 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.117 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.117 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.117 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.117 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.117 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.118 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.118 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.118 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.118 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.118 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.118 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.118 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.118 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.118 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.118 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.118 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.118 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.118 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.118 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.119 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.119 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.119 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.119 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.119 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.119 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.119 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.119 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.119 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.119 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.119 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.119 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.119 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.119 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.120 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.120 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.120 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.120 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.120 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.120 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.120 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.120 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.120 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.120 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.120 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.120 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.120 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.121 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.121 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.121 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.121 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.121 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.121 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.121 INFO:teuthology.orchestra.run.smithi160.stdout: "max_completed_events": { 2024-03-07T07:43:33.121 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "max_completed_events", 2024-03-07T07:43:33.121 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.121 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.121 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.121 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "50", 2024-03-07T07:43:33.121 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.121 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.121 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.121 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "number of past completed events to remember", 2024-03-07T07:43:33.121 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.122 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.122 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.122 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.122 INFO:teuthology.orchestra.run.smithi160.stdout: "sleep_interval": { 2024-03-07T07:43:33.122 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "sleep_interval", 2024-03-07T07:43:33.122 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:33.122 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.122 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.122 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "5", 2024-03-07T07:43:33.122 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.122 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.122 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.122 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "how long the module is going to sleep", 2024-03-07T07:43:33.122 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.122 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.123 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.123 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.123 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.123 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.123 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.123 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "prometheus", 2024-03-07T07:43:33.123 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.123 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.123 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.123 INFO:teuthology.orchestra.run.smithi160.stdout: "cache": { 2024-03-07T07:43:33.123 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "cache", 2024-03-07T07:43:33.123 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.123 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.123 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.123 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.123 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.124 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.124 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.124 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.124 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.124 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.124 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.124 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.124 INFO:teuthology.orchestra.run.smithi160.stdout: "exclude_perf_counters": { 2024-03-07T07:43:33.124 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "exclude_perf_counters", 2024-03-07T07:43:33.124 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.124 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.124 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.124 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.125 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.125 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.125 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.125 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-07T07:43:33.125 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-07T07:43:33.125 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.125 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.125 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.125 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.125 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.125 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.125 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.125 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.126 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.126 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.126 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.126 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.126 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.126 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.126 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.126 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.126 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.126 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.126 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.126 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.126 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.126 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.126 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.127 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.127 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.127 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.127 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.127 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.127 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.127 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.127 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.127 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.127 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.127 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.127 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.127 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.127 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.127 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.128 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.128 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.128 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.128 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.128 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.128 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.128 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.128 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.128 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.128 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.128 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.128 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.128 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.129 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.129 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.129 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.129 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.129 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.129 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.129 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.129 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.129 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.129 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.129 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.129 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.129 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.129 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.129 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.130 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.130 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.130 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.130 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.130 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.130 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.130 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.130 INFO:teuthology.orchestra.run.smithi160.stdout: "rbd_stats_pools": { 2024-03-07T07:43:33.130 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "rbd_stats_pools", 2024-03-07T07:43:33.130 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.130 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.130 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.130 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.131 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.131 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.131 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.131 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.131 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.131 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.131 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.131 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.131 INFO:teuthology.orchestra.run.smithi160.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-07T07:43:33.131 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-07T07:43:33.131 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.131 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.131 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.131 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "300", 2024-03-07T07:43:33.131 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.132 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.132 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.132 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.132 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.132 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.132 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.132 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.132 INFO:teuthology.orchestra.run.smithi160.stdout: "scrape_interval": { 2024-03-07T07:43:33.132 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "scrape_interval", 2024-03-07T07:43:33.132 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "float", 2024-03-07T07:43:33.132 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.132 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.132 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "15.0", 2024-03-07T07:43:33.133 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.133 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.133 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.133 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.133 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.133 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.133 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.133 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.133 INFO:teuthology.orchestra.run.smithi160.stdout: "server_addr": { 2024-03-07T07:43:33.133 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "server_addr", 2024-03-07T07:43:33.133 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.133 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.133 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.133 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "::", 2024-03-07T07:43:33.133 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.134 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.134 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.134 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-07T07:43:33.134 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.134 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.134 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.134 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.134 INFO:teuthology.orchestra.run.smithi160.stdout: "server_port": { 2024-03-07T07:43:33.134 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "server_port", 2024-03-07T07:43:33.134 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.134 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.134 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.134 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "9283", 2024-03-07T07:43:33.134 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.134 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.134 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.135 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-07T07:43:33.135 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.135 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.135 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.135 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.135 INFO:teuthology.orchestra.run.smithi160.stdout: "stale_cache_strategy": { 2024-03-07T07:43:33.135 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "stale_cache_strategy", 2024-03-07T07:43:33.135 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.135 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.135 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.135 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "log", 2024-03-07T07:43:33.135 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.135 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.135 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.135 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.136 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.136 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.136 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.136 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.136 INFO:teuthology.orchestra.run.smithi160.stdout: "standby_behaviour": { 2024-03-07T07:43:33.136 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "standby_behaviour", 2024-03-07T07:43:33.136 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.136 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.136 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.136 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "default", 2024-03-07T07:43:33.136 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.136 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.136 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.137 INFO:teuthology.orchestra.run.smithi160.stdout: "default", 2024-03-07T07:43:33.137 INFO:teuthology.orchestra.run.smithi160.stdout: "error" 2024-03-07T07:43:33.137 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.137 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.137 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.137 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.137 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.137 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.137 INFO:teuthology.orchestra.run.smithi160.stdout: "standby_error_status_code": { 2024-03-07T07:43:33.137 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "standby_error_status_code", 2024-03-07T07:43:33.137 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.137 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.137 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.138 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "500", 2024-03-07T07:43:33.138 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "400", 2024-03-07T07:43:33.138 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "599", 2024-03-07T07:43:33.138 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.138 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.138 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.138 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.138 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.138 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.138 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.138 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.138 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.138 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "rbd_support", 2024-03-07T07:43:33.138 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.139 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.139 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.139 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.139 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.139 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.139 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.139 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.139 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.139 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.139 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.139 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.139 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.139 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.139 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.140 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.140 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.140 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.140 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.140 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.140 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.140 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.140 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.140 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.140 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.140 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.140 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.140 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.141 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.141 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.141 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.141 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.141 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.141 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.141 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.141 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.141 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.141 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.141 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.141 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.141 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.141 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.142 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.142 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.142 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.142 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.142 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.142 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.142 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.142 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.142 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.142 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.142 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.142 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.142 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.142 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.143 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.143 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.143 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.143 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.143 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.143 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.143 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.143 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.143 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.143 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.143 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.143 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.143 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.143 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.144 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.144 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.144 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.144 INFO:teuthology.orchestra.run.smithi160.stdout: "max_concurrent_snap_create": { 2024-03-07T07:43:33.144 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "max_concurrent_snap_create", 2024-03-07T07:43:33.144 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.144 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.144 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.144 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "10", 2024-03-07T07:43:33.144 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.144 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.144 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.144 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.144 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.145 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.145 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.145 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.145 INFO:teuthology.orchestra.run.smithi160.stdout: "mirror_snapshot_schedule": { 2024-03-07T07:43:33.145 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "mirror_snapshot_schedule", 2024-03-07T07:43:33.145 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.145 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.145 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.145 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.145 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.145 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.145 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.145 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.145 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.146 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.146 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.146 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.146 INFO:teuthology.orchestra.run.smithi160.stdout: "trash_purge_schedule": { 2024-03-07T07:43:33.146 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "trash_purge_schedule", 2024-03-07T07:43:33.146 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.146 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.146 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.146 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.146 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.146 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.146 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.146 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.146 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.146 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.146 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.147 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.147 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.147 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.147 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.147 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "restful", 2024-03-07T07:43:33.147 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.147 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.147 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.147 INFO:teuthology.orchestra.run.smithi160.stdout: "enable_auth": { 2024-03-07T07:43:33.147 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "enable_auth", 2024-03-07T07:43:33.147 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.147 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.147 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.147 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.147 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.148 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.148 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.148 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.148 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.148 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.148 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.148 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.148 INFO:teuthology.orchestra.run.smithi160.stdout: "key_file": { 2024-03-07T07:43:33.148 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "key_file", 2024-03-07T07:43:33.148 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.148 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.148 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.148 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.148 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.149 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.149 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.149 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.149 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.149 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.149 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.149 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.149 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.149 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.149 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.149 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.149 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.149 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.149 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.149 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.150 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.150 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.150 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.150 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.150 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.150 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.150 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.150 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.150 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.150 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.150 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.150 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.150 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.150 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.151 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.151 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.151 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.151 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.151 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.151 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.151 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.151 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.151 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.151 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.151 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.152 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.152 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.152 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.152 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.152 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.152 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.152 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.152 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.152 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.152 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.152 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.152 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.152 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.152 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.153 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.153 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.153 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.153 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.153 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.153 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.153 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.153 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.153 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.153 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.153 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.153 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.153 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.153 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.154 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.154 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.154 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.154 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.154 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.154 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.154 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.154 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.154 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.154 INFO:teuthology.orchestra.run.smithi160.stdout: "server_addr": { 2024-03-07T07:43:33.154 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "server_addr", 2024-03-07T07:43:33.154 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.154 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.154 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.155 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.155 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.155 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.155 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.155 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.155 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.155 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.155 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.155 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.155 INFO:teuthology.orchestra.run.smithi160.stdout: "server_port": { 2024-03-07T07:43:33.155 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "server_port", 2024-03-07T07:43:33.155 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.155 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.155 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.156 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.156 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.156 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.156 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.156 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.156 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.156 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.156 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.156 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.156 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.156 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.156 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.156 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "rgw", 2024-03-07T07:43:33.156 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.157 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.157 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.157 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.157 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.157 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.157 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.157 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.157 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.157 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.157 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.157 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.157 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.157 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.157 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.158 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.158 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.158 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.158 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.158 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.158 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.158 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.158 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.158 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.158 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.158 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.158 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.158 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.158 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.159 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.159 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.159 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.159 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.159 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.159 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.159 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.159 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.159 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.159 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.159 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.159 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.159 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.159 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.160 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.160 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.160 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.160 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.160 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.160 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.160 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.160 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.160 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.160 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.160 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.160 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.160 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.161 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.161 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.161 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.161 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.161 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.161 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.161 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.161 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.161 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.161 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.161 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.161 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.161 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.161 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.161 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.162 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.162 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.162 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.162 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.162 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.162 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "rook", 2024-03-07T07:43:33.162 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.162 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.162 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.162 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.162 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.162 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.162 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.162 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.162 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.163 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.163 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.163 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.163 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.163 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.163 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.163 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.163 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.163 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.163 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.163 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.163 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.163 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.163 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.164 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.164 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.164 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.164 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.164 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.164 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.164 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.164 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.164 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.164 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.164 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.164 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.164 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.164 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.165 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.165 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.165 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.165 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.165 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.165 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.165 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.165 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.165 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.165 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.165 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.165 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.165 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.165 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.166 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.166 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.166 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.166 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.166 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.166 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.166 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.166 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.166 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.166 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.166 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.166 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.166 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.166 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.167 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.167 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.167 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.167 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.167 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.167 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.167 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.167 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.167 INFO:teuthology.orchestra.run.smithi160.stdout: "storage_class": { 2024-03-07T07:43:33.167 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "storage_class", 2024-03-07T07:43:33.167 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.167 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.167 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.167 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "local", 2024-03-07T07:43:33.167 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.168 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.168 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.168 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-07T07:43:33.168 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.168 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.168 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.168 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.168 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.168 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.168 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.168 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "selftest", 2024-03-07T07:43:33.168 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.168 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.169 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.169 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.169 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.169 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.169 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.169 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.169 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.169 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.169 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.169 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.169 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.169 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.169 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.170 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.170 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.170 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.170 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.170 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.170 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.170 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.170 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.170 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.170 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.170 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.170 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.170 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.170 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.171 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.171 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.171 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.171 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.171 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.171 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.171 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.171 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.171 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.171 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.171 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.171 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.171 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.171 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.172 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.172 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.172 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.172 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.172 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.172 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.172 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.172 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.172 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.172 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.172 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.172 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.172 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.173 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.173 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.173 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.173 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.173 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.173 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.173 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.173 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.173 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.173 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.173 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.173 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.173 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.174 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.174 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.174 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.174 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.174 INFO:teuthology.orchestra.run.smithi160.stdout: "roption1": { 2024-03-07T07:43:33.174 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "roption1", 2024-03-07T07:43:33.174 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.174 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.174 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.174 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.174 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.174 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.174 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.174 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.175 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.175 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.175 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.175 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.175 INFO:teuthology.orchestra.run.smithi160.stdout: "roption2": { 2024-03-07T07:43:33.175 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "roption2", 2024-03-07T07:43:33.175 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.175 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.175 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.175 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "xyz", 2024-03-07T07:43:33.175 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.175 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.175 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.176 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.176 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.176 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.176 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.176 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.176 INFO:teuthology.orchestra.run.smithi160.stdout: "rwoption1": { 2024-03-07T07:43:33.176 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "rwoption1", 2024-03-07T07:43:33.176 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.176 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.176 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.176 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.176 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.176 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.176 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.176 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.177 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.177 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.177 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.177 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.177 INFO:teuthology.orchestra.run.smithi160.stdout: "rwoption2": { 2024-03-07T07:43:33.177 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "rwoption2", 2024-03-07T07:43:33.177 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.177 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.177 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.177 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.177 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.177 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.177 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.177 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.177 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.178 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.178 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.178 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.178 INFO:teuthology.orchestra.run.smithi160.stdout: "rwoption3": { 2024-03-07T07:43:33.178 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "rwoption3", 2024-03-07T07:43:33.178 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "float", 2024-03-07T07:43:33.178 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.178 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.178 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.178 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.178 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.178 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.178 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.178 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.178 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.179 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.179 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.179 INFO:teuthology.orchestra.run.smithi160.stdout: "rwoption4": { 2024-03-07T07:43:33.179 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "rwoption4", 2024-03-07T07:43:33.179 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.179 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.179 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.179 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.179 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.179 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.179 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.179 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.179 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.179 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.180 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.180 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.180 INFO:teuthology.orchestra.run.smithi160.stdout: "rwoption5": { 2024-03-07T07:43:33.180 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "rwoption5", 2024-03-07T07:43:33.180 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.180 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.180 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.180 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.180 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.180 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.180 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.180 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.180 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.180 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.181 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.181 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.181 INFO:teuthology.orchestra.run.smithi160.stdout: "rwoption6": { 2024-03-07T07:43:33.181 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "rwoption6", 2024-03-07T07:43:33.181 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.181 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.181 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.181 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.181 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.181 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.181 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.181 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.181 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.181 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.182 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.182 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.182 INFO:teuthology.orchestra.run.smithi160.stdout: "rwoption7": { 2024-03-07T07:43:33.182 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "rwoption7", 2024-03-07T07:43:33.182 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.182 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.182 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.182 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.182 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "1", 2024-03-07T07:43:33.182 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "42", 2024-03-07T07:43:33.182 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.182 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.182 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.182 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.182 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.183 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.183 INFO:teuthology.orchestra.run.smithi160.stdout: "testkey": { 2024-03-07T07:43:33.183 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "testkey", 2024-03-07T07:43:33.183 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.183 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.183 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.183 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.183 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.183 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.183 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.183 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.183 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.183 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.183 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.183 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.184 INFO:teuthology.orchestra.run.smithi160.stdout: "testlkey": { 2024-03-07T07:43:33.184 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "testlkey", 2024-03-07T07:43:33.184 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.184 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.184 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.184 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.184 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.184 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.184 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.184 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.184 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.184 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.184 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.185 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.185 INFO:teuthology.orchestra.run.smithi160.stdout: "testnewline": { 2024-03-07T07:43:33.185 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "testnewline", 2024-03-07T07:43:33.185 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.185 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.185 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.185 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.185 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.185 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.185 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.185 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.185 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.185 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.185 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.185 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.186 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.186 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.186 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.186 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "snap_schedule", 2024-03-07T07:43:33.186 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.186 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.186 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.186 INFO:teuthology.orchestra.run.smithi160.stdout: "allow_m_granularity": { 2024-03-07T07:43:33.186 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "allow_m_granularity", 2024-03-07T07:43:33.186 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.186 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.186 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.186 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.186 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.187 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.187 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.187 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "allow minute scheduled snapshots", 2024-03-07T07:43:33.187 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.187 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.187 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.187 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.187 INFO:teuthology.orchestra.run.smithi160.stdout: "dump_on_update": { 2024-03-07T07:43:33.187 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "dump_on_update", 2024-03-07T07:43:33.187 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.187 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.187 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.187 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.188 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.188 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.188 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.188 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "dump database to debug log on update", 2024-03-07T07:43:33.188 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.188 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.188 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.188 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.188 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.188 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.188 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.188 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.188 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.188 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.189 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.189 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.189 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.189 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.189 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.189 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.189 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.189 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.189 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.189 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.189 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.189 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.189 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.189 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.189 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.190 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.190 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.190 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.190 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.190 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.190 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.190 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.190 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.190 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.190 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.190 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.190 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.190 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.191 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.191 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.191 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.191 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.191 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.191 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.191 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.191 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.191 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.191 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.191 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.191 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.191 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.191 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.192 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.192 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.192 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.192 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.192 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.192 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.192 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.192 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.192 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.192 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.192 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.192 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.192 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.192 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.193 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.193 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.193 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.193 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.193 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.193 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.193 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.193 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.193 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.193 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.193 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.193 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "stats", 2024-03-07T07:43:33.194 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.194 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.194 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.194 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.194 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.194 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.194 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.194 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.194 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.194 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.194 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.194 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.194 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.194 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.195 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.195 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.195 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.195 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.195 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.195 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.195 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.195 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.195 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.195 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.195 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.195 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.195 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.195 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.196 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.196 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.196 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.196 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.196 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.196 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.196 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.196 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.196 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.196 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.196 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.196 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.196 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.196 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.197 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.197 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.197 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.197 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.197 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.197 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.197 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.197 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.197 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.197 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.197 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.197 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.197 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.197 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.197 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.198 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.198 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.198 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.198 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.198 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.198 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.198 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.198 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.198 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.198 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.198 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.198 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.198 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.198 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.199 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.199 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.199 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.199 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.199 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.199 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "status", 2024-03-07T07:43:33.199 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.199 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.199 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.199 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.199 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.199 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.199 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.199 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.200 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.200 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.200 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.200 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.200 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.200 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.200 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.200 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.200 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.200 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.200 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.200 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.200 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.201 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.201 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.201 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.201 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.201 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.201 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.201 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.201 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.201 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.201 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.201 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.201 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.201 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.201 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.202 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.202 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.202 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.202 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.202 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.202 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.202 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.202 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.202 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.202 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.202 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.202 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.202 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.202 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.202 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.203 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.203 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.203 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.203 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.203 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.203 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.203 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.203 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.203 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.203 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.203 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.203 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.203 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.204 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.204 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.204 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.204 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.204 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.204 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.204 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.204 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.204 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.204 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.204 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.204 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.204 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.204 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "telegraf", 2024-03-07T07:43:33.205 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.205 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.205 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.205 INFO:teuthology.orchestra.run.smithi160.stdout: "address": { 2024-03-07T07:43:33.205 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "address", 2024-03-07T07:43:33.205 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.205 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.205 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.205 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-07T07:43:33.205 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.205 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.205 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.205 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.205 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.205 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.206 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.206 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.206 INFO:teuthology.orchestra.run.smithi160.stdout: "interval": { 2024-03-07T07:43:33.206 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "interval", 2024-03-07T07:43:33.206 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:33.206 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.206 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.206 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "15", 2024-03-07T07:43:33.206 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.206 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.206 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.206 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.206 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.206 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.207 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.207 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.207 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.207 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.207 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.207 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.207 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.207 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.207 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.207 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.207 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.207 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.207 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.207 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.208 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.208 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.208 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.208 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.208 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.208 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.208 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.208 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.208 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.208 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.208 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.208 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.208 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.209 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.209 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.209 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.209 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.209 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.209 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.209 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.209 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.209 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.209 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.209 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.209 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.209 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.209 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.210 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.210 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.210 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.210 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.210 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.210 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.210 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.210 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.210 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.210 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.210 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.210 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.210 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.211 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.211 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.211 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.211 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.211 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.211 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.211 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.211 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.211 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.211 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.211 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.211 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.211 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.211 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.212 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.212 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.212 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.212 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.212 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.212 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.212 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.212 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "telemetry", 2024-03-07T07:43:33.212 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.212 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.212 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.212 INFO:teuthology.orchestra.run.smithi160.stdout: "channel_basic": { 2024-03-07T07:43:33.212 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "channel_basic", 2024-03-07T07:43:33.213 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.213 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.213 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.213 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.213 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.213 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.213 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.213 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-07T07:43:33.213 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.213 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.213 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.213 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.213 INFO:teuthology.orchestra.run.smithi160.stdout: "channel_crash": { 2024-03-07T07:43:33.214 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "channel_crash", 2024-03-07T07:43:33.214 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.214 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.214 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.214 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.214 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.214 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.214 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.214 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-07T07:43:33.214 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.214 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.214 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.214 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.214 INFO:teuthology.orchestra.run.smithi160.stdout: "channel_device": { 2024-03-07T07:43:33.215 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "channel_device", 2024-03-07T07:43:33.215 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.215 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.215 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.215 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.215 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.215 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.215 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.215 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-07T07:43:33.215 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.215 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.215 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.215 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.215 INFO:teuthology.orchestra.run.smithi160.stdout: "channel_ident": { 2024-03-07T07:43:33.216 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "channel_ident", 2024-03-07T07:43:33.216 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.216 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.216 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.216 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.216 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.216 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.216 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.216 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-07T07:43:33.216 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.216 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.216 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.216 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.217 INFO:teuthology.orchestra.run.smithi160.stdout: "channel_perf": { 2024-03-07T07:43:33.217 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "channel_perf", 2024-03-07T07:43:33.217 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.217 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.217 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.217 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.217 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.217 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.217 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.217 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-07T07:43:33.217 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.217 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.217 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.217 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.218 INFO:teuthology.orchestra.run.smithi160.stdout: "contact": { 2024-03-07T07:43:33.218 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "contact", 2024-03-07T07:43:33.218 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.218 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.218 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.218 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.218 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.218 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.218 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.218 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.218 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.218 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.218 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.219 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.219 INFO:teuthology.orchestra.run.smithi160.stdout: "description": { 2024-03-07T07:43:33.219 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "description", 2024-03-07T07:43:33.219 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.219 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.219 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.219 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.219 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.219 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.219 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.219 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.219 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.219 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.219 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.219 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.220 INFO:teuthology.orchestra.run.smithi160.stdout: "device_url": { 2024-03-07T07:43:33.220 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "device_url", 2024-03-07T07:43:33.220 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.220 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.220 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.220 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-07T07:43:33.220 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.220 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.220 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.220 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.220 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.220 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.220 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.220 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.221 INFO:teuthology.orchestra.run.smithi160.stdout: "enabled": { 2024-03-07T07:43:33.221 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "enabled", 2024-03-07T07:43:33.221 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.221 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.221 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.221 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.221 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.221 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.221 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.221 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.221 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.221 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.221 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.221 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.221 INFO:teuthology.orchestra.run.smithi160.stdout: "interval": { 2024-03-07T07:43:33.222 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "interval", 2024-03-07T07:43:33.222 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.222 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.222 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.222 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "24", 2024-03-07T07:43:33.222 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "8", 2024-03-07T07:43:33.222 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.222 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.222 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.222 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.222 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.222 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.222 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.222 INFO:teuthology.orchestra.run.smithi160.stdout: "last_opt_revision": { 2024-03-07T07:43:33.222 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "last_opt_revision", 2024-03-07T07:43:33.223 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.223 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.223 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.223 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "1", 2024-03-07T07:43:33.223 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.223 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.223 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.223 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.223 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.223 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.223 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.223 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.223 INFO:teuthology.orchestra.run.smithi160.stdout: "leaderboard": { 2024-03-07T07:43:33.223 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "leaderboard", 2024-03-07T07:43:33.224 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.224 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.224 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.224 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.224 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.224 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.224 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.224 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.224 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.224 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.224 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.224 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.224 INFO:teuthology.orchestra.run.smithi160.stdout: "leaderboard_description": { 2024-03-07T07:43:33.224 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "leaderboard_description", 2024-03-07T07:43:33.225 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.225 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.225 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.225 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.225 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.225 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.225 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.225 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.226 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.227 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.227 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.227 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.227 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.227 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.227 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.227 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.227 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.227 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.227 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.227 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.227 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.227 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.227 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.228 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.228 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.228 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.228 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.228 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.228 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.228 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.228 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.228 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.229 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.229 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.229 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.229 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.229 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.229 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.229 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.229 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.229 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.229 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.229 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.230 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.230 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.230 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.230 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.230 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.230 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.230 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.230 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.230 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.230 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.230 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.231 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.231 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.231 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.231 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.231 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.231 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.231 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.231 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.231 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.231 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.231 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.232 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.232 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.232 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.232 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.232 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.232 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.232 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.232 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.232 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.232 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.232 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.233 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.233 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.233 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.233 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.233 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.233 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.233 INFO:teuthology.orchestra.run.smithi160.stdout: "organization": { 2024-03-07T07:43:33.233 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "organization", 2024-03-07T07:43:33.233 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.233 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.233 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.233 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.234 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.234 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.234 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.234 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.234 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.234 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.234 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.234 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.234 INFO:teuthology.orchestra.run.smithi160.stdout: "proxy": { 2024-03-07T07:43:33.234 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "proxy", 2024-03-07T07:43:33.234 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.234 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.235 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.235 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.235 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.235 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.235 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.235 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.235 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.235 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.235 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.235 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.235 INFO:teuthology.orchestra.run.smithi160.stdout: "url": { 2024-03-07T07:43:33.235 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "url", 2024-03-07T07:43:33.236 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.236 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.236 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.236 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-07T07:43:33.236 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.236 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.236 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.236 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.236 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.236 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.236 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.237 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.237 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.237 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.237 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.237 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "test_orchestrator", 2024-03-07T07:43:33.237 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.237 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.237 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.237 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.237 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.237 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.238 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.238 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.238 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.238 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.238 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.238 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.238 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.238 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.238 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.238 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.238 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.239 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.239 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.239 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.239 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.239 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.239 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.239 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.239 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.239 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.239 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.239 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.240 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.240 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.240 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.240 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.240 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.240 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.240 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.240 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.240 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.240 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.240 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.240 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.241 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.241 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.241 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.241 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.241 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.241 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.241 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.241 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.241 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.241 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.242 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.242 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.242 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.242 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.242 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.242 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.242 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.242 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.242 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.242 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.242 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.242 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.243 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.243 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.243 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.243 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.243 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.243 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.243 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.243 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.243 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.243 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.243 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.243 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.243 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.244 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.244 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "volumes", 2024-03-07T07:43:33.244 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.244 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.244 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.244 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.244 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.244 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.244 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.244 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.244 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.245 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.245 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.245 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.245 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.245 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.245 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.245 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.245 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.245 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.245 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.245 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.245 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.246 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.246 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.246 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.246 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.246 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.246 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.246 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.246 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.246 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.246 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.246 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.246 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.246 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.247 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.247 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.247 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.247 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.247 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.247 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.247 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.247 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.247 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.247 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.247 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.247 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.248 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.248 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.248 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.248 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.248 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.248 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.248 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.248 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.248 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.248 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.248 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.248 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.249 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.249 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.249 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.249 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.249 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.249 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.249 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.249 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.249 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.249 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.249 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.249 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.249 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.250 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.250 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.250 INFO:teuthology.orchestra.run.smithi160.stdout: "max_concurrent_clones": { 2024-03-07T07:43:33.250 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "max_concurrent_clones", 2024-03-07T07:43:33.250 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.250 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.250 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.250 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "4", 2024-03-07T07:43:33.250 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.250 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.251 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.251 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-07T07:43:33.251 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.251 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.251 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.251 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.251 INFO:teuthology.orchestra.run.smithi160.stdout: "periodic_async_work": { 2024-03-07T07:43:33.251 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "periodic_async_work", 2024-03-07T07:43:33.251 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.251 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.251 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.252 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.252 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.252 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.252 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.252 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Periodically check for async work", 2024-03-07T07:43:33.252 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.252 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.252 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.252 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.253 INFO:teuthology.orchestra.run.smithi160.stdout: "snapshot_clone_delay": { 2024-03-07T07:43:33.253 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "snapshot_clone_delay", 2024-03-07T07:43:33.253 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.253 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.253 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.253 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "0", 2024-03-07T07:43:33.253 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.253 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.253 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.253 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-07T07:43:33.253 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.253 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.254 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.254 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.254 INFO:teuthology.orchestra.run.smithi160.stdout: "snapshot_clone_no_wait": { 2024-03-07T07:43:33.254 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "snapshot_clone_no_wait", 2024-03-07T07:43:33.254 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.254 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.254 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.254 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.254 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.254 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.254 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.254 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-07T07:43:33.254 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.255 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.255 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.255 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.255 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.255 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.255 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.255 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "zabbix", 2024-03-07T07:43:33.255 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.255 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.255 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.255 INFO:teuthology.orchestra.run.smithi160.stdout: "discovery_interval": { 2024-03-07T07:43:33.256 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "discovery_interval", 2024-03-07T07:43:33.256 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "uint", 2024-03-07T07:43:33.256 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.256 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.256 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "100", 2024-03-07T07:43:33.256 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.256 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.256 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.256 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.256 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.256 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.257 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.257 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.257 INFO:teuthology.orchestra.run.smithi160.stdout: "identifier": { 2024-03-07T07:43:33.257 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "identifier", 2024-03-07T07:43:33.257 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.257 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.257 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.257 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.257 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.257 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.257 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.257 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.258 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.258 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.258 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.258 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.258 INFO:teuthology.orchestra.run.smithi160.stdout: "interval": { 2024-03-07T07:43:33.258 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "interval", 2024-03-07T07:43:33.258 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:33.258 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.258 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.258 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "60", 2024-03-07T07:43:33.258 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.259 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.259 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.259 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.259 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.259 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.259 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.259 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.259 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.259 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.259 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.259 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.259 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.260 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.260 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.260 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.260 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.260 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.260 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.260 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.260 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.260 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.260 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.261 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.261 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.261 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.261 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.261 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.261 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.261 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.261 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.261 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.261 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.261 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.261 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.262 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.262 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.262 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.262 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.262 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.262 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.262 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.262 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.262 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.262 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.262 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.262 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.263 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.263 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.263 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.263 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.263 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.263 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.263 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.263 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.263 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.263 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.263 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.263 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.263 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.264 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.264 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.264 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.264 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.264 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.264 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.264 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.264 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.264 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.264 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.264 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.264 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.265 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.265 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.265 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.265 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.265 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.265 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.265 INFO:teuthology.orchestra.run.smithi160.stdout: "zabbix_host": { 2024-03-07T07:43:33.265 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "zabbix_host", 2024-03-07T07:43:33.265 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.265 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.265 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.265 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.265 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.266 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.266 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.266 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.266 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.266 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.266 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.266 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.266 INFO:teuthology.orchestra.run.smithi160.stdout: "zabbix_port": { 2024-03-07T07:43:33.266 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "zabbix_port", 2024-03-07T07:43:33.266 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.266 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.266 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.266 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "10051", 2024-03-07T07:43:33.267 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.267 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.267 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.267 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.267 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.267 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.267 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.267 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.267 INFO:teuthology.orchestra.run.smithi160.stdout: "zabbix_sender": { 2024-03-07T07:43:33.267 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "zabbix_sender", 2024-03-07T07:43:33.267 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.267 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.267 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.268 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-07T07:43:33.268 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.268 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.268 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.268 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.268 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.268 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.268 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.268 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.268 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.268 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.268 INFO:teuthology.orchestra.run.smithi160.stdout: ] 2024-03-07T07:43:33.268 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.269 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.269 INFO:teuthology.orchestra.run.smithi160.stdout: "gid": 24119, 2024-03-07T07:43:33.269 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "y", 2024-03-07T07:43:33.269 INFO:teuthology.orchestra.run.smithi160.stdout: "mgr_features": 4540701547738038271, 2024-03-07T07:43:33.269 INFO:teuthology.orchestra.run.smithi160.stdout: "available_modules": [ 2024-03-07T07:43:33.269 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.269 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "alerts", 2024-03-07T07:43:33.269 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.269 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.269 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.269 INFO:teuthology.orchestra.run.smithi160.stdout: "interval": { 2024-03-07T07:43:33.270 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "interval", 2024-03-07T07:43:33.270 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:33.270 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.270 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.270 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "60", 2024-03-07T07:43:33.270 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.270 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.270 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.270 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "How frequently to reexamine health status", 2024-03-07T07:43:33.270 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.270 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.270 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.271 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.271 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.271 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.271 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.271 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.271 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.271 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.271 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.271 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.271 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.271 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.271 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.272 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.272 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.272 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.272 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.272 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.272 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.272 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.272 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.272 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.272 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.273 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.273 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.273 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.273 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.273 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.273 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.273 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.273 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.273 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.273 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.273 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.273 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.274 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.274 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.274 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.274 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.274 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.274 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.274 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.274 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.274 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.274 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.274 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.274 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.274 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.275 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.275 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.275 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.275 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.275 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.275 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.275 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.275 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.275 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.275 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.275 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.275 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.276 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.276 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.276 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.276 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.276 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.276 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.276 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.276 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.276 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.276 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.276 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.276 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.277 INFO:teuthology.orchestra.run.smithi160.stdout: "smtp_destination": { 2024-03-07T07:43:33.277 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "smtp_destination", 2024-03-07T07:43:33.277 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.277 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.277 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.277 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.277 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.277 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.277 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.278 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Email address to send alerts to", 2024-03-07T07:43:33.278 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.278 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.278 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.278 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.278 INFO:teuthology.orchestra.run.smithi160.stdout: "smtp_from_name": { 2024-03-07T07:43:33.278 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "smtp_from_name", 2024-03-07T07:43:33.278 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.278 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.278 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.278 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "Ceph", 2024-03-07T07:43:33.278 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.279 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.279 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.279 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Email From: name", 2024-03-07T07:43:33.279 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.279 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.279 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.279 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.279 INFO:teuthology.orchestra.run.smithi160.stdout: "smtp_host": { 2024-03-07T07:43:33.279 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "smtp_host", 2024-03-07T07:43:33.279 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.279 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.280 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.280 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.280 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.280 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.280 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.280 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "SMTP server", 2024-03-07T07:43:33.280 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.280 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.280 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.280 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.280 INFO:teuthology.orchestra.run.smithi160.stdout: "smtp_password": { 2024-03-07T07:43:33.281 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "smtp_password", 2024-03-07T07:43:33.281 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.281 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.281 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.281 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.281 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.281 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.281 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.281 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Password to authenticate with", 2024-03-07T07:43:33.281 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.281 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.281 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.282 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.282 INFO:teuthology.orchestra.run.smithi160.stdout: "smtp_port": { 2024-03-07T07:43:33.282 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "smtp_port", 2024-03-07T07:43:33.282 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.282 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.282 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.282 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "465", 2024-03-07T07:43:33.282 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.282 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.282 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.282 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "SMTP port", 2024-03-07T07:43:33.282 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.283 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.283 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.283 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.283 INFO:teuthology.orchestra.run.smithi160.stdout: "smtp_sender": { 2024-03-07T07:43:33.283 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "smtp_sender", 2024-03-07T07:43:33.283 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.283 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.283 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.283 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.283 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.283 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.284 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.284 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "SMTP envelope sender", 2024-03-07T07:43:33.284 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.284 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.284 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.284 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.284 INFO:teuthology.orchestra.run.smithi160.stdout: "smtp_ssl": { 2024-03-07T07:43:33.284 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "smtp_ssl", 2024-03-07T07:43:33.284 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.284 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.284 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.284 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.285 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.285 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.285 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.285 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-07T07:43:33.285 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.285 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.285 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.285 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.285 INFO:teuthology.orchestra.run.smithi160.stdout: "smtp_user": { 2024-03-07T07:43:33.285 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "smtp_user", 2024-03-07T07:43:33.285 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.285 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.285 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.286 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.286 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.286 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.286 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.286 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "User to authenticate as", 2024-03-07T07:43:33.286 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.286 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.286 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.286 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.286 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.287 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.287 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.287 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "balancer", 2024-03-07T07:43:33.287 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.287 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.287 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.287 INFO:teuthology.orchestra.run.smithi160.stdout: "active": { 2024-03-07T07:43:33.287 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "active", 2024-03-07T07:43:33.287 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.287 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.287 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.287 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.287 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.288 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.288 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.288 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "automatically balance PGs across cluster", 2024-03-07T07:43:33.288 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.288 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.288 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.288 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.288 INFO:teuthology.orchestra.run.smithi160.stdout: "begin_time": { 2024-03-07T07:43:33.288 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "begin_time", 2024-03-07T07:43:33.288 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.288 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.289 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.289 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "0000", 2024-03-07T07:43:33.289 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.289 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.289 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.289 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "beginning time of day to automatically balance", 2024-03-07T07:43:33.289 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-07T07:43:33.289 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.289 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.289 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.289 INFO:teuthology.orchestra.run.smithi160.stdout: "begin_weekday": { 2024-03-07T07:43:33.289 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "begin_weekday", 2024-03-07T07:43:33.290 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "uint", 2024-03-07T07:43:33.290 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.290 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.290 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "0", 2024-03-07T07:43:33.290 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "0", 2024-03-07T07:43:33.290 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "6", 2024-03-07T07:43:33.290 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.290 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-07T07:43:33.290 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-07T07:43:33.290 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.290 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.290 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.291 INFO:teuthology.orchestra.run.smithi160.stdout: "crush_compat_max_iterations": { 2024-03-07T07:43:33.291 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "crush_compat_max_iterations", 2024-03-07T07:43:33.291 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "uint", 2024-03-07T07:43:33.291 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.291 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.291 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "25", 2024-03-07T07:43:33.291 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "1", 2024-03-07T07:43:33.291 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "250", 2024-03-07T07:43:33.291 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.291 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-07T07:43:33.291 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.291 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.291 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.292 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.292 INFO:teuthology.orchestra.run.smithi160.stdout: "crush_compat_metrics": { 2024-03-07T07:43:33.292 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "crush_compat_metrics", 2024-03-07T07:43:33.292 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.292 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.292 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.292 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "pgs,objects,bytes", 2024-03-07T07:43:33.292 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.292 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.292 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.292 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-07T07:43:33.292 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-07T07:43:33.293 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.293 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.293 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.293 INFO:teuthology.orchestra.run.smithi160.stdout: "crush_compat_step": { 2024-03-07T07:43:33.293 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "crush_compat_step", 2024-03-07T07:43:33.293 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "float", 2024-03-07T07:43:33.293 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.293 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.293 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "0.5", 2024-03-07T07:43:33.293 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "0.001", 2024-03-07T07:43:33.293 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "0.999", 2024-03-07T07:43:33.293 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.293 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "aggressiveness of optimization", 2024-03-07T07:43:33.294 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-07T07:43:33.294 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.294 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.294 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.294 INFO:teuthology.orchestra.run.smithi160.stdout: "end_time": { 2024-03-07T07:43:33.294 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "end_time", 2024-03-07T07:43:33.294 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.294 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.294 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.294 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "2359", 2024-03-07T07:43:33.294 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.294 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.294 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.295 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "ending time of day to automatically balance", 2024-03-07T07:43:33.295 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-07T07:43:33.295 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.295 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.295 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.295 INFO:teuthology.orchestra.run.smithi160.stdout: "end_weekday": { 2024-03-07T07:43:33.295 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "end_weekday", 2024-03-07T07:43:33.295 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "uint", 2024-03-07T07:43:33.295 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.295 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.295 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "0", 2024-03-07T07:43:33.295 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "0", 2024-03-07T07:43:33.296 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "6", 2024-03-07T07:43:33.296 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.296 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-07T07:43:33.296 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-07T07:43:33.296 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.296 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.296 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.296 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.296 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.296 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.296 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.297 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.297 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.297 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.297 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.297 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.297 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.297 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.297 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.297 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.297 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.297 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.297 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.298 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.298 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.298 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.298 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.298 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.298 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.298 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.298 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.298 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.298 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.298 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.298 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.298 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.299 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.299 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.299 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.299 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.299 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.299 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.299 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.299 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.299 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.300 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.300 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.300 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.300 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.300 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.300 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.300 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.300 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.300 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.300 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.300 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.300 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.301 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.301 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.301 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.301 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.301 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.301 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.301 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.301 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.301 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.301 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.302 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.302 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.302 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.302 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.302 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.302 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.302 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.302 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.302 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.302 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.302 INFO:teuthology.orchestra.run.smithi160.stdout: "min_score": { 2024-03-07T07:43:33.302 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "min_score", 2024-03-07T07:43:33.303 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "float", 2024-03-07T07:43:33.303 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.303 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.303 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "0", 2024-03-07T07:43:33.303 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.303 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.303 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.303 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-07T07:43:33.303 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.303 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.303 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.303 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.304 INFO:teuthology.orchestra.run.smithi160.stdout: "mode": { 2024-03-07T07:43:33.304 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "mode", 2024-03-07T07:43:33.304 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.304 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.304 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.304 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "upmap", 2024-03-07T07:43:33.304 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.304 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.304 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.304 INFO:teuthology.orchestra.run.smithi160.stdout: "crush-compat", 2024-03-07T07:43:33.304 INFO:teuthology.orchestra.run.smithi160.stdout: "none", 2024-03-07T07:43:33.304 INFO:teuthology.orchestra.run.smithi160.stdout: "read", 2024-03-07T07:43:33.305 INFO:teuthology.orchestra.run.smithi160.stdout: "upmap", 2024-03-07T07:43:33.305 INFO:teuthology.orchestra.run.smithi160.stdout: "upmap-read" 2024-03-07T07:43:33.305 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.305 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Balancer mode", 2024-03-07T07:43:33.305 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.305 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.305 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.305 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.305 INFO:teuthology.orchestra.run.smithi160.stdout: "pool_ids": { 2024-03-07T07:43:33.305 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "pool_ids", 2024-03-07T07:43:33.305 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.305 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.305 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.306 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.306 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.306 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.306 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.306 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-07T07:43:33.306 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.306 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.306 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.306 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.306 INFO:teuthology.orchestra.run.smithi160.stdout: "sleep_interval": { 2024-03-07T07:43:33.306 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "sleep_interval", 2024-03-07T07:43:33.306 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:33.307 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.307 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.307 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "60", 2024-03-07T07:43:33.307 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.307 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.307 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.307 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-07T07:43:33.307 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.307 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.307 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.307 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.307 INFO:teuthology.orchestra.run.smithi160.stdout: "upmap_max_deviation": { 2024-03-07T07:43:33.308 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "upmap_max_deviation", 2024-03-07T07:43:33.308 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.308 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.308 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.308 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "5", 2024-03-07T07:43:33.308 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "1", 2024-03-07T07:43:33.308 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.308 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.308 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-07T07:43:33.308 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-07T07:43:33.308 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.309 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.309 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.309 INFO:teuthology.orchestra.run.smithi160.stdout: "upmap_max_optimizations": { 2024-03-07T07:43:33.309 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "upmap_max_optimizations", 2024-03-07T07:43:33.309 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "uint", 2024-03-07T07:43:33.309 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.309 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.309 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "10", 2024-03-07T07:43:33.309 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.309 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.309 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.310 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-07T07:43:33.310 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.310 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.310 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.310 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.310 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.310 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.310 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.310 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "cephadm", 2024-03-07T07:43:33.310 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.310 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.310 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.310 INFO:teuthology.orchestra.run.smithi160.stdout: "agent_down_multiplier": { 2024-03-07T07:43:33.311 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "agent_down_multiplier", 2024-03-07T07:43:33.311 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "float", 2024-03-07T07:43:33.311 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.311 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.311 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "3.0", 2024-03-07T07:43:33.311 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.311 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.311 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.311 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-07T07:43:33.311 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.311 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.311 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.312 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.312 INFO:teuthology.orchestra.run.smithi160.stdout: "agent_refresh_rate": { 2024-03-07T07:43:33.312 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "agent_refresh_rate", 2024-03-07T07:43:33.312 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:33.312 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.312 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.312 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "20", 2024-03-07T07:43:33.312 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.312 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.312 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.312 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-07T07:43:33.313 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.313 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.313 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.313 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.313 INFO:teuthology.orchestra.run.smithi160.stdout: "agent_starting_port": { 2024-03-07T07:43:33.313 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "agent_starting_port", 2024-03-07T07:43:33.313 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.313 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.313 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.313 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "4721", 2024-03-07T07:43:33.313 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.313 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.313 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.313 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-07T07:43:33.314 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.314 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.314 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.314 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.314 INFO:teuthology.orchestra.run.smithi160.stdout: "allow_ptrace": { 2024-03-07T07:43:33.314 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "allow_ptrace", 2024-03-07T07:43:33.314 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.314 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.314 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.314 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.314 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.314 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.314 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.315 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-07T07:43:33.315 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-07T07:43:33.315 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.315 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.315 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.315 INFO:teuthology.orchestra.run.smithi160.stdout: "autotune_interval": { 2024-03-07T07:43:33.315 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "autotune_interval", 2024-03-07T07:43:33.315 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:33.315 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.315 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.315 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "600", 2024-03-07T07:43:33.315 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.316 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.316 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.316 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-07T07:43:33.316 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.316 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.316 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.316 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.316 INFO:teuthology.orchestra.run.smithi160.stdout: "autotune_memory_target_ratio": { 2024-03-07T07:43:33.316 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "autotune_memory_target_ratio", 2024-03-07T07:43:33.316 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "float", 2024-03-07T07:43:33.316 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.316 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.317 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "0.7", 2024-03-07T07:43:33.317 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.317 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.317 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.317 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-07T07:43:33.317 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.317 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.317 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.318 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.318 INFO:teuthology.orchestra.run.smithi160.stdout: "cephadm_log_destination": { 2024-03-07T07:43:33.318 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "cephadm_log_destination", 2024-03-07T07:43:33.318 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.318 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.318 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.318 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.318 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.318 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.318 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.318 INFO:teuthology.orchestra.run.smithi160.stdout: "file", 2024-03-07T07:43:33.318 INFO:teuthology.orchestra.run.smithi160.stdout: "file,syslog", 2024-03-07T07:43:33.319 INFO:teuthology.orchestra.run.smithi160.stdout: "syslog" 2024-03-07T07:43:33.319 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.319 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-07T07:43:33.319 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.319 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.319 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.319 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.319 INFO:teuthology.orchestra.run.smithi160.stdout: "cgroups_split": { 2024-03-07T07:43:33.319 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "cgroups_split", 2024-03-07T07:43:33.319 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.320 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.320 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.320 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.320 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.320 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.320 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.320 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-07T07:43:33.320 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.320 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.320 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.320 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.320 INFO:teuthology.orchestra.run.smithi160.stdout: "config_checks_enabled": { 2024-03-07T07:43:33.321 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "config_checks_enabled", 2024-03-07T07:43:33.321 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.321 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.321 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.321 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.321 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.321 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.321 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.321 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-07T07:43:33.321 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.322 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.322 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.322 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.322 INFO:teuthology.orchestra.run.smithi160.stdout: "config_dashboard": { 2024-03-07T07:43:33.322 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "config_dashboard", 2024-03-07T07:43:33.322 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.322 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.322 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.322 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.322 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.323 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.323 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.323 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-07T07:43:33.323 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.323 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.323 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.323 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.323 INFO:teuthology.orchestra.run.smithi160.stdout: "container_image_alertmanager": { 2024-03-07T07:43:33.323 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "container_image_alertmanager", 2024-03-07T07:43:33.323 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.323 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.323 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.323 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-07T07:43:33.324 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.324 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.324 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.324 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Prometheus container image", 2024-03-07T07:43:33.324 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.324 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.324 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.324 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.324 INFO:teuthology.orchestra.run.smithi160.stdout: "container_image_base": { 2024-03-07T07:43:33.324 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "container_image_base", 2024-03-07T07:43:33.324 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.325 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.325 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.325 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-07T07:43:33.325 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.325 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.325 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.325 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Container image name, without the tag", 2024-03-07T07:43:33.325 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.325 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.325 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.325 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.325 INFO:teuthology.orchestra.run.smithi160.stdout: "container_image_elasticsearch": { 2024-03-07T07:43:33.325 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "container_image_elasticsearch", 2024-03-07T07:43:33.325 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.326 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.326 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.326 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-07T07:43:33.326 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.326 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.326 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.326 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "elasticsearch container image", 2024-03-07T07:43:33.326 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.326 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.326 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.326 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.326 INFO:teuthology.orchestra.run.smithi160.stdout: "container_image_grafana": { 2024-03-07T07:43:33.326 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "container_image_grafana", 2024-03-07T07:43:33.327 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.327 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.327 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.327 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-07T07:43:33.327 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.327 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.327 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.327 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Prometheus container image", 2024-03-07T07:43:33.327 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.327 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.327 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.327 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.328 INFO:teuthology.orchestra.run.smithi160.stdout: "container_image_haproxy": { 2024-03-07T07:43:33.328 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "container_image_haproxy", 2024-03-07T07:43:33.328 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.328 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.328 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.328 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-07T07:43:33.328 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.328 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.328 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.328 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "HAproxy container image", 2024-03-07T07:43:33.328 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.328 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.329 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.329 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.329 INFO:teuthology.orchestra.run.smithi160.stdout: "container_image_jaeger_agent": { 2024-03-07T07:43:33.329 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "container_image_jaeger_agent", 2024-03-07T07:43:33.329 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.329 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.329 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.329 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-07T07:43:33.329 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.329 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.329 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.329 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Jaeger agent container image", 2024-03-07T07:43:33.329 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.330 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.330 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.330 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.330 INFO:teuthology.orchestra.run.smithi160.stdout: "container_image_jaeger_collector": { 2024-03-07T07:43:33.330 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "container_image_jaeger_collector", 2024-03-07T07:43:33.330 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.330 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.330 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.330 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-07T07:43:33.330 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.330 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.331 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.331 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Jaeger collector container image", 2024-03-07T07:43:33.331 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.331 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.331 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.331 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.331 INFO:teuthology.orchestra.run.smithi160.stdout: "container_image_jaeger_query": { 2024-03-07T07:43:33.331 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "container_image_jaeger_query", 2024-03-07T07:43:33.331 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.331 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.331 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.331 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-07T07:43:33.332 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.332 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.332 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.332 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Jaeger query container image", 2024-03-07T07:43:33.332 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.332 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.332 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.332 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.332 INFO:teuthology.orchestra.run.smithi160.stdout: "container_image_keepalived": { 2024-03-07T07:43:33.332 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "container_image_keepalived", 2024-03-07T07:43:33.332 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.332 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.333 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.333 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-07T07:43:33.333 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.333 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.333 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.333 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Keepalived container image", 2024-03-07T07:43:33.333 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.333 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.333 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.333 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.333 INFO:teuthology.orchestra.run.smithi160.stdout: "container_image_loki": { 2024-03-07T07:43:33.333 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "container_image_loki", 2024-03-07T07:43:33.334 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.334 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.334 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.334 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-07T07:43:33.334 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.334 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.334 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.334 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Loki container image", 2024-03-07T07:43:33.334 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.334 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.334 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.334 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.334 INFO:teuthology.orchestra.run.smithi160.stdout: "container_image_node_exporter": { 2024-03-07T07:43:33.335 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "container_image_node_exporter", 2024-03-07T07:43:33.335 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.335 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.335 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.335 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-07T07:43:33.335 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.335 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.335 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.335 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Prometheus container image", 2024-03-07T07:43:33.335 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.335 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.335 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.336 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.336 INFO:teuthology.orchestra.run.smithi160.stdout: "container_image_nvmeof": { 2024-03-07T07:43:33.336 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "container_image_nvmeof", 2024-03-07T07:43:33.336 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.336 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.336 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.336 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-07T07:43:33.336 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.336 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.336 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.336 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Nvme-of container image", 2024-03-07T07:43:33.337 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.337 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.337 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.337 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.337 INFO:teuthology.orchestra.run.smithi160.stdout: "container_image_prometheus": { 2024-03-07T07:43:33.337 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "container_image_prometheus", 2024-03-07T07:43:33.337 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.337 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.337 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.337 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-07T07:43:33.337 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.337 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.338 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.338 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Prometheus container image", 2024-03-07T07:43:33.338 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.338 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.338 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.338 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.338 INFO:teuthology.orchestra.run.smithi160.stdout: "container_image_promtail": { 2024-03-07T07:43:33.338 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "container_image_promtail", 2024-03-07T07:43:33.338 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.338 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.338 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.339 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-07T07:43:33.339 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.339 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.339 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.339 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Promtail container image", 2024-03-07T07:43:33.339 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.339 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.339 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.339 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.339 INFO:teuthology.orchestra.run.smithi160.stdout: "container_image_snmp_gateway": { 2024-03-07T07:43:33.339 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "container_image_snmp_gateway", 2024-03-07T07:43:33.339 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.339 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.340 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.340 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-07T07:43:33.340 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.340 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.340 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.340 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "SNMP Gateway container image", 2024-03-07T07:43:33.340 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.340 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.340 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.340 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.340 INFO:teuthology.orchestra.run.smithi160.stdout: "container_init": { 2024-03-07T07:43:33.341 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "container_init", 2024-03-07T07:43:33.341 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.341 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.341 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.341 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.341 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.341 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.341 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.341 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Run podman/docker with `--init`", 2024-03-07T07:43:33.341 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.341 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.341 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.342 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.342 INFO:teuthology.orchestra.run.smithi160.stdout: "daemon_cache_timeout": { 2024-03-07T07:43:33.342 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "daemon_cache_timeout", 2024-03-07T07:43:33.342 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:33.342 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.342 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.342 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "600", 2024-03-07T07:43:33.342 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.342 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.342 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.342 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-07T07:43:33.342 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.343 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.343 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.343 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.343 INFO:teuthology.orchestra.run.smithi160.stdout: "default_cephadm_command_timeout": { 2024-03-07T07:43:33.343 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "default_cephadm_command_timeout", 2024-03-07T07:43:33.343 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.343 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.343 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.343 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "900", 2024-03-07T07:43:33.343 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.343 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.343 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.344 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-07T07:43:33.344 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.344 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.344 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.344 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.344 INFO:teuthology.orchestra.run.smithi160.stdout: "default_registry": { 2024-03-07T07:43:33.344 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "default_registry", 2024-03-07T07:43:33.344 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.344 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.344 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.344 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "docker.io", 2024-03-07T07:43:33.345 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.345 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.345 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.345 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-07T07:43:33.345 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.345 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.345 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.345 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.345 INFO:teuthology.orchestra.run.smithi160.stdout: "device_cache_timeout": { 2024-03-07T07:43:33.345 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "device_cache_timeout", 2024-03-07T07:43:33.345 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:33.345 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.346 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.346 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "1800", 2024-03-07T07:43:33.346 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.346 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.346 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.346 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "seconds to cache device inventory", 2024-03-07T07:43:33.346 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.346 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.346 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.346 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.346 INFO:teuthology.orchestra.run.smithi160.stdout: "device_enhanced_scan": { 2024-03-07T07:43:33.347 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "device_enhanced_scan", 2024-03-07T07:43:33.347 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.347 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.347 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.347 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.347 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.347 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.347 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.347 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-07T07:43:33.347 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.347 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.347 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.348 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.348 INFO:teuthology.orchestra.run.smithi160.stdout: "facts_cache_timeout": { 2024-03-07T07:43:33.348 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "facts_cache_timeout", 2024-03-07T07:43:33.348 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:33.348 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.348 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.348 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "60", 2024-03-07T07:43:33.348 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.348 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.348 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.349 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "seconds to cache host facts data", 2024-03-07T07:43:33.349 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.349 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.349 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.349 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.349 INFO:teuthology.orchestra.run.smithi160.stdout: "host_check_interval": { 2024-03-07T07:43:33.349 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "host_check_interval", 2024-03-07T07:43:33.349 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:33.349 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.349 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.349 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "600", 2024-03-07T07:43:33.349 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.350 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.350 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.350 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "how frequently to perform a host check", 2024-03-07T07:43:33.350 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.350 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.350 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.350 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.350 INFO:teuthology.orchestra.run.smithi160.stdout: "hw_monitoring": { 2024-03-07T07:43:33.350 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "hw_monitoring", 2024-03-07T07:43:33.350 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.351 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.351 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.351 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.351 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.351 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.351 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.351 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-07T07:43:33.351 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.351 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.351 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.351 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.351 INFO:teuthology.orchestra.run.smithi160.stdout: "inventory_list_all": { 2024-03-07T07:43:33.352 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "inventory_list_all", 2024-03-07T07:43:33.352 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.352 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.352 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.352 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.352 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.352 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.352 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.352 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-07T07:43:33.352 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.352 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.353 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.353 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.353 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.353 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.353 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.353 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.353 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.353 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.353 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.353 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.353 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.353 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.354 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.354 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.354 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.354 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.354 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.354 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.354 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.354 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.354 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.354 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.354 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.354 INFO:teuthology.orchestra.run.smithi160.stdout: "log_refresh_metadata": { 2024-03-07T07:43:33.355 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_refresh_metadata", 2024-03-07T07:43:33.355 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.355 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.355 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.355 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.355 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.355 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.355 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.355 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-07T07:43:33.355 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.355 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.355 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.356 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.356 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.356 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.356 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.356 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.356 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.356 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.356 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.356 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.356 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.356 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-07T07:43:33.357 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.357 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.357 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.357 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.357 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.357 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.357 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.357 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.357 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.357 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.357 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.358 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.358 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.358 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.358 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.358 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.358 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.358 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.358 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.358 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.358 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.358 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.358 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.358 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.359 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.359 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.359 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.359 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.359 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.359 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.359 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.359 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.359 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.359 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.359 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.359 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.359 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.360 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.360 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.360 INFO:teuthology.orchestra.run.smithi160.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-07T07:43:33.360 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-07T07:43:33.360 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.360 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.360 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.360 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.360 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.360 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.360 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.360 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-07T07:43:33.361 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.361 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.361 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.361 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.361 INFO:teuthology.orchestra.run.smithi160.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-07T07:43:33.361 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-07T07:43:33.361 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.361 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.361 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.361 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "*", 2024-03-07T07:43:33.361 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.362 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.362 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.362 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-07T07:43:33.362 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.362 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.362 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.362 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.362 INFO:teuthology.orchestra.run.smithi160.stdout: "max_count_per_host": { 2024-03-07T07:43:33.362 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "max_count_per_host", 2024-03-07T07:43:33.362 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.362 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.363 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.363 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "10", 2024-03-07T07:43:33.363 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.363 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.363 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.363 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "max number of daemons per service per host", 2024-03-07T07:43:33.363 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.363 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.363 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.363 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.363 INFO:teuthology.orchestra.run.smithi160.stdout: "max_osd_draining_count": { 2024-03-07T07:43:33.363 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "max_osd_draining_count", 2024-03-07T07:43:33.364 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.364 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.364 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.364 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "10", 2024-03-07T07:43:33.364 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.364 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.364 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.364 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-07T07:43:33.364 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.364 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.364 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.364 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.365 INFO:teuthology.orchestra.run.smithi160.stdout: "migration_current": { 2024-03-07T07:43:33.365 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "migration_current", 2024-03-07T07:43:33.365 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.365 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.365 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.365 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.365 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.365 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.365 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.365 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "internal - do not modify", 2024-03-07T07:43:33.365 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.365 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.366 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.366 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.366 INFO:teuthology.orchestra.run.smithi160.stdout: "mode": { 2024-03-07T07:43:33.366 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "mode", 2024-03-07T07:43:33.366 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.366 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.366 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.366 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "root", 2024-03-07T07:43:33.366 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.366 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.366 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.366 INFO:teuthology.orchestra.run.smithi160.stdout: "cephadm-package", 2024-03-07T07:43:33.367 INFO:teuthology.orchestra.run.smithi160.stdout: "root" 2024-03-07T07:43:33.367 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.367 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "mode for remote execution of cephadm", 2024-03-07T07:43:33.367 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.367 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.367 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.367 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.367 INFO:teuthology.orchestra.run.smithi160.stdout: "oob_default_addr": { 2024-03-07T07:43:33.367 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "oob_default_addr", 2024-03-07T07:43:33.367 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.367 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.367 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.367 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "169.254.1.1", 2024-03-07T07:43:33.367 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.368 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.368 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.368 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-07T07:43:33.368 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.368 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.368 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.368 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.368 INFO:teuthology.orchestra.run.smithi160.stdout: "prometheus_alerts_path": { 2024-03-07T07:43:33.368 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "prometheus_alerts_path", 2024-03-07T07:43:33.368 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.369 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.369 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.369 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-07T07:43:33.369 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.369 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.369 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.369 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-07T07:43:33.369 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.369 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.369 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.369 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.369 INFO:teuthology.orchestra.run.smithi160.stdout: "registry_insecure": { 2024-03-07T07:43:33.369 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "registry_insecure", 2024-03-07T07:43:33.370 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.370 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.370 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.370 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.370 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.370 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.370 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.370 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-07T07:43:33.370 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.370 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.370 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.370 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.371 INFO:teuthology.orchestra.run.smithi160.stdout: "registry_password": { 2024-03-07T07:43:33.371 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "registry_password", 2024-03-07T07:43:33.371 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.371 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.371 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.371 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.371 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.371 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.371 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.371 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-07T07:43:33.372 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.372 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.372 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.372 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.372 INFO:teuthology.orchestra.run.smithi160.stdout: "registry_url": { 2024-03-07T07:43:33.372 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "registry_url", 2024-03-07T07:43:33.372 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.372 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.372 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.372 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.372 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.373 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.373 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.373 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-07T07:43:33.373 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.373 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.373 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.373 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.373 INFO:teuthology.orchestra.run.smithi160.stdout: "registry_username": { 2024-03-07T07:43:33.373 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "registry_username", 2024-03-07T07:43:33.373 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.373 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.373 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.374 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.374 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.374 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.374 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.374 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-07T07:43:33.374 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.374 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.374 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.374 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.374 INFO:teuthology.orchestra.run.smithi160.stdout: "secure_monitoring_stack": { 2024-03-07T07:43:33.374 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "secure_monitoring_stack", 2024-03-07T07:43:33.374 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.375 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.375 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.375 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.375 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.375 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.375 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.375 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-07T07:43:33.375 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.375 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.375 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.375 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.376 INFO:teuthology.orchestra.run.smithi160.stdout: "service_discovery_port": { 2024-03-07T07:43:33.376 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "service_discovery_port", 2024-03-07T07:43:33.376 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.376 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.376 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.376 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "8765", 2024-03-07T07:43:33.376 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.376 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.376 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.376 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "cephadm service discovery port", 2024-03-07T07:43:33.376 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.376 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.377 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.377 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.377 INFO:teuthology.orchestra.run.smithi160.stdout: "ssh_config_file": { 2024-03-07T07:43:33.377 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "ssh_config_file", 2024-03-07T07:43:33.377 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.377 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.377 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.377 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.377 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.377 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.377 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.377 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-07T07:43:33.378 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.378 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.378 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.378 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.378 INFO:teuthology.orchestra.run.smithi160.stdout: "use_agent": { 2024-03-07T07:43:33.378 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "use_agent", 2024-03-07T07:43:33.378 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.378 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.378 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.378 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.378 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.378 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.379 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.379 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-07T07:43:33.379 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.379 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.379 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.379 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.379 INFO:teuthology.orchestra.run.smithi160.stdout: "use_repo_digest": { 2024-03-07T07:43:33.379 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "use_repo_digest", 2024-03-07T07:43:33.379 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.379 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.379 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.380 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.380 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.380 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.380 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.380 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-07T07:43:33.380 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.380 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.380 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.380 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.380 INFO:teuthology.orchestra.run.smithi160.stdout: "warn_on_failed_host_check": { 2024-03-07T07:43:33.380 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "warn_on_failed_host_check", 2024-03-07T07:43:33.381 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.381 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.381 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.381 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.381 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.381 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.381 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.381 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "raise a health warning if the host check fails", 2024-03-07T07:43:33.381 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.381 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.381 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.381 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.382 INFO:teuthology.orchestra.run.smithi160.stdout: "warn_on_stray_daemons": { 2024-03-07T07:43:33.382 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "warn_on_stray_daemons", 2024-03-07T07:43:33.382 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.382 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.382 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.382 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.382 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.382 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.382 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.382 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-07T07:43:33.382 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.383 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.383 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.383 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.383 INFO:teuthology.orchestra.run.smithi160.stdout: "warn_on_stray_hosts": { 2024-03-07T07:43:33.383 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "warn_on_stray_hosts", 2024-03-07T07:43:33.383 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.383 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.383 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.383 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.383 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.384 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.384 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.384 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-07T07:43:33.384 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.384 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.384 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.384 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.384 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.384 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.384 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.384 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "crash", 2024-03-07T07:43:33.384 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.385 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.385 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.385 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.385 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.385 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.385 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.385 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.385 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.385 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.385 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.385 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.385 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.385 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.386 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.386 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.386 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.386 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.386 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.386 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.386 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.386 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.386 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.386 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.386 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.386 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.387 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.387 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.387 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.387 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.387 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.387 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.387 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.387 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.387 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.387 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.387 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.387 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.387 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.388 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.388 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.388 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.388 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.388 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.388 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.388 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.388 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.388 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.388 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.388 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.388 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.389 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.389 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.389 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.389 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.389 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.389 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.389 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.389 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.389 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.389 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.389 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.389 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.390 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.390 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.390 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.390 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.390 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.390 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.390 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.390 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.390 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.390 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.391 INFO:teuthology.orchestra.run.smithi160.stdout: "retain_interval": { 2024-03-07T07:43:33.391 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "retain_interval", 2024-03-07T07:43:33.391 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:33.391 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.391 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.391 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "31536000", 2024-03-07T07:43:33.391 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.391 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.391 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.391 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-07T07:43:33.391 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.392 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.392 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.392 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.392 INFO:teuthology.orchestra.run.smithi160.stdout: "warn_recent_interval": { 2024-03-07T07:43:33.392 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "warn_recent_interval", 2024-03-07T07:43:33.392 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:33.392 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.392 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.392 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "1209600", 2024-03-07T07:43:33.392 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.393 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.393 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.393 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-07T07:43:33.393 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.393 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.393 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.393 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.393 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.393 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.393 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.393 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "dashboard", 2024-03-07T07:43:33.393 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.394 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.394 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.394 INFO:teuthology.orchestra.run.smithi160.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-07T07:43:33.394 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-07T07:43:33.394 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.394 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.394 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.394 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "10", 2024-03-07T07:43:33.394 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.394 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.394 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.394 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.395 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.395 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.395 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.395 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.395 INFO:teuthology.orchestra.run.smithi160.stdout: "ALERTMANAGER_API_HOST": { 2024-03-07T07:43:33.395 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-07T07:43:33.395 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.395 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.395 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.395 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.395 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.395 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.396 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.396 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.396 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.396 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.396 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.396 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.396 INFO:teuthology.orchestra.run.smithi160.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-07T07:43:33.396 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-07T07:43:33.396 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.396 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.396 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.397 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.397 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.397 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.397 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.397 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.397 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.397 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.397 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.397 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.397 INFO:teuthology.orchestra.run.smithi160.stdout: "AUDIT_API_ENABLED": { 2024-03-07T07:43:33.397 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "AUDIT_API_ENABLED", 2024-03-07T07:43:33.397 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.398 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.398 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.398 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.398 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.398 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.398 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.398 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.398 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.398 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.398 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.398 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.398 INFO:teuthology.orchestra.run.smithi160.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-07T07:43:33.398 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-07T07:43:33.399 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.399 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.399 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.399 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.399 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.399 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.399 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.399 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.399 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.399 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.399 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.399 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.400 INFO:teuthology.orchestra.run.smithi160.stdout: "ENABLE_BROWSABLE_API": { 2024-03-07T07:43:33.400 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-07T07:43:33.400 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.400 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.400 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.400 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.400 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.400 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.400 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.400 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.400 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.401 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.401 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.401 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.401 INFO:teuthology.orchestra.run.smithi160.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-07T07:43:33.401 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-07T07:43:33.401 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.401 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.401 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.401 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.401 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.402 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.402 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.402 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.402 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.402 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.402 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.402 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.402 INFO:teuthology.orchestra.run.smithi160.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-07T07:43:33.402 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-07T07:43:33.402 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.402 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.402 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.402 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.403 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.403 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.403 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.403 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.403 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.403 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.403 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.403 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.403 INFO:teuthology.orchestra.run.smithi160.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-07T07:43:33.403 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-07T07:43:33.403 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.403 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.403 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.404 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.404 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.404 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.404 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.404 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.404 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.404 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.404 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.404 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.404 INFO:teuthology.orchestra.run.smithi160.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-07T07:43:33.404 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-07T07:43:33.404 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.405 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.405 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.405 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.405 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.405 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.405 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.405 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.405 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.405 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.405 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.405 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.406 INFO:teuthology.orchestra.run.smithi160.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-07T07:43:33.406 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-07T07:43:33.406 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.406 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.406 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.406 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.406 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.406 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.406 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.406 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.406 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.406 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.406 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.406 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.407 INFO:teuthology.orchestra.run.smithi160.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-07T07:43:33.407 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-07T07:43:33.407 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.407 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.407 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.407 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.407 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.407 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.407 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.407 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.407 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.407 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.408 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.408 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.408 INFO:teuthology.orchestra.run.smithi160.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-07T07:43:33.408 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-07T07:43:33.408 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.408 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.408 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.408 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.408 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.408 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.408 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.408 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.409 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.409 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.409 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.409 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.409 INFO:teuthology.orchestra.run.smithi160.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-07T07:43:33.409 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-07T07:43:33.409 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.409 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.409 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.409 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.409 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.409 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.410 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.410 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.410 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.410 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.410 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.410 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.410 INFO:teuthology.orchestra.run.smithi160.stdout: "GRAFANA_API_PASSWORD": { 2024-03-07T07:43:33.410 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-07T07:43:33.410 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.410 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.410 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.410 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "admin", 2024-03-07T07:43:33.411 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.411 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.411 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.411 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.411 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.411 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.411 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.411 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.411 INFO:teuthology.orchestra.run.smithi160.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-07T07:43:33.411 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-07T07:43:33.411 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.412 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.412 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.412 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.412 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.412 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.412 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.412 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.412 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.412 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.412 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.412 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.412 INFO:teuthology.orchestra.run.smithi160.stdout: "GRAFANA_API_URL": { 2024-03-07T07:43:33.413 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "GRAFANA_API_URL", 2024-03-07T07:43:33.413 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.413 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.413 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.413 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.413 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.413 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.413 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.413 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.413 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.413 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.413 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.414 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.414 INFO:teuthology.orchestra.run.smithi160.stdout: "GRAFANA_API_USERNAME": { 2024-03-07T07:43:33.414 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-07T07:43:33.414 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.414 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.414 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.414 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "admin", 2024-03-07T07:43:33.414 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.414 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.414 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.414 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.414 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.415 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.415 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.415 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.415 INFO:teuthology.orchestra.run.smithi160.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-07T07:43:33.415 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-07T07:43:33.415 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.415 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.415 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.415 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.415 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.415 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.415 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.416 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.416 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.416 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.416 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.416 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.416 INFO:teuthology.orchestra.run.smithi160.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-07T07:43:33.416 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-07T07:43:33.416 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.416 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.416 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.416 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.416 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.417 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.417 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.417 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.417 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.417 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.417 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.417 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.417 INFO:teuthology.orchestra.run.smithi160.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-07T07:43:33.417 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-07T07:43:33.417 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.417 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.417 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.417 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.418 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.418 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.418 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.418 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.418 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.418 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.418 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.418 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.418 INFO:teuthology.orchestra.run.smithi160.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-07T07:43:33.418 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-07T07:43:33.418 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.418 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.419 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.419 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.419 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.419 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.419 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.419 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.419 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.419 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.419 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.419 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.419 INFO:teuthology.orchestra.run.smithi160.stdout: "PROMETHEUS_API_HOST": { 2024-03-07T07:43:33.419 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-07T07:43:33.419 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.420 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.420 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.420 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.420 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.420 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.420 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.420 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.420 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.420 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.420 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.420 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.420 INFO:teuthology.orchestra.run.smithi160.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-07T07:43:33.421 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-07T07:43:33.421 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.421 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.421 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.421 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.421 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.421 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.421 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.421 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.421 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.421 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.421 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.421 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.422 INFO:teuthology.orchestra.run.smithi160.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-07T07:43:33.422 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-07T07:43:33.422 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.422 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.422 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.422 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.422 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.422 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.422 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.422 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.422 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.422 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.423 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.423 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.423 INFO:teuthology.orchestra.run.smithi160.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-07T07:43:33.423 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-07T07:43:33.423 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.423 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.423 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.423 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.423 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.423 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.423 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.423 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.424 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.424 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.424 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.424 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.424 INFO:teuthology.orchestra.run.smithi160.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-07T07:43:33.424 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-07T07:43:33.424 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.424 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.424 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.424 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.424 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.425 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.425 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.425 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.425 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.425 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.425 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.425 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.425 INFO:teuthology.orchestra.run.smithi160.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-07T07:43:33.425 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-07T07:43:33.425 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.425 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.425 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.426 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.426 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.426 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.426 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.426 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.426 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.426 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.426 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.426 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.426 INFO:teuthology.orchestra.run.smithi160.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-07T07:43:33.426 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-07T07:43:33.427 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.427 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.427 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.427 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.427 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.427 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.427 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.427 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.427 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.427 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.427 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.427 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.427 INFO:teuthology.orchestra.run.smithi160.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-07T07:43:33.428 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-07T07:43:33.428 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.428 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.428 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.428 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.428 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.428 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.428 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.428 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.428 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.428 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.429 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.429 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.429 INFO:teuthology.orchestra.run.smithi160.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-07T07:43:33.429 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-07T07:43:33.429 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.429 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.429 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.429 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.429 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.429 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.429 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.429 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.430 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.430 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.430 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.430 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.430 INFO:teuthology.orchestra.run.smithi160.stdout: "PWD_POLICY_ENABLED": { 2024-03-07T07:43:33.430 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-07T07:43:33.430 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.430 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.430 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.430 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.430 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.431 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.431 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.431 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.431 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.431 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.431 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.431 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.431 INFO:teuthology.orchestra.run.smithi160.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-07T07:43:33.431 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-07T07:43:33.431 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.431 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.431 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.432 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-07T07:43:33.432 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.432 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.432 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.432 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.432 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.432 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.432 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.432 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.432 INFO:teuthology.orchestra.run.smithi160.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-07T07:43:33.432 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-07T07:43:33.432 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.433 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.433 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.433 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "10", 2024-03-07T07:43:33.433 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.433 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.433 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.433 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.435 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.435 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.435 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.435 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.435 INFO:teuthology.orchestra.run.smithi160.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-07T07:43:33.435 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-07T07:43:33.435 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.435 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.435 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.435 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "8", 2024-03-07T07:43:33.435 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.436 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.436 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.436 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.436 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.436 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.436 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.436 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.436 INFO:teuthology.orchestra.run.smithi160.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-07T07:43:33.436 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-07T07:43:33.436 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.436 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.437 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.437 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "45", 2024-03-07T07:43:33.437 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.437 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.437 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.437 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.437 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.437 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.437 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.437 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.437 INFO:teuthology.orchestra.run.smithi160.stdout: "RGW_API_ACCESS_KEY": { 2024-03-07T07:43:33.438 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-07T07:43:33.438 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.438 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.438 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.438 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.438 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.438 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.438 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.438 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.438 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.439 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.439 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.439 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.439 INFO:teuthology.orchestra.run.smithi160.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-07T07:43:33.439 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-07T07:43:33.439 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.439 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.439 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.439 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "admin", 2024-03-07T07:43:33.439 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.439 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.440 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.440 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.440 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.440 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.440 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.440 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.440 INFO:teuthology.orchestra.run.smithi160.stdout: "RGW_API_SECRET_KEY": { 2024-03-07T07:43:33.440 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-07T07:43:33.440 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.440 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.440 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.440 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.441 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.441 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.441 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.441 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.441 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.441 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.441 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.441 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.441 INFO:teuthology.orchestra.run.smithi160.stdout: "RGW_API_SSL_VERIFY": { 2024-03-07T07:43:33.441 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-07T07:43:33.441 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.441 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.442 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.442 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.442 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.442 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.442 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.442 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.442 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.442 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.442 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.442 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.442 INFO:teuthology.orchestra.run.smithi160.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-07T07:43:33.442 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-07T07:43:33.443 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.443 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.443 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.443 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "0", 2024-03-07T07:43:33.443 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.443 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.443 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.443 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.443 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.443 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.443 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.443 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.444 INFO:teuthology.orchestra.run.smithi160.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-07T07:43:33.444 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-07T07:43:33.444 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.444 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.444 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.444 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "10", 2024-03-07T07:43:33.444 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.444 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.444 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.444 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.444 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.444 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.445 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.445 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.445 INFO:teuthology.orchestra.run.smithi160.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-07T07:43:33.445 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-07T07:43:33.445 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.445 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.445 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.445 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "5", 2024-03-07T07:43:33.445 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.445 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.445 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.445 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.445 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.446 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.446 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.446 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.446 INFO:teuthology.orchestra.run.smithi160.stdout: "cross_origin_url": { 2024-03-07T07:43:33.446 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "cross_origin_url", 2024-03-07T07:43:33.446 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.446 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.446 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.446 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.446 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.446 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.446 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.447 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.447 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.447 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.447 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.447 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.447 INFO:teuthology.orchestra.run.smithi160.stdout: "crt_file": { 2024-03-07T07:43:33.447 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "crt_file", 2024-03-07T07:43:33.447 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.447 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.447 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.447 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.447 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.448 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.448 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.448 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.448 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.448 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.448 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.448 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.448 INFO:teuthology.orchestra.run.smithi160.stdout: "debug": { 2024-03-07T07:43:33.448 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "debug", 2024-03-07T07:43:33.448 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.448 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.448 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.449 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.449 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.449 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.449 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.449 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Enable/disable debug options", 2024-03-07T07:43:33.449 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.449 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.449 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.449 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.449 INFO:teuthology.orchestra.run.smithi160.stdout: "jwt_token_ttl": { 2024-03-07T07:43:33.449 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "jwt_token_ttl", 2024-03-07T07:43:33.449 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.450 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.450 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.450 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "28800", 2024-03-07T07:43:33.450 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.450 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.450 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.450 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.450 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.450 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.450 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.451 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.451 INFO:teuthology.orchestra.run.smithi160.stdout: "key_file": { 2024-03-07T07:43:33.451 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "key_file", 2024-03-07T07:43:33.451 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.451 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.451 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.451 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.451 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.451 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.451 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.451 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.451 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.452 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.452 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.452 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.452 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.452 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.452 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.452 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.452 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.452 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.452 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.452 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.452 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.453 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.453 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.453 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.453 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.453 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.453 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.453 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.453 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.453 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.453 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.453 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.453 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.453 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.454 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.454 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.454 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.454 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.454 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.454 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.454 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.454 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.454 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.454 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.454 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.454 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.454 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.455 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.455 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.455 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.455 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.455 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.455 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.455 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.455 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.455 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.455 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.455 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.456 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.456 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.456 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.456 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.456 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.456 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.456 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.456 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.456 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.456 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.456 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.456 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.457 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.457 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.457 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.457 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.457 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.457 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.457 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.457 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.457 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.457 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.457 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.457 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.458 INFO:teuthology.orchestra.run.smithi160.stdout: "motd": { 2024-03-07T07:43:33.458 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "motd", 2024-03-07T07:43:33.458 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.458 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.458 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.458 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.458 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.458 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.458 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.458 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "The message of the day", 2024-03-07T07:43:33.458 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.458 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.459 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.459 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.459 INFO:teuthology.orchestra.run.smithi160.stdout: "redirect_resolve_ip_addr": { 2024-03-07T07:43:33.459 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "redirect_resolve_ip_addr", 2024-03-07T07:43:33.459 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.459 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.459 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.459 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.459 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.459 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.459 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.459 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.460 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.460 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.460 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.460 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.460 INFO:teuthology.orchestra.run.smithi160.stdout: "server_addr": { 2024-03-07T07:43:33.460 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "server_addr", 2024-03-07T07:43:33.460 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.460 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.460 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.460 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "::", 2024-03-07T07:43:33.460 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.460 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.461 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.461 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.461 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.461 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.461 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.461 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.461 INFO:teuthology.orchestra.run.smithi160.stdout: "server_port": { 2024-03-07T07:43:33.461 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "server_port", 2024-03-07T07:43:33.461 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.461 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.461 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.462 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "8080", 2024-03-07T07:43:33.462 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.462 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.462 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.462 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.462 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.462 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.462 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.462 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.462 INFO:teuthology.orchestra.run.smithi160.stdout: "ssl": { 2024-03-07T07:43:33.462 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "ssl", 2024-03-07T07:43:33.462 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.462 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.463 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.463 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.463 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.463 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.463 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.463 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.463 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.463 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.463 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.463 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.463 INFO:teuthology.orchestra.run.smithi160.stdout: "ssl_server_port": { 2024-03-07T07:43:33.463 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "ssl_server_port", 2024-03-07T07:43:33.464 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.464 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.464 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.464 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "8443", 2024-03-07T07:43:33.464 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.464 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.464 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.464 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.464 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.464 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.464 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.464 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.465 INFO:teuthology.orchestra.run.smithi160.stdout: "standby_behaviour": { 2024-03-07T07:43:33.465 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "standby_behaviour", 2024-03-07T07:43:33.465 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.465 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.465 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.465 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "redirect", 2024-03-07T07:43:33.465 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.465 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.465 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.465 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.465 INFO:teuthology.orchestra.run.smithi160.stdout: "redirect" 2024-03-07T07:43:33.465 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.465 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.466 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.466 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.466 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.466 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.466 INFO:teuthology.orchestra.run.smithi160.stdout: "standby_error_status_code": { 2024-03-07T07:43:33.466 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "standby_error_status_code", 2024-03-07T07:43:33.466 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.466 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.466 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.466 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "500", 2024-03-07T07:43:33.466 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "400", 2024-03-07T07:43:33.466 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "599", 2024-03-07T07:43:33.467 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.467 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.467 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.467 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.467 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.467 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.467 INFO:teuthology.orchestra.run.smithi160.stdout: "url_prefix": { 2024-03-07T07:43:33.467 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "url_prefix", 2024-03-07T07:43:33.467 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.467 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.467 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.467 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.467 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.468 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.468 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.468 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.468 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.468 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.468 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.468 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.468 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.468 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.468 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.468 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "devicehealth", 2024-03-07T07:43:33.469 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.469 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.469 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.469 INFO:teuthology.orchestra.run.smithi160.stdout: "enable_monitoring": { 2024-03-07T07:43:33.469 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "enable_monitoring", 2024-03-07T07:43:33.469 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.469 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.469 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.469 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.469 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.469 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.470 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.470 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "monitor device health metrics", 2024-03-07T07:43:33.470 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.470 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.470 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.470 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.470 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.470 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.470 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.470 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.470 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.471 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.471 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.471 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.471 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.471 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.471 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.471 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.471 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.471 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.471 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.472 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.472 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.472 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.472 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.472 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.472 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.472 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.472 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.472 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.472 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.472 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.473 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.473 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.473 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.473 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.473 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.473 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.473 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.473 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.473 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.473 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.473 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.473 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.474 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.474 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.474 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.474 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.474 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.474 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.474 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.474 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.474 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.474 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.474 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.474 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.475 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.475 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.475 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.475 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.475 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.475 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.475 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.475 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.475 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.475 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.475 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.475 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.476 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.476 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.476 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.476 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.476 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.476 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.476 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.476 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.476 INFO:teuthology.orchestra.run.smithi160.stdout: "mark_out_threshold": { 2024-03-07T07:43:33.476 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "mark_out_threshold", 2024-03-07T07:43:33.477 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:33.477 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.477 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.477 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "2419200", 2024-03-07T07:43:33.477 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.477 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.477 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.477 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-07T07:43:33.477 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.477 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.477 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.477 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.478 INFO:teuthology.orchestra.run.smithi160.stdout: "pool_name": { 2024-03-07T07:43:33.478 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "pool_name", 2024-03-07T07:43:33.478 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.478 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.478 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.478 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "device_health_metrics", 2024-03-07T07:43:33.478 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.478 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.478 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.478 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-07T07:43:33.478 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.478 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.478 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.479 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.479 INFO:teuthology.orchestra.run.smithi160.stdout: "retention_period": { 2024-03-07T07:43:33.479 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "retention_period", 2024-03-07T07:43:33.479 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:33.479 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.479 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.479 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "15552000", 2024-03-07T07:43:33.479 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.479 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.479 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.479 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "how long to retain device health metrics", 2024-03-07T07:43:33.480 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.480 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.480 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.480 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.480 INFO:teuthology.orchestra.run.smithi160.stdout: "scrape_frequency": { 2024-03-07T07:43:33.480 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "scrape_frequency", 2024-03-07T07:43:33.480 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:33.480 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.480 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.480 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "86400", 2024-03-07T07:43:33.480 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.481 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.481 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.481 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-07T07:43:33.481 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.481 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.481 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.481 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.481 INFO:teuthology.orchestra.run.smithi160.stdout: "self_heal": { 2024-03-07T07:43:33.481 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "self_heal", 2024-03-07T07:43:33.481 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.482 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.482 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.482 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.482 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.482 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.482 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.482 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-07T07:43:33.482 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.482 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.482 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.482 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.483 INFO:teuthology.orchestra.run.smithi160.stdout: "sleep_interval": { 2024-03-07T07:43:33.483 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "sleep_interval", 2024-03-07T07:43:33.483 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:33.483 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.483 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.483 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "600", 2024-03-07T07:43:33.483 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.483 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.483 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.483 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "how frequently to wake up and check device health", 2024-03-07T07:43:33.483 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.483 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.483 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.484 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.484 INFO:teuthology.orchestra.run.smithi160.stdout: "warn_threshold": { 2024-03-07T07:43:33.484 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "warn_threshold", 2024-03-07T07:43:33.484 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:33.484 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.484 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.484 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "7257600", 2024-03-07T07:43:33.484 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.484 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.484 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.484 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-07T07:43:33.484 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.485 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.485 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.485 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.485 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.485 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.485 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.485 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "diskprediction_local", 2024-03-07T07:43:33.485 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.485 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.485 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.485 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.485 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.485 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.485 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.486 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.486 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.486 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.486 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.486 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.486 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.486 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.486 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.486 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.486 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.486 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.486 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.487 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.487 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.487 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.487 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.487 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.487 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.487 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.487 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.487 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.487 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.487 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.487 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.488 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.488 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.488 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.488 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.488 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.488 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.488 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.488 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.488 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.488 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.488 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.489 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.489 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.489 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.489 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.489 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.489 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.489 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.489 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.489 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.489 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.489 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.490 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.490 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.490 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.490 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.490 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.490 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.490 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.490 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.490 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.490 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.490 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.490 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.490 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.491 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.491 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.491 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.491 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.491 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.491 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.491 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.491 INFO:teuthology.orchestra.run.smithi160.stdout: "predict_interval": { 2024-03-07T07:43:33.491 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "predict_interval", 2024-03-07T07:43:33.491 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.491 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.491 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.492 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "86400", 2024-03-07T07:43:33.492 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.492 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.492 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.492 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.492 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.492 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.492 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.492 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.492 INFO:teuthology.orchestra.run.smithi160.stdout: "predictor_model": { 2024-03-07T07:43:33.492 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "predictor_model", 2024-03-07T07:43:33.492 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.493 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.493 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.493 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "prophetstor", 2024-03-07T07:43:33.493 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.493 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.493 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.493 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.493 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.493 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.493 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.493 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.494 INFO:teuthology.orchestra.run.smithi160.stdout: "sleep_interval": { 2024-03-07T07:43:33.494 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "sleep_interval", 2024-03-07T07:43:33.494 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.494 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.494 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.494 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "600", 2024-03-07T07:43:33.494 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.494 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.494 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.494 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.494 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.494 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.495 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.495 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.495 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.495 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.495 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.495 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "influx", 2024-03-07T07:43:33.495 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": false, 2024-03-07T07:43:33.495 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "influxdb python module not found", 2024-03-07T07:43:33.495 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.495 INFO:teuthology.orchestra.run.smithi160.stdout: "batch_size": { 2024-03-07T07:43:33.495 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "batch_size", 2024-03-07T07:43:33.495 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.496 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.496 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.496 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "5000", 2024-03-07T07:43:33.496 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.496 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.496 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.496 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-07T07:43:33.496 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.496 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.497 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.497 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.497 INFO:teuthology.orchestra.run.smithi160.stdout: "database": { 2024-03-07T07:43:33.497 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "database", 2024-03-07T07:43:33.497 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.497 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.497 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.497 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "ceph", 2024-03-07T07:43:33.497 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.497 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.497 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.497 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-07T07:43:33.497 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.498 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.498 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.498 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.498 INFO:teuthology.orchestra.run.smithi160.stdout: "hostname": { 2024-03-07T07:43:33.498 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "hostname", 2024-03-07T07:43:33.498 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.498 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.498 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.498 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.498 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.498 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.498 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.499 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "InfluxDB server hostname", 2024-03-07T07:43:33.499 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.499 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.499 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.499 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.499 INFO:teuthology.orchestra.run.smithi160.stdout: "interval": { 2024-03-07T07:43:33.499 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "interval", 2024-03-07T07:43:33.499 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:33.499 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.499 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.499 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "30", 2024-03-07T07:43:33.500 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "5", 2024-03-07T07:43:33.500 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.500 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.500 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-07T07:43:33.500 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.500 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.500 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.500 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.500 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.500 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.500 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.500 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.501 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.501 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.501 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.501 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.501 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.501 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.501 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.501 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.501 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.501 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.501 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.501 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.502 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.502 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.502 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.502 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.502 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.502 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.502 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.502 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.502 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.502 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.502 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.502 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.503 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.503 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.503 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.503 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.503 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.503 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.503 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.503 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.503 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.503 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.503 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.504 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.504 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.504 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.504 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.504 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.504 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.504 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.504 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.504 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.504 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.504 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.504 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.505 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.505 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.505 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.505 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.505 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.505 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.505 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.505 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.505 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.505 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.505 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.505 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.505 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.505 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.506 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.506 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.506 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.506 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.506 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.506 INFO:teuthology.orchestra.run.smithi160.stdout: "password": { 2024-03-07T07:43:33.506 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "password", 2024-03-07T07:43:33.506 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.506 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.506 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.506 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.507 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.507 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.507 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.507 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "password of InfluxDB server user", 2024-03-07T07:43:33.507 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.507 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.507 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.507 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.507 INFO:teuthology.orchestra.run.smithi160.stdout: "port": { 2024-03-07T07:43:33.507 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "port", 2024-03-07T07:43:33.507 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.507 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.508 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.508 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "8086", 2024-03-07T07:43:33.508 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.508 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.508 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.508 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "InfluxDB server port", 2024-03-07T07:43:33.508 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.508 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.508 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.508 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.508 INFO:teuthology.orchestra.run.smithi160.stdout: "ssl": { 2024-03-07T07:43:33.508 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "ssl", 2024-03-07T07:43:33.508 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.509 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.509 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.509 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "false", 2024-03-07T07:43:33.509 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.509 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.509 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.509 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-07T07:43:33.509 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.509 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.509 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.509 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.509 INFO:teuthology.orchestra.run.smithi160.stdout: "threads": { 2024-03-07T07:43:33.509 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "threads", 2024-03-07T07:43:33.510 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.510 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.510 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.510 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "5", 2024-03-07T07:43:33.510 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "1", 2024-03-07T07:43:33.510 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "32", 2024-03-07T07:43:33.510 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.510 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-07T07:43:33.510 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.510 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.510 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.510 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.511 INFO:teuthology.orchestra.run.smithi160.stdout: "username": { 2024-03-07T07:43:33.511 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "username", 2024-03-07T07:43:33.511 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.511 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.511 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.511 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.511 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.511 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.511 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.511 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "username of InfluxDB server user", 2024-03-07T07:43:33.511 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.511 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.512 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.512 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.512 INFO:teuthology.orchestra.run.smithi160.stdout: "verify_ssl": { 2024-03-07T07:43:33.512 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "verify_ssl", 2024-03-07T07:43:33.512 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.512 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.512 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.512 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "true", 2024-03-07T07:43:33.512 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.512 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.512 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.512 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-07T07:43:33.513 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.513 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.513 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.513 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.513 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.513 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.513 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.513 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "insights", 2024-03-07T07:43:33.513 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.513 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.513 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.513 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.513 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.514 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.514 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.514 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.514 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.514 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.514 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.514 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.514 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.514 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.514 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.514 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.514 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.514 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.515 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.515 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.515 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.515 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.515 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.515 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.515 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.515 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.515 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.515 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.515 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.515 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.515 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.515 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.516 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.516 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.516 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.516 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.516 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.516 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.516 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.516 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.516 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.516 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.516 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.516 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.516 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.517 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.517 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.517 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.517 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.517 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.517 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.517 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.517 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.517 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.517 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.517 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.517 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.517 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.518 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.518 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.518 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.518 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.518 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.518 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.518 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.518 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.518 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.518 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.518 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.519 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.519 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.519 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.519 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.519 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.519 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.519 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.519 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "iostat", 2024-03-07T07:43:33.519 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.519 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.519 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.519 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.519 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.520 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.520 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.520 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.520 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.520 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.520 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.520 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.520 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.520 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.520 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.520 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.521 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.521 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.521 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.521 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.521 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.521 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.521 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.521 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.521 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.522 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.522 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.522 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.522 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.522 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.522 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.522 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.522 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.522 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.522 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.522 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.523 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.523 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.523 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.523 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.523 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.523 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.523 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.523 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.523 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.523 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.523 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.523 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.524 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.524 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.524 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.524 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.524 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.524 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.524 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.524 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.524 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.524 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.524 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.525 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.525 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.525 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.525 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.525 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.525 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.525 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.525 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.525 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.525 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.525 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.526 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.526 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.526 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.526 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.526 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.526 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.526 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "k8sevents", 2024-03-07T07:43:33.526 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.526 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.526 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.526 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_event_retention_days": { 2024-03-07T07:43:33.526 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "ceph_event_retention_days", 2024-03-07T07:43:33.526 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.527 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.527 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.527 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "7", 2024-03-07T07:43:33.527 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.527 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.527 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.527 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-07T07:43:33.527 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.527 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.527 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.527 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.527 INFO:teuthology.orchestra.run.smithi160.stdout: "config_check_secs": { 2024-03-07T07:43:33.527 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "config_check_secs", 2024-03-07T07:43:33.528 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.528 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.528 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.528 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "10", 2024-03-07T07:43:33.528 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "10", 2024-03-07T07:43:33.528 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.528 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.528 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-07T07:43:33.528 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.528 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.528 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.528 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.529 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.529 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.529 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.529 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.529 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.529 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.529 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.529 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.529 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.529 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.529 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.529 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.530 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.530 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.530 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.530 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.530 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.530 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.530 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.530 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.530 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.530 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.530 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.530 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.531 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.531 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.531 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.531 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.531 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.531 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.531 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.531 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.531 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.531 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.531 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.531 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.532 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.532 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.532 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.532 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.532 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.532 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.532 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.532 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.532 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.532 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.532 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.533 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.533 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.533 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.533 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.533 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.533 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.533 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.533 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.533 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.533 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.533 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.533 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.533 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.534 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.534 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.534 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.534 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.534 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.534 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.534 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.534 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.534 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.534 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.534 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.534 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.535 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.535 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "localpool", 2024-03-07T07:43:33.535 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.535 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.535 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.535 INFO:teuthology.orchestra.run.smithi160.stdout: "failure_domain": { 2024-03-07T07:43:33.535 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "failure_domain", 2024-03-07T07:43:33.535 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.535 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.535 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.535 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "host", 2024-03-07T07:43:33.535 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.536 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.536 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.536 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "failure domain for any created local pool", 2024-03-07T07:43:33.536 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-07T07:43:33.536 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.536 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.536 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.536 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.536 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.536 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.537 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.537 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.537 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.537 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.537 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.537 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.537 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.537 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.537 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.537 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.538 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.538 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.538 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.538 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.538 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.538 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.538 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.538 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.538 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.538 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.538 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.539 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.539 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.539 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.539 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.539 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.539 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.539 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.539 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.539 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.539 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.539 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.539 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.540 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.540 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.540 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.540 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.540 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.540 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.540 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.540 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.540 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.540 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.540 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.541 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.541 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.541 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.541 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.541 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.541 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.541 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.541 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.541 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.541 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.541 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.542 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.542 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.542 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.542 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.542 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.542 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.542 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.542 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.542 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.542 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.542 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.542 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.543 INFO:teuthology.orchestra.run.smithi160.stdout: "min_size": { 2024-03-07T07:43:33.543 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "min_size", 2024-03-07T07:43:33.543 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.543 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.543 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.543 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.543 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.543 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.543 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.543 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "default min_size for any created local pool", 2024-03-07T07:43:33.543 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-07T07:43:33.543 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.544 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.544 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.544 INFO:teuthology.orchestra.run.smithi160.stdout: "num_rep": { 2024-03-07T07:43:33.544 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "num_rep", 2024-03-07T07:43:33.544 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.544 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.544 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.544 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "3", 2024-03-07T07:43:33.544 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.544 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.545 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.545 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "default replica count for any created local pool", 2024-03-07T07:43:33.545 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.545 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.545 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.545 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.545 INFO:teuthology.orchestra.run.smithi160.stdout: "pg_num": { 2024-03-07T07:43:33.545 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "pg_num", 2024-03-07T07:43:33.545 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.545 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.545 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.546 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "128", 2024-03-07T07:43:33.546 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.546 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.546 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.546 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "default pg_num for any created local pool", 2024-03-07T07:43:33.546 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.546 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.546 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.546 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.546 INFO:teuthology.orchestra.run.smithi160.stdout: "prefix": { 2024-03-07T07:43:33.546 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "prefix", 2024-03-07T07:43:33.546 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.547 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.547 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.547 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.547 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.547 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.547 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.547 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "name prefix for any created local pool", 2024-03-07T07:43:33.547 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.547 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.547 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.547 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.547 INFO:teuthology.orchestra.run.smithi160.stdout: "subtree": { 2024-03-07T07:43:33.548 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "subtree", 2024-03-07T07:43:33.548 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.548 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.548 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.548 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "rack", 2024-03-07T07:43:33.548 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.548 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.548 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.548 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-07T07:43:33.548 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-07T07:43:33.548 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.549 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.549 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.549 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.549 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.549 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.549 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "mds_autoscaler", 2024-03-07T07:43:33.549 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.549 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.549 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.549 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.549 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.550 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.550 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.550 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.550 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.550 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.550 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.550 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.550 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.550 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.550 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.550 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.551 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.551 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.551 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.551 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.551 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.551 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.551 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.551 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.552 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.552 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.552 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.552 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.552 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.552 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.552 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.552 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.552 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.552 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.552 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.552 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.553 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.553 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.553 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.553 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.553 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.553 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.553 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.553 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.553 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.553 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.553 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.554 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.554 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.554 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.554 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.554 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.554 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.554 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.554 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.554 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.554 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.554 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.554 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.555 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.555 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.555 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.555 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.555 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.555 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.555 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.555 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.555 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.555 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.555 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.555 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.556 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.556 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.556 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.556 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.556 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.556 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "mirroring", 2024-03-07T07:43:33.556 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.556 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.556 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.556 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.556 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.557 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.557 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.557 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.557 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.557 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.557 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.557 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.557 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.557 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.557 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.557 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.557 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.557 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.558 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.558 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.558 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.558 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.558 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.558 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.558 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.558 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.558 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.558 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.559 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.559 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.559 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.559 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.559 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.559 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.559 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.559 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.559 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.559 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.559 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.559 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.560 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.560 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.560 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.560 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.560 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.560 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.560 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.560 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.560 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.560 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.560 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.561 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.561 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.561 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.561 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.561 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.561 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.561 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.561 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.561 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.561 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.561 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.562 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.562 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.562 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.562 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.562 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.562 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.562 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.562 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.562 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.562 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.562 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.562 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.563 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.563 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.563 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "nfs", 2024-03-07T07:43:33.563 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.563 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.563 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.563 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.563 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.563 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.563 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.563 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.563 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.564 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.564 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.564 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.564 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.564 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.564 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.564 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.564 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.564 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.564 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.564 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.564 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.565 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.565 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.565 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.565 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.565 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.565 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.565 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.565 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.565 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.565 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.565 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.565 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.566 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.566 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.566 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.566 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.566 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.566 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.566 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.566 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.566 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.566 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.566 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.566 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.567 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.567 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.567 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.567 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.567 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.567 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.567 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.567 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.567 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.567 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.567 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.568 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.568 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.568 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.568 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.568 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.568 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.568 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.568 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.568 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.568 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.568 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.568 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.569 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.569 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.569 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.569 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.569 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.569 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.569 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.569 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.569 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "orchestrator", 2024-03-07T07:43:33.569 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.569 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.569 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.569 INFO:teuthology.orchestra.run.smithi160.stdout: "fail_fs": { 2024-03-07T07:43:33.570 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "fail_fs", 2024-03-07T07:43:33.570 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.570 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.570 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.570 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.570 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.570 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.570 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.570 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-07T07:43:33.570 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.570 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.570 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.570 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.571 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.571 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.571 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.571 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.571 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.571 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.571 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.571 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.571 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.571 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.571 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.572 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.572 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.572 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.572 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.572 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.572 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.572 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.572 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.572 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.572 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.572 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.572 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.573 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.573 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.573 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.573 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.573 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.573 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.573 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.573 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.573 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.573 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.573 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.574 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.574 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.574 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.574 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.574 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.574 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.574 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.574 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.574 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.574 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.574 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.575 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.575 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.575 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.575 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.575 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.575 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.575 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.575 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.575 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.575 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.575 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.575 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.575 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.575 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.576 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.576 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.576 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.576 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.576 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.576 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.576 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.576 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.576 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.576 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.576 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.576 INFO:teuthology.orchestra.run.smithi160.stdout: "orchestrator": { 2024-03-07T07:43:33.577 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "orchestrator", 2024-03-07T07:43:33.577 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.577 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.577 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.577 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.577 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.577 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.577 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.577 INFO:teuthology.orchestra.run.smithi160.stdout: "cephadm", 2024-03-07T07:43:33.577 INFO:teuthology.orchestra.run.smithi160.stdout: "rook", 2024-03-07T07:43:33.577 INFO:teuthology.orchestra.run.smithi160.stdout: "test_orchestrator" 2024-03-07T07:43:33.577 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.577 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Orchestrator backend", 2024-03-07T07:43:33.577 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.578 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.578 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.578 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.578 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.578 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.578 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.578 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "osd_perf_query", 2024-03-07T07:43:33.578 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.578 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.578 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.578 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.578 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.579 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.579 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.579 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.579 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.579 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.579 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.579 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.579 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.579 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.579 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.579 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.579 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.579 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.579 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.580 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.580 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.580 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.580 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.580 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.580 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.580 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.580 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.580 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.580 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.581 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.581 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.581 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.581 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.581 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.581 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.581 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.581 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.581 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.581 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.581 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.582 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.582 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.582 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.582 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.582 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.582 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.582 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.582 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.582 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.582 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.582 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.583 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.583 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.583 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.583 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.583 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.583 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.583 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.583 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.583 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.583 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.583 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.583 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.584 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.584 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.584 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.584 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.584 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.584 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.584 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.584 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.584 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.584 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.585 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.585 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.585 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.585 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "osd_support", 2024-03-07T07:43:33.585 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.585 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.585 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.585 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.585 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.585 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.585 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.586 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.586 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.586 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.586 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.586 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.586 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.586 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.586 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.586 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.586 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.586 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.586 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.587 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.587 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.587 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.587 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.587 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.587 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.587 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.587 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.587 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.587 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.587 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.587 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.588 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.588 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.588 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.588 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.588 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.588 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.588 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.588 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.588 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.588 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.589 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.589 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.589 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.589 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.589 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.589 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.589 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.589 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.589 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.589 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.589 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.590 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.590 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.590 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.590 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.590 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.590 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.590 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.590 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.590 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.590 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.590 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.591 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.591 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.591 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.591 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.591 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.591 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.591 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.591 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.591 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.591 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.592 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.592 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.592 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.592 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "pg_autoscaler", 2024-03-07T07:43:33.592 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.592 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.592 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.592 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.592 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.592 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.592 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.593 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.593 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.593 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.593 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.593 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.593 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.593 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.593 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.593 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.593 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.593 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.593 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.594 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.594 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.594 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.594 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.594 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.594 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.594 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.594 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.594 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.594 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.594 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.594 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.595 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.595 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.595 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.595 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.595 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.595 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.595 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.595 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.595 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.595 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.595 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.595 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.595 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.596 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.596 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.596 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.596 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.596 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.596 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.596 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.596 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.596 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.596 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.596 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.597 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.597 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.597 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.597 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.597 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.597 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.597 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.597 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.597 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.597 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.597 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.597 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.598 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.598 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.598 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.598 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.598 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.598 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.598 INFO:teuthology.orchestra.run.smithi160.stdout: "sleep_interval": { 2024-03-07T07:43:33.598 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "sleep_interval", 2024-03-07T07:43:33.598 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:33.598 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.598 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.598 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "60", 2024-03-07T07:43:33.599 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.599 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.599 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.599 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.599 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.599 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.599 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.599 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.599 INFO:teuthology.orchestra.run.smithi160.stdout: "threshold": { 2024-03-07T07:43:33.599 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "threshold", 2024-03-07T07:43:33.599 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "float", 2024-03-07T07:43:33.600 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.600 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.600 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "3.0", 2024-03-07T07:43:33.600 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "1.0", 2024-03-07T07:43:33.600 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.600 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.600 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "scaling threshold", 2024-03-07T07:43:33.600 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-07T07:43:33.600 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.600 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.600 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.601 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.601 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.601 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.601 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "progress", 2024-03-07T07:43:33.601 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.601 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.601 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.601 INFO:teuthology.orchestra.run.smithi160.stdout: "allow_pg_recovery_event": { 2024-03-07T07:43:33.601 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "allow_pg_recovery_event", 2024-03-07T07:43:33.601 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.601 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.602 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.602 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.602 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.602 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.602 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.602 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-07T07:43:33.602 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.602 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.603 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.603 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.603 INFO:teuthology.orchestra.run.smithi160.stdout: "enabled": { 2024-03-07T07:43:33.603 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "enabled", 2024-03-07T07:43:33.603 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.603 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.603 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.603 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.603 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.603 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.603 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.604 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.604 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.604 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.604 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.604 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.604 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.604 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.604 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.604 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.604 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.604 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.604 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.605 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.605 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.605 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.605 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.605 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.605 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.605 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.605 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.605 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.605 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.605 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.605 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.605 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.606 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.606 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.606 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.606 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.606 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.606 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.606 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.606 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.606 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.606 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.606 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.606 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.606 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.606 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.607 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.607 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.607 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.607 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.607 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.607 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.607 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.607 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.607 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.607 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.607 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.607 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.607 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.608 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.608 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.608 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.608 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.608 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.608 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.608 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.608 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.608 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.608 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.608 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.609 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.609 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.609 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.609 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.609 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.609 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.609 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.609 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.609 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.609 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.609 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.609 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.610 INFO:teuthology.orchestra.run.smithi160.stdout: "max_completed_events": { 2024-03-07T07:43:33.610 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "max_completed_events", 2024-03-07T07:43:33.610 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.610 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.610 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.610 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "50", 2024-03-07T07:43:33.610 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.610 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.610 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.610 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "number of past completed events to remember", 2024-03-07T07:43:33.610 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.611 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.611 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.611 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.611 INFO:teuthology.orchestra.run.smithi160.stdout: "sleep_interval": { 2024-03-07T07:43:33.611 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "sleep_interval", 2024-03-07T07:43:33.611 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:33.611 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.611 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.611 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "5", 2024-03-07T07:43:33.611 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.612 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.612 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.612 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "how long the module is going to sleep", 2024-03-07T07:43:33.612 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.612 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.612 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.612 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.612 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.612 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.612 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.613 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "prometheus", 2024-03-07T07:43:33.613 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.613 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.613 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.613 INFO:teuthology.orchestra.run.smithi160.stdout: "cache": { 2024-03-07T07:43:33.613 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "cache", 2024-03-07T07:43:33.613 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.613 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.613 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.613 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.613 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.614 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.614 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.614 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.614 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.614 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.614 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.614 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.614 INFO:teuthology.orchestra.run.smithi160.stdout: "exclude_perf_counters": { 2024-03-07T07:43:33.614 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "exclude_perf_counters", 2024-03-07T07:43:33.614 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.614 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.614 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.615 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.615 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.615 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.615 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.615 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-07T07:43:33.615 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-07T07:43:33.615 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.615 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.615 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.616 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.616 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.616 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.616 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.616 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.616 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.616 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.616 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.616 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.616 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.616 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.616 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.617 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.617 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.617 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.617 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.617 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.617 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.617 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.617 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.617 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.617 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.618 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.618 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.618 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.618 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.618 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.618 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.618 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.618 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.618 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.618 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.618 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.618 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.619 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.619 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.619 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.619 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.619 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.619 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.619 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.619 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.619 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.619 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.619 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.619 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.619 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.620 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.620 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.620 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.620 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.620 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.620 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.620 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.620 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.620 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.620 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.620 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.620 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.621 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.621 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.621 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.621 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.621 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.621 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.621 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.621 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.621 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.621 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.621 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.621 INFO:teuthology.orchestra.run.smithi160.stdout: "rbd_stats_pools": { 2024-03-07T07:43:33.622 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "rbd_stats_pools", 2024-03-07T07:43:33.622 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.622 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.622 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.622 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.622 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.622 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.622 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.622 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.622 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.622 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.622 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.622 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.623 INFO:teuthology.orchestra.run.smithi160.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-07T07:43:33.623 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-07T07:43:33.623 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.623 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.623 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.623 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "300", 2024-03-07T07:43:33.623 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.623 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.623 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.623 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.623 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.623 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.624 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.624 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.624 INFO:teuthology.orchestra.run.smithi160.stdout: "scrape_interval": { 2024-03-07T07:43:33.624 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "scrape_interval", 2024-03-07T07:43:33.624 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "float", 2024-03-07T07:43:33.624 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.624 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.624 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "15.0", 2024-03-07T07:43:33.624 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.624 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.624 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.624 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.624 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.625 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.625 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.625 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.625 INFO:teuthology.orchestra.run.smithi160.stdout: "server_addr": { 2024-03-07T07:43:33.625 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "server_addr", 2024-03-07T07:43:33.625 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.625 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.625 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.625 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "::", 2024-03-07T07:43:33.625 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.625 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.625 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.626 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-07T07:43:33.626 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.626 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.626 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.626 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.626 INFO:teuthology.orchestra.run.smithi160.stdout: "server_port": { 2024-03-07T07:43:33.626 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "server_port", 2024-03-07T07:43:33.626 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.626 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.626 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.626 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "9283", 2024-03-07T07:43:33.626 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.626 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.626 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.627 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-07T07:43:33.627 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.627 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.627 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.627 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.627 INFO:teuthology.orchestra.run.smithi160.stdout: "stale_cache_strategy": { 2024-03-07T07:43:33.627 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "stale_cache_strategy", 2024-03-07T07:43:33.627 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.627 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.627 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.627 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "log", 2024-03-07T07:43:33.627 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.628 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.628 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.628 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.628 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.628 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.628 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.628 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.628 INFO:teuthology.orchestra.run.smithi160.stdout: "standby_behaviour": { 2024-03-07T07:43:33.628 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "standby_behaviour", 2024-03-07T07:43:33.628 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.628 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.629 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.629 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "default", 2024-03-07T07:43:33.629 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.629 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.629 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.629 INFO:teuthology.orchestra.run.smithi160.stdout: "default", 2024-03-07T07:43:33.629 INFO:teuthology.orchestra.run.smithi160.stdout: "error" 2024-03-07T07:43:33.629 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.629 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.629 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.629 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.629 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.629 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.629 INFO:teuthology.orchestra.run.smithi160.stdout: "standby_error_status_code": { 2024-03-07T07:43:33.630 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "standby_error_status_code", 2024-03-07T07:43:33.630 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.630 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.630 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.630 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "500", 2024-03-07T07:43:33.630 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "400", 2024-03-07T07:43:33.630 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "599", 2024-03-07T07:43:33.630 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.630 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.630 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.630 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.631 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.631 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.631 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.631 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.631 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.631 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "rbd_support", 2024-03-07T07:43:33.631 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.631 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.631 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.632 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.632 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.632 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.632 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.632 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.632 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.632 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.632 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.632 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.632 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.632 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.632 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.632 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.633 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.633 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.633 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.633 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.633 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.633 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.633 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.633 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.633 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.633 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.633 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.634 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.634 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.634 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.634 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.634 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.634 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.634 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.634 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.634 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.634 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.634 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.635 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.635 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.635 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.635 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.635 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.635 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.635 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.635 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.635 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.635 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.635 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.635 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.635 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.636 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.636 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.636 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.636 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.636 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.636 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.636 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.636 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.636 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.636 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.636 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.637 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.637 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.637 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.637 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.637 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.637 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.637 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.637 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.637 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.637 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.639 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.639 INFO:teuthology.orchestra.run.smithi160.stdout: "max_concurrent_snap_create": { 2024-03-07T07:43:33.639 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "max_concurrent_snap_create", 2024-03-07T07:43:33.639 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.639 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.639 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.639 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "10", 2024-03-07T07:43:33.639 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.639 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.640 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.640 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.640 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.640 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.640 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.640 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.640 INFO:teuthology.orchestra.run.smithi160.stdout: "mirror_snapshot_schedule": { 2024-03-07T07:43:33.640 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "mirror_snapshot_schedule", 2024-03-07T07:43:33.640 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.640 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.641 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.641 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.641 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.641 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.641 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.641 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.641 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.641 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.641 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.641 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.641 INFO:teuthology.orchestra.run.smithi160.stdout: "trash_purge_schedule": { 2024-03-07T07:43:33.642 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "trash_purge_schedule", 2024-03-07T07:43:33.642 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.642 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.642 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.642 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.642 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.642 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.642 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.642 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.642 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.642 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.642 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.642 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.643 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.643 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.643 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.643 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "restful", 2024-03-07T07:43:33.643 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.643 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.643 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.643 INFO:teuthology.orchestra.run.smithi160.stdout: "enable_auth": { 2024-03-07T07:43:33.643 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "enable_auth", 2024-03-07T07:43:33.643 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.643 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.644 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.644 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.644 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.644 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.644 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.644 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.644 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.644 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.644 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.645 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.645 INFO:teuthology.orchestra.run.smithi160.stdout: "key_file": { 2024-03-07T07:43:33.645 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "key_file", 2024-03-07T07:43:33.645 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.645 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.645 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.645 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.645 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.645 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.645 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.645 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.645 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.646 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.646 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.646 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.646 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.646 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.646 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.646 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.646 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.646 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.646 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.646 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.647 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.647 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.647 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.647 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.647 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.647 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.647 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.647 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.647 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.647 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.647 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.647 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.647 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.648 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.648 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.648 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.648 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.648 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.648 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.648 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.648 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.648 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.648 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.648 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.649 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.649 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.649 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.649 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.649 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.649 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.649 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.649 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.649 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.649 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.649 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.649 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.649 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.650 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.650 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.650 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.650 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.650 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.650 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.650 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.650 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.650 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.650 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.650 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.650 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.650 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.650 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.651 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.651 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.651 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.651 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.651 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.651 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.651 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.651 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.651 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.651 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.651 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.652 INFO:teuthology.orchestra.run.smithi160.stdout: "server_addr": { 2024-03-07T07:43:33.652 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "server_addr", 2024-03-07T07:43:33.652 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.652 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.652 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.652 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.652 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.652 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.652 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.652 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.652 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.653 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.653 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.653 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.653 INFO:teuthology.orchestra.run.smithi160.stdout: "server_port": { 2024-03-07T07:43:33.653 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "server_port", 2024-03-07T07:43:33.653 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.653 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.653 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.653 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.653 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.653 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.653 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.654 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.654 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.654 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.654 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.654 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.654 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.654 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.654 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.654 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "rgw", 2024-03-07T07:43:33.654 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.654 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.654 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.654 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.655 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.655 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.655 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.655 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.655 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.655 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.655 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.655 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.655 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.655 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.655 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.655 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.655 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.655 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.656 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.656 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.656 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.656 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.656 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.656 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.656 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.656 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.656 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.656 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.656 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.656 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.657 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.657 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.657 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.657 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.657 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.657 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.657 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.657 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.657 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.657 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.657 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.657 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.658 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.658 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.658 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.658 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.658 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.658 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.658 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.658 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.658 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.658 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.658 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.658 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.659 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.659 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.659 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.659 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.659 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.659 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.659 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.659 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.659 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.659 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.660 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.660 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.660 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.660 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.660 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.660 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.660 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.660 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.660 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.660 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.660 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.661 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.661 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "rook", 2024-03-07T07:43:33.661 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.661 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.661 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.661 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.661 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.661 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.661 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.661 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.662 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.662 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.662 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.662 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.662 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.662 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.662 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.662 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.662 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.662 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.662 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.662 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.662 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.663 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.663 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.663 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.663 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.663 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.663 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.663 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.663 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.663 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.663 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.664 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.664 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.664 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.664 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.664 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.664 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.664 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.664 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.664 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.664 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.664 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.664 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.665 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.665 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.665 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.665 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.665 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.665 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.665 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.665 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.665 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.665 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.665 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.666 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.666 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.666 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.666 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.666 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.666 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.666 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.666 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.666 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.666 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.666 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.666 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.666 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.667 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.667 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.667 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.667 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.667 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.667 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.667 INFO:teuthology.orchestra.run.smithi160.stdout: "storage_class": { 2024-03-07T07:43:33.667 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "storage_class", 2024-03-07T07:43:33.667 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.667 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.667 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.667 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "local", 2024-03-07T07:43:33.667 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.667 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.668 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.668 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-07T07:43:33.668 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.668 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.668 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.668 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.668 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.668 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.668 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.668 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "selftest", 2024-03-07T07:43:33.668 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.668 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.669 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.669 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.669 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.669 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.669 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.669 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.669 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.669 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.669 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.669 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.669 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.669 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.670 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.670 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.670 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.670 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.670 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.670 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.670 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.670 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.670 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.670 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.670 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.670 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.671 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.671 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.671 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.671 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.671 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.671 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.671 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.671 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.671 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.671 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.671 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.671 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.671 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.672 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.672 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.672 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.672 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.672 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.672 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.672 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.672 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.672 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.672 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.672 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.672 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.673 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.673 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.673 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.673 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.673 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.673 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.673 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.673 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.674 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.674 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.674 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.674 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.674 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.674 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.674 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.674 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.675 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.675 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.675 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.675 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.675 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.675 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.675 INFO:teuthology.orchestra.run.smithi160.stdout: "roption1": { 2024-03-07T07:43:33.675 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "roption1", 2024-03-07T07:43:33.675 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.675 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.675 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.675 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.676 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.676 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.676 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.676 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.676 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.676 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.676 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.676 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.676 INFO:teuthology.orchestra.run.smithi160.stdout: "roption2": { 2024-03-07T07:43:33.676 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "roption2", 2024-03-07T07:43:33.676 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.676 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.677 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.677 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "xyz", 2024-03-07T07:43:33.677 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.677 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.677 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.677 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.677 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.677 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.677 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.677 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.677 INFO:teuthology.orchestra.run.smithi160.stdout: "rwoption1": { 2024-03-07T07:43:33.677 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "rwoption1", 2024-03-07T07:43:33.678 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.678 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.678 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.678 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.678 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.678 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.678 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.678 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.678 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.678 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.678 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.678 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.679 INFO:teuthology.orchestra.run.smithi160.stdout: "rwoption2": { 2024-03-07T07:43:33.679 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "rwoption2", 2024-03-07T07:43:33.679 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.679 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.679 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.679 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.679 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.679 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.679 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.679 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.679 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.679 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.679 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.680 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.680 INFO:teuthology.orchestra.run.smithi160.stdout: "rwoption3": { 2024-03-07T07:43:33.680 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "rwoption3", 2024-03-07T07:43:33.680 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "float", 2024-03-07T07:43:33.680 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.680 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.680 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.680 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.680 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.680 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.680 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.681 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.681 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.681 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.681 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.681 INFO:teuthology.orchestra.run.smithi160.stdout: "rwoption4": { 2024-03-07T07:43:33.681 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "rwoption4", 2024-03-07T07:43:33.681 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.681 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.681 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.681 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.681 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.682 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.682 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.682 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.682 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.682 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.682 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.682 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.682 INFO:teuthology.orchestra.run.smithi160.stdout: "rwoption5": { 2024-03-07T07:43:33.682 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "rwoption5", 2024-03-07T07:43:33.683 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.683 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.683 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.683 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.683 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.683 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.683 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.683 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.683 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.683 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.683 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.683 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.684 INFO:teuthology.orchestra.run.smithi160.stdout: "rwoption6": { 2024-03-07T07:43:33.684 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "rwoption6", 2024-03-07T07:43:33.684 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.684 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.684 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.684 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.684 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.684 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.684 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.684 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.684 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.685 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.685 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.685 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.685 INFO:teuthology.orchestra.run.smithi160.stdout: "rwoption7": { 2024-03-07T07:43:33.685 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "rwoption7", 2024-03-07T07:43:33.685 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.685 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.685 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.685 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.685 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "1", 2024-03-07T07:43:33.685 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "42", 2024-03-07T07:43:33.685 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.686 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.686 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.686 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.686 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.686 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.686 INFO:teuthology.orchestra.run.smithi160.stdout: "testkey": { 2024-03-07T07:43:33.686 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "testkey", 2024-03-07T07:43:33.686 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.686 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.686 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.686 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.687 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.687 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.687 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.687 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.687 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.687 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.687 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.687 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.687 INFO:teuthology.orchestra.run.smithi160.stdout: "testlkey": { 2024-03-07T07:43:33.687 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "testlkey", 2024-03-07T07:43:33.687 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.688 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.688 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.688 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.688 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.688 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.688 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.688 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.688 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.688 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.688 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.688 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.688 INFO:teuthology.orchestra.run.smithi160.stdout: "testnewline": { 2024-03-07T07:43:33.689 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "testnewline", 2024-03-07T07:43:33.689 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.689 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.689 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.689 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.689 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.689 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.689 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.689 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.689 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.689 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.689 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.690 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.690 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.690 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.690 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.690 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "snap_schedule", 2024-03-07T07:43:33.690 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.690 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.690 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.690 INFO:teuthology.orchestra.run.smithi160.stdout: "allow_m_granularity": { 2024-03-07T07:43:33.690 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "allow_m_granularity", 2024-03-07T07:43:33.690 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.690 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.691 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.691 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.691 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.691 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.691 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.691 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "allow minute scheduled snapshots", 2024-03-07T07:43:33.691 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.691 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.691 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.691 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.691 INFO:teuthology.orchestra.run.smithi160.stdout: "dump_on_update": { 2024-03-07T07:43:33.692 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "dump_on_update", 2024-03-07T07:43:33.692 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.692 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.692 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.692 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.692 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.692 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.692 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.692 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "dump database to debug log on update", 2024-03-07T07:43:33.692 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.692 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.692 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.693 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.693 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.693 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.693 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.693 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.693 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.693 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.693 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.693 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.693 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.693 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.693 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.693 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.694 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.694 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.694 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.694 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.694 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.694 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.694 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.694 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.694 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.694 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.694 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.694 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.695 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.695 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.695 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.695 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.695 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.695 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.695 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.695 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.696 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.696 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.696 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.696 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.696 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.696 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.696 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.696 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.696 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.696 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.697 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.697 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.697 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.697 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.697 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.697 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.697 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.697 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.698 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.698 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.698 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.698 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.698 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.698 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.698 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.698 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.698 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.698 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.699 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.699 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.699 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.699 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.699 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.699 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.699 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.700 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.700 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.700 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.700 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.700 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.700 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.700 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "stats", 2024-03-07T07:43:33.700 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.700 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.700 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.701 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.701 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.701 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.701 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.701 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.701 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.701 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.701 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.701 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.702 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.702 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.702 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.702 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.702 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.702 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.702 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.702 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.702 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.702 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.703 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.703 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.703 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.703 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.703 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.703 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.703 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.703 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.703 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.703 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.704 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.704 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.704 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.704 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.704 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.704 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.704 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.704 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.704 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.704 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.704 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.705 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.705 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.705 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.705 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.705 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.705 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.705 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.705 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.705 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.706 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.706 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.706 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.706 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.706 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.706 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.706 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.706 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.706 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.706 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.706 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.706 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.707 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.707 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.707 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.707 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.707 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.708 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.708 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.708 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.708 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.708 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.709 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.709 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.709 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "status", 2024-03-07T07:43:33.709 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.710 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.710 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.710 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.710 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.710 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.711 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.711 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.711 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.711 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.711 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.712 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.712 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.712 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.712 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.712 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.712 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.713 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.713 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.713 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.713 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.713 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.713 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.713 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.714 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.714 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.714 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.714 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.714 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.714 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.715 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.715 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.715 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.715 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.715 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.715 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.715 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.716 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.716 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.716 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.716 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.716 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.716 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.716 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.717 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.717 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.717 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.717 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.717 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.717 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.717 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.718 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.718 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.718 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.718 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.718 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.718 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.718 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.718 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.718 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.718 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.718 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.718 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.718 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.719 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.719 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.719 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.719 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.719 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.719 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.719 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.719 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.719 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.719 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.719 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.719 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.720 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "telegraf", 2024-03-07T07:43:33.720 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.720 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.720 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.720 INFO:teuthology.orchestra.run.smithi160.stdout: "address": { 2024-03-07T07:43:33.720 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "address", 2024-03-07T07:43:33.720 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.720 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.720 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.720 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-07T07:43:33.720 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.720 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.721 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.721 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.721 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.721 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.721 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.721 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.721 INFO:teuthology.orchestra.run.smithi160.stdout: "interval": { 2024-03-07T07:43:33.721 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "interval", 2024-03-07T07:43:33.721 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:33.721 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.721 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.721 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "15", 2024-03-07T07:43:33.722 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.722 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.722 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.722 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.722 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.722 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.722 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.722 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.722 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.722 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.722 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.722 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.723 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.723 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.723 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.723 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.723 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.723 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.723 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.723 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.723 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.723 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.723 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.723 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.723 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.724 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.724 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.724 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.724 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.724 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.724 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.724 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.724 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.724 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.724 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.724 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.724 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.725 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.725 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.725 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.725 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.725 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.725 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.725 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.725 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.725 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.725 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.725 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.725 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.725 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.726 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.726 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.726 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.726 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.726 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.726 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.726 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.726 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.726 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.726 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.726 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.726 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.727 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.727 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.727 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.727 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.727 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.727 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.727 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.727 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.727 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.727 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.727 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.727 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.728 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.728 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.728 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.728 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.728 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.728 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.728 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.728 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "telemetry", 2024-03-07T07:43:33.728 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.728 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.728 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.728 INFO:teuthology.orchestra.run.smithi160.stdout: "channel_basic": { 2024-03-07T07:43:33.729 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "channel_basic", 2024-03-07T07:43:33.729 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.729 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.729 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.729 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.729 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.729 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.729 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.729 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-07T07:43:33.729 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.729 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.729 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.730 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.730 INFO:teuthology.orchestra.run.smithi160.stdout: "channel_crash": { 2024-03-07T07:43:33.730 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "channel_crash", 2024-03-07T07:43:33.730 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.730 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.730 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.730 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.730 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.730 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.730 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.730 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-07T07:43:33.730 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.730 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.731 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.731 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.731 INFO:teuthology.orchestra.run.smithi160.stdout: "channel_device": { 2024-03-07T07:43:33.731 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "channel_device", 2024-03-07T07:43:33.731 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.731 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.731 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.731 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.731 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.731 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.731 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.732 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-07T07:43:33.732 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.732 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.732 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.732 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.732 INFO:teuthology.orchestra.run.smithi160.stdout: "channel_ident": { 2024-03-07T07:43:33.732 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "channel_ident", 2024-03-07T07:43:33.732 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.733 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.733 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.733 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.733 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.733 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.733 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.733 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-07T07:43:33.733 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.733 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.734 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.734 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.734 INFO:teuthology.orchestra.run.smithi160.stdout: "channel_perf": { 2024-03-07T07:43:33.734 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "channel_perf", 2024-03-07T07:43:33.734 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.734 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.734 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.734 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.734 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.734 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.734 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.735 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-07T07:43:33.735 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.735 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.735 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.735 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.735 INFO:teuthology.orchestra.run.smithi160.stdout: "contact": { 2024-03-07T07:43:33.735 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "contact", 2024-03-07T07:43:33.735 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.735 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.735 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.736 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.736 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.736 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.736 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.736 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.736 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.736 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.736 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.736 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.736 INFO:teuthology.orchestra.run.smithi160.stdout: "description": { 2024-03-07T07:43:33.737 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "description", 2024-03-07T07:43:33.737 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.737 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.737 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.737 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.737 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.738 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.738 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.738 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.738 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.738 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.738 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.738 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.738 INFO:teuthology.orchestra.run.smithi160.stdout: "device_url": { 2024-03-07T07:43:33.738 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "device_url", 2024-03-07T07:43:33.738 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.738 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.738 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.739 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-07T07:43:33.739 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.739 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.739 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.739 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.739 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.739 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.739 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.739 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.739 INFO:teuthology.orchestra.run.smithi160.stdout: "enabled": { 2024-03-07T07:43:33.739 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "enabled", 2024-03-07T07:43:33.739 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.740 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.740 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.740 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.740 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.740 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.740 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.740 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.740 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.740 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.740 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.740 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.741 INFO:teuthology.orchestra.run.smithi160.stdout: "interval": { 2024-03-07T07:43:33.741 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "interval", 2024-03-07T07:43:33.741 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.741 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.741 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.741 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "24", 2024-03-07T07:43:33.741 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "8", 2024-03-07T07:43:33.741 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.741 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.741 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.741 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.742 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.742 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.742 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.742 INFO:teuthology.orchestra.run.smithi160.stdout: "last_opt_revision": { 2024-03-07T07:43:33.742 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "last_opt_revision", 2024-03-07T07:43:33.742 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.742 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.742 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.742 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "1", 2024-03-07T07:43:33.742 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.742 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.742 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.743 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.743 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.743 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.743 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.743 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.743 INFO:teuthology.orchestra.run.smithi160.stdout: "leaderboard": { 2024-03-07T07:43:33.743 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "leaderboard", 2024-03-07T07:43:33.743 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.743 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.743 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.743 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.744 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.744 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.744 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.744 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.744 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.744 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.744 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.744 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.744 INFO:teuthology.orchestra.run.smithi160.stdout: "leaderboard_description": { 2024-03-07T07:43:33.744 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "leaderboard_description", 2024-03-07T07:43:33.744 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.744 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.745 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.745 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.745 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.745 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.745 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.745 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.745 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.745 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.745 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.745 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.745 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.745 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.746 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.746 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.746 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.746 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.746 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.746 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.746 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.746 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.746 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.746 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.746 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.747 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.747 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.747 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.747 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.747 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.747 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.747 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.747 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.747 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.747 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.747 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.747 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.748 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.748 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.748 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.748 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.748 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.748 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.748 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.748 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.748 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.748 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.748 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.748 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.749 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.749 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.749 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.749 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.749 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.749 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.749 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.749 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.749 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.749 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.749 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.750 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.750 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.750 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.750 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.750 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.750 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.750 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.750 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.750 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.750 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.750 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.751 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.751 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.751 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.751 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.751 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.751 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.751 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.751 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.751 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.751 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.751 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.752 INFO:teuthology.orchestra.run.smithi160.stdout: "organization": { 2024-03-07T07:43:33.752 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "organization", 2024-03-07T07:43:33.752 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.752 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.752 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.752 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.752 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.752 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.752 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.752 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.752 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.753 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.753 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.753 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.753 INFO:teuthology.orchestra.run.smithi160.stdout: "proxy": { 2024-03-07T07:43:33.753 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "proxy", 2024-03-07T07:43:33.753 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.753 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.753 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.753 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.753 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.753 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.754 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.754 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.754 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.754 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.754 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.754 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.754 INFO:teuthology.orchestra.run.smithi160.stdout: "url": { 2024-03-07T07:43:33.754 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "url", 2024-03-07T07:43:33.754 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.754 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.754 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.755 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-07T07:43:33.755 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.755 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.755 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.755 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.755 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.755 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.755 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.755 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.755 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.755 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.756 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.756 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "test_orchestrator", 2024-03-07T07:43:33.756 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.756 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.756 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.756 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.756 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.756 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.756 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.756 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.757 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.757 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.757 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.757 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.757 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.757 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.757 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.757 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.757 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.757 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.758 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.758 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.758 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.758 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.758 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.758 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.758 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.758 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.758 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.758 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.758 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.759 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.759 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.759 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.759 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.759 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.759 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.759 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.759 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.759 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.759 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.759 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.759 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.760 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.760 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.760 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.760 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.760 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.760 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.760 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.760 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.760 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.760 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.760 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.760 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.761 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.761 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.761 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.761 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.761 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.761 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.761 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.761 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.761 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.761 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.761 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.761 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.762 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.762 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.762 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.762 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.762 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.762 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.762 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.762 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.762 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.762 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.762 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.762 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "volumes", 2024-03-07T07:43:33.763 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.763 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.763 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.763 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.763 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.763 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.763 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.763 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.763 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.763 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.763 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.763 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.764 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.764 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.764 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.764 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.764 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.764 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.764 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.764 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.764 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.764 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.764 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.764 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.764 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.765 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.765 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.765 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.765 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.765 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.765 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.765 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.765 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.765 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.765 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.765 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.765 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.766 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.766 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.766 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.766 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.766 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.766 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.766 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.766 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.766 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.766 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.766 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.767 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.767 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.767 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.767 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.767 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.767 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.767 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.767 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.767 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.768 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.768 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.768 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.768 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.768 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.768 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.768 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.768 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.768 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.768 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.768 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.769 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.769 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.769 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.769 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.769 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.769 INFO:teuthology.orchestra.run.smithi160.stdout: "max_concurrent_clones": { 2024-03-07T07:43:33.769 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "max_concurrent_clones", 2024-03-07T07:43:33.769 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.769 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.769 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.769 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "4", 2024-03-07T07:43:33.770 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.770 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.770 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.770 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-07T07:43:33.770 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.770 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.770 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.770 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.770 INFO:teuthology.orchestra.run.smithi160.stdout: "periodic_async_work": { 2024-03-07T07:43:33.770 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "periodic_async_work", 2024-03-07T07:43:33.770 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.770 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.771 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.771 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.771 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.771 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.771 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.771 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Periodically check for async work", 2024-03-07T07:43:33.771 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.771 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.771 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.771 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.771 INFO:teuthology.orchestra.run.smithi160.stdout: "snapshot_clone_delay": { 2024-03-07T07:43:33.771 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "snapshot_clone_delay", 2024-03-07T07:43:33.772 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.772 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.772 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.772 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "0", 2024-03-07T07:43:33.772 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.772 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.772 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.772 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-07T07:43:33.772 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.772 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.772 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.772 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.773 INFO:teuthology.orchestra.run.smithi160.stdout: "snapshot_clone_no_wait": { 2024-03-07T07:43:33.773 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "snapshot_clone_no_wait", 2024-03-07T07:43:33.773 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.773 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.773 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.773 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.773 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.773 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.773 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.773 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-07T07:43:33.773 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.773 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.773 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.774 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.774 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.774 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.774 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.774 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "zabbix", 2024-03-07T07:43:33.774 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.774 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.774 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.774 INFO:teuthology.orchestra.run.smithi160.stdout: "discovery_interval": { 2024-03-07T07:43:33.774 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "discovery_interval", 2024-03-07T07:43:33.774 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "uint", 2024-03-07T07:43:33.774 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.774 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.775 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "100", 2024-03-07T07:43:33.775 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.775 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.775 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.775 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.775 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.775 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.775 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.775 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.775 INFO:teuthology.orchestra.run.smithi160.stdout: "identifier": { 2024-03-07T07:43:33.775 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "identifier", 2024-03-07T07:43:33.775 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.775 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.776 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.776 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.776 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.776 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.776 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.776 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.776 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.776 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.776 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.776 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.776 INFO:teuthology.orchestra.run.smithi160.stdout: "interval": { 2024-03-07T07:43:33.776 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "interval", 2024-03-07T07:43:33.777 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:33.777 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.777 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.777 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "60", 2024-03-07T07:43:33.777 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.777 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.777 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.777 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.777 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.777 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.778 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.778 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.778 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.778 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.778 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.778 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.778 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.778 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.778 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.778 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.778 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.779 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.779 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.779 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.779 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.779 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.779 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.779 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.779 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.779 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.779 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.779 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.779 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.780 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.780 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.780 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.780 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.780 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.780 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.780 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.780 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.780 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.780 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.780 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.781 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.781 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.781 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.781 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.781 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.781 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.781 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.781 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.781 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.781 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.781 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.782 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.782 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.782 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.782 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.782 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.782 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.782 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.782 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.782 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.782 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.782 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.782 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.783 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.783 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.783 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.783 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.783 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.783 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.783 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.783 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.783 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.783 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.783 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.784 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.784 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.784 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.784 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.784 INFO:teuthology.orchestra.run.smithi160.stdout: "zabbix_host": { 2024-03-07T07:43:33.784 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "zabbix_host", 2024-03-07T07:43:33.784 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.784 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.784 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.784 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.784 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.784 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.785 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.785 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.785 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.785 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.785 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.785 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.785 INFO:teuthology.orchestra.run.smithi160.stdout: "zabbix_port": { 2024-03-07T07:43:33.785 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "zabbix_port", 2024-03-07T07:43:33.785 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.785 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.786 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.786 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "10051", 2024-03-07T07:43:33.786 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.786 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.786 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.786 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.786 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.786 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.786 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.786 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.786 INFO:teuthology.orchestra.run.smithi160.stdout: "zabbix_sender": { 2024-03-07T07:43:33.786 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "zabbix_sender", 2024-03-07T07:43:33.786 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.787 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.787 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.787 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-07T07:43:33.787 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.787 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.787 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.787 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.787 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.787 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.787 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.787 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.787 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.787 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.787 INFO:teuthology.orchestra.run.smithi160.stdout: ] 2024-03-07T07:43:33.788 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.788 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.788 INFO:teuthology.orchestra.run.smithi160.stdout: "modules": [ 2024-03-07T07:43:33.788 INFO:teuthology.orchestra.run.smithi160.stdout: "cephadm", 2024-03-07T07:43:33.788 INFO:teuthology.orchestra.run.smithi160.stdout: "dashboard", 2024-03-07T07:43:33.788 INFO:teuthology.orchestra.run.smithi160.stdout: "iostat", 2024-03-07T07:43:33.788 INFO:teuthology.orchestra.run.smithi160.stdout: "nfs", 2024-03-07T07:43:33.788 INFO:teuthology.orchestra.run.smithi160.stdout: "restful" 2024-03-07T07:43:33.789 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.789 INFO:teuthology.orchestra.run.smithi160.stdout: "available_modules": [ 2024-03-07T07:43:33.789 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.789 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "alerts", 2024-03-07T07:43:33.789 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.789 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.789 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.789 INFO:teuthology.orchestra.run.smithi160.stdout: "interval": { 2024-03-07T07:43:33.789 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "interval", 2024-03-07T07:43:33.789 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:33.789 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.789 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.790 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "60", 2024-03-07T07:43:33.790 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.790 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.790 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.790 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "How frequently to reexamine health status", 2024-03-07T07:43:33.790 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.790 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.790 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.790 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.790 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.790 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.790 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.790 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.790 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.791 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.791 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.791 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.791 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.791 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.791 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.791 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.791 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.791 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.791 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.791 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.791 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.792 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.792 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.792 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.792 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.792 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.792 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.792 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.792 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.792 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.792 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.792 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.792 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.792 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.793 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.793 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.793 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.793 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.793 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.793 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.793 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.793 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.793 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.793 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.793 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.793 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.793 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.793 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.794 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.794 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.794 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.794 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.794 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.794 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.794 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.794 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.794 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.794 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.794 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.794 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.794 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.795 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.795 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.795 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.795 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.795 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.795 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.795 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.795 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.795 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.795 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.795 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.795 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.795 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.795 INFO:teuthology.orchestra.run.smithi160.stdout: "smtp_destination": { 2024-03-07T07:43:33.796 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "smtp_destination", 2024-03-07T07:43:33.796 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.796 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.796 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.796 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.796 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.796 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.796 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.796 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Email address to send alerts to", 2024-03-07T07:43:33.796 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.796 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.796 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.797 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.797 INFO:teuthology.orchestra.run.smithi160.stdout: "smtp_from_name": { 2024-03-07T07:43:33.797 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "smtp_from_name", 2024-03-07T07:43:33.797 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.797 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.797 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.797 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "Ceph", 2024-03-07T07:43:33.797 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.797 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.797 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.797 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Email From: name", 2024-03-07T07:43:33.797 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.797 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.797 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.798 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.798 INFO:teuthology.orchestra.run.smithi160.stdout: "smtp_host": { 2024-03-07T07:43:33.798 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "smtp_host", 2024-03-07T07:43:33.798 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.798 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.798 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.798 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.798 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.798 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.798 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.798 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "SMTP server", 2024-03-07T07:43:33.799 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.799 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.799 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.799 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.799 INFO:teuthology.orchestra.run.smithi160.stdout: "smtp_password": { 2024-03-07T07:43:33.799 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "smtp_password", 2024-03-07T07:43:33.799 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.799 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.799 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.799 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.799 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.800 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.800 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.800 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Password to authenticate with", 2024-03-07T07:43:33.800 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.800 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.800 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.800 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.800 INFO:teuthology.orchestra.run.smithi160.stdout: "smtp_port": { 2024-03-07T07:43:33.800 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "smtp_port", 2024-03-07T07:43:33.801 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.801 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.801 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.801 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "465", 2024-03-07T07:43:33.801 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.801 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.801 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.801 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "SMTP port", 2024-03-07T07:43:33.801 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.801 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.802 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.802 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.802 INFO:teuthology.orchestra.run.smithi160.stdout: "smtp_sender": { 2024-03-07T07:43:33.802 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "smtp_sender", 2024-03-07T07:43:33.802 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.802 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.802 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.802 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.802 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.802 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.803 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.803 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "SMTP envelope sender", 2024-03-07T07:43:33.803 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.803 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.803 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.803 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.803 INFO:teuthology.orchestra.run.smithi160.stdout: "smtp_ssl": { 2024-03-07T07:43:33.803 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "smtp_ssl", 2024-03-07T07:43:33.803 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.803 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.803 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.803 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.803 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.804 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.804 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.804 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-07T07:43:33.804 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.804 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.804 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.804 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.804 INFO:teuthology.orchestra.run.smithi160.stdout: "smtp_user": { 2024-03-07T07:43:33.804 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "smtp_user", 2024-03-07T07:43:33.804 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.805 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.805 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.805 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.805 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.805 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.805 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.805 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "User to authenticate as", 2024-03-07T07:43:33.805 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.805 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.805 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.805 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.805 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.806 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.806 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.806 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "balancer", 2024-03-07T07:43:33.806 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.806 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.806 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.806 INFO:teuthology.orchestra.run.smithi160.stdout: "active": { 2024-03-07T07:43:33.806 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "active", 2024-03-07T07:43:33.806 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.806 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.806 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.806 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.807 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.807 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.807 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.807 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "automatically balance PGs across cluster", 2024-03-07T07:43:33.807 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.807 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.807 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.807 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.807 INFO:teuthology.orchestra.run.smithi160.stdout: "begin_time": { 2024-03-07T07:43:33.807 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "begin_time", 2024-03-07T07:43:33.807 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.808 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.808 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.808 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "0000", 2024-03-07T07:43:33.808 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.808 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.808 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.808 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "beginning time of day to automatically balance", 2024-03-07T07:43:33.808 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-07T07:43:33.808 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.808 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.808 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.808 INFO:teuthology.orchestra.run.smithi160.stdout: "begin_weekday": { 2024-03-07T07:43:33.809 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "begin_weekday", 2024-03-07T07:43:33.809 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "uint", 2024-03-07T07:43:33.809 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.809 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.809 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "0", 2024-03-07T07:43:33.809 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "0", 2024-03-07T07:43:33.809 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "6", 2024-03-07T07:43:33.809 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.809 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-07T07:43:33.809 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-07T07:43:33.809 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.809 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.810 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.810 INFO:teuthology.orchestra.run.smithi160.stdout: "crush_compat_max_iterations": { 2024-03-07T07:43:33.810 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "crush_compat_max_iterations", 2024-03-07T07:43:33.810 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "uint", 2024-03-07T07:43:33.810 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.810 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.810 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "25", 2024-03-07T07:43:33.810 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "1", 2024-03-07T07:43:33.810 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "250", 2024-03-07T07:43:33.810 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.810 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-07T07:43:33.810 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.811 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.811 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.811 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.811 INFO:teuthology.orchestra.run.smithi160.stdout: "crush_compat_metrics": { 2024-03-07T07:43:33.811 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "crush_compat_metrics", 2024-03-07T07:43:33.811 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.811 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.811 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.811 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "pgs,objects,bytes", 2024-03-07T07:43:33.811 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.811 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.811 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.812 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-07T07:43:33.812 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-07T07:43:33.812 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.812 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.812 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.812 INFO:teuthology.orchestra.run.smithi160.stdout: "crush_compat_step": { 2024-03-07T07:43:33.812 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "crush_compat_step", 2024-03-07T07:43:33.812 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "float", 2024-03-07T07:43:33.812 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.812 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.812 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "0.5", 2024-03-07T07:43:33.812 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "0.001", 2024-03-07T07:43:33.813 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "0.999", 2024-03-07T07:43:33.813 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.813 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "aggressiveness of optimization", 2024-03-07T07:43:33.813 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-07T07:43:33.813 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.813 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.813 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.813 INFO:teuthology.orchestra.run.smithi160.stdout: "end_time": { 2024-03-07T07:43:33.813 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "end_time", 2024-03-07T07:43:33.813 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.813 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.813 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.813 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "2359", 2024-03-07T07:43:33.814 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.814 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.814 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.814 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "ending time of day to automatically balance", 2024-03-07T07:43:33.814 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-07T07:43:33.814 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.814 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.814 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.814 INFO:teuthology.orchestra.run.smithi160.stdout: "end_weekday": { 2024-03-07T07:43:33.814 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "end_weekday", 2024-03-07T07:43:33.814 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "uint", 2024-03-07T07:43:33.814 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.815 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.815 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "0", 2024-03-07T07:43:33.815 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "0", 2024-03-07T07:43:33.815 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "6", 2024-03-07T07:43:33.815 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.815 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-07T07:43:33.815 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-07T07:43:33.815 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.815 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.815 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.815 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.815 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.815 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.816 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.816 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.816 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.816 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.816 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.816 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.816 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.816 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.816 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.816 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.816 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.816 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.817 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.817 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.817 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.817 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.817 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.817 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.817 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.817 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.817 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.817 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.818 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.818 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.818 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.818 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.818 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.818 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.818 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.818 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.818 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.818 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.818 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.819 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.819 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.819 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.819 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.819 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.819 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.819 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.819 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.819 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.819 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.819 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.820 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.820 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.820 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.820 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.820 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.820 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.820 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.820 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.820 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.821 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.821 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.821 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.821 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.821 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.821 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.821 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.821 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.821 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.821 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.821 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.822 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.822 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.822 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.822 INFO:teuthology.orchestra.run.smithi160.stdout: "min_score": { 2024-03-07T07:43:33.822 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "min_score", 2024-03-07T07:43:33.822 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "float", 2024-03-07T07:43:33.822 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.822 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.822 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "0", 2024-03-07T07:43:33.822 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.822 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.822 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.822 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-07T07:43:33.823 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.823 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.823 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.823 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.823 INFO:teuthology.orchestra.run.smithi160.stdout: "mode": { 2024-03-07T07:43:33.823 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "mode", 2024-03-07T07:43:33.823 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.823 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.823 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.823 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "upmap", 2024-03-07T07:43:33.823 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.823 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.823 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.824 INFO:teuthology.orchestra.run.smithi160.stdout: "crush-compat", 2024-03-07T07:43:33.824 INFO:teuthology.orchestra.run.smithi160.stdout: "none", 2024-03-07T07:43:33.824 INFO:teuthology.orchestra.run.smithi160.stdout: "read", 2024-03-07T07:43:33.824 INFO:teuthology.orchestra.run.smithi160.stdout: "upmap", 2024-03-07T07:43:33.824 INFO:teuthology.orchestra.run.smithi160.stdout: "upmap-read" 2024-03-07T07:43:33.824 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.824 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Balancer mode", 2024-03-07T07:43:33.824 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.824 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.824 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.824 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.824 INFO:teuthology.orchestra.run.smithi160.stdout: "pool_ids": { 2024-03-07T07:43:33.824 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "pool_ids", 2024-03-07T07:43:33.825 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.825 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.825 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.825 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.825 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.825 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.825 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.825 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-07T07:43:33.825 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.825 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.825 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.825 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.825 INFO:teuthology.orchestra.run.smithi160.stdout: "sleep_interval": { 2024-03-07T07:43:33.825 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "sleep_interval", 2024-03-07T07:43:33.826 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:33.826 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.826 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.826 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "60", 2024-03-07T07:43:33.826 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.826 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.826 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.826 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-07T07:43:33.826 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.826 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.826 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.827 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.827 INFO:teuthology.orchestra.run.smithi160.stdout: "upmap_max_deviation": { 2024-03-07T07:43:33.827 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "upmap_max_deviation", 2024-03-07T07:43:33.827 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.827 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.827 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.827 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "5", 2024-03-07T07:43:33.827 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "1", 2024-03-07T07:43:33.827 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.827 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.827 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-07T07:43:33.827 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-07T07:43:33.827 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.828 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.828 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.828 INFO:teuthology.orchestra.run.smithi160.stdout: "upmap_max_optimizations": { 2024-03-07T07:43:33.828 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "upmap_max_optimizations", 2024-03-07T07:43:33.828 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "uint", 2024-03-07T07:43:33.828 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.828 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.828 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "10", 2024-03-07T07:43:33.828 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.828 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.828 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.828 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-07T07:43:33.829 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.829 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.829 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.829 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.829 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.829 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.829 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.829 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "cephadm", 2024-03-07T07:43:33.829 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.829 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.829 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.829 INFO:teuthology.orchestra.run.smithi160.stdout: "agent_down_multiplier": { 2024-03-07T07:43:33.829 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "agent_down_multiplier", 2024-03-07T07:43:33.829 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "float", 2024-03-07T07:43:33.830 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.830 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.830 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "3.0", 2024-03-07T07:43:33.830 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.830 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.830 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.830 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-07T07:43:33.830 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.830 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.830 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.830 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.830 INFO:teuthology.orchestra.run.smithi160.stdout: "agent_refresh_rate": { 2024-03-07T07:43:33.831 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "agent_refresh_rate", 2024-03-07T07:43:33.831 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:33.831 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.831 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.831 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "20", 2024-03-07T07:43:33.831 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.831 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.831 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.831 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-07T07:43:33.831 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.831 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.831 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.832 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.832 INFO:teuthology.orchestra.run.smithi160.stdout: "agent_starting_port": { 2024-03-07T07:43:33.832 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "agent_starting_port", 2024-03-07T07:43:33.832 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.832 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.832 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.832 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "4721", 2024-03-07T07:43:33.832 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.832 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.832 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.833 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-07T07:43:33.833 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.833 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.833 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.833 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.833 INFO:teuthology.orchestra.run.smithi160.stdout: "allow_ptrace": { 2024-03-07T07:43:33.833 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "allow_ptrace", 2024-03-07T07:43:33.833 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.833 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.833 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.833 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.834 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.834 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.834 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.834 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-07T07:43:33.834 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-07T07:43:33.834 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.834 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.834 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.834 INFO:teuthology.orchestra.run.smithi160.stdout: "autotune_interval": { 2024-03-07T07:43:33.834 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "autotune_interval", 2024-03-07T07:43:33.834 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:33.834 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.834 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.835 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "600", 2024-03-07T07:43:33.835 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.835 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.835 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.835 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-07T07:43:33.835 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.835 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.835 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.835 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.835 INFO:teuthology.orchestra.run.smithi160.stdout: "autotune_memory_target_ratio": { 2024-03-07T07:43:33.835 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "autotune_memory_target_ratio", 2024-03-07T07:43:33.835 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "float", 2024-03-07T07:43:33.836 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.836 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.836 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "0.7", 2024-03-07T07:43:33.836 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.836 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.836 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.836 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-07T07:43:33.836 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.836 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.836 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.837 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.837 INFO:teuthology.orchestra.run.smithi160.stdout: "cephadm_log_destination": { 2024-03-07T07:43:33.837 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "cephadm_log_destination", 2024-03-07T07:43:33.837 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.837 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.837 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.837 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.837 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.837 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.837 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.837 INFO:teuthology.orchestra.run.smithi160.stdout: "file", 2024-03-07T07:43:33.837 INFO:teuthology.orchestra.run.smithi160.stdout: "file,syslog", 2024-03-07T07:43:33.837 INFO:teuthology.orchestra.run.smithi160.stdout: "syslog" 2024-03-07T07:43:33.838 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.838 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-07T07:43:33.838 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.838 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.838 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.838 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.838 INFO:teuthology.orchestra.run.smithi160.stdout: "cgroups_split": { 2024-03-07T07:43:33.838 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "cgroups_split", 2024-03-07T07:43:33.838 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.838 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.839 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.839 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.839 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.839 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.839 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.839 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-07T07:43:33.839 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.839 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.839 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.839 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.839 INFO:teuthology.orchestra.run.smithi160.stdout: "config_checks_enabled": { 2024-03-07T07:43:33.839 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "config_checks_enabled", 2024-03-07T07:43:33.839 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.840 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.840 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.840 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.840 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.840 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.840 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.840 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-07T07:43:33.840 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.840 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.841 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.841 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.841 INFO:teuthology.orchestra.run.smithi160.stdout: "config_dashboard": { 2024-03-07T07:43:33.841 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "config_dashboard", 2024-03-07T07:43:33.841 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.841 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.841 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.841 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.841 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.841 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.841 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.841 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-07T07:43:33.843 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.843 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.843 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.843 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.843 INFO:teuthology.orchestra.run.smithi160.stdout: "container_image_alertmanager": { 2024-03-07T07:43:33.843 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "container_image_alertmanager", 2024-03-07T07:43:33.843 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.844 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.844 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.844 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-07T07:43:33.844 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.844 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.844 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.844 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Prometheus container image", 2024-03-07T07:43:33.844 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.844 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.844 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.844 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.845 INFO:teuthology.orchestra.run.smithi160.stdout: "container_image_base": { 2024-03-07T07:43:33.845 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "container_image_base", 2024-03-07T07:43:33.845 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.845 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.845 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.845 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-07T07:43:33.845 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.845 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.845 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.845 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Container image name, without the tag", 2024-03-07T07:43:33.845 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.846 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.846 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.846 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.846 INFO:teuthology.orchestra.run.smithi160.stdout: "container_image_elasticsearch": { 2024-03-07T07:43:33.846 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "container_image_elasticsearch", 2024-03-07T07:43:33.846 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.846 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.846 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.846 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-07T07:43:33.846 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.846 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.847 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.847 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "elasticsearch container image", 2024-03-07T07:43:33.847 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.847 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.847 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.847 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.847 INFO:teuthology.orchestra.run.smithi160.stdout: "container_image_grafana": { 2024-03-07T07:43:33.847 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "container_image_grafana", 2024-03-07T07:43:33.847 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.847 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.847 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.847 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-07T07:43:33.847 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.848 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.848 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.848 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Prometheus container image", 2024-03-07T07:43:33.848 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.848 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.848 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.848 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.848 INFO:teuthology.orchestra.run.smithi160.stdout: "container_image_haproxy": { 2024-03-07T07:43:33.848 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "container_image_haproxy", 2024-03-07T07:43:33.848 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.848 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.848 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.849 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-07T07:43:33.849 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.849 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.849 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.849 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "HAproxy container image", 2024-03-07T07:43:33.849 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.849 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.849 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.849 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.849 INFO:teuthology.orchestra.run.smithi160.stdout: "container_image_jaeger_agent": { 2024-03-07T07:43:33.849 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "container_image_jaeger_agent", 2024-03-07T07:43:33.849 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.849 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.850 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.850 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-07T07:43:33.850 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.850 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.850 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.850 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Jaeger agent container image", 2024-03-07T07:43:33.850 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.850 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.850 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.850 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.850 INFO:teuthology.orchestra.run.smithi160.stdout: "container_image_jaeger_collector": { 2024-03-07T07:43:33.850 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "container_image_jaeger_collector", 2024-03-07T07:43:33.850 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.851 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.851 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.851 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-07T07:43:33.851 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.851 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.851 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.851 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Jaeger collector container image", 2024-03-07T07:43:33.851 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.851 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.851 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.851 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.852 INFO:teuthology.orchestra.run.smithi160.stdout: "container_image_jaeger_query": { 2024-03-07T07:43:33.852 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "container_image_jaeger_query", 2024-03-07T07:43:33.852 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.852 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.852 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.852 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-07T07:43:33.852 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.852 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.852 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.852 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Jaeger query container image", 2024-03-07T07:43:33.852 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.853 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.853 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.853 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.853 INFO:teuthology.orchestra.run.smithi160.stdout: "container_image_keepalived": { 2024-03-07T07:43:33.853 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "container_image_keepalived", 2024-03-07T07:43:33.853 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.853 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.853 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.853 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-07T07:43:33.853 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.853 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.853 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.853 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Keepalived container image", 2024-03-07T07:43:33.854 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.854 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.854 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.854 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.854 INFO:teuthology.orchestra.run.smithi160.stdout: "container_image_loki": { 2024-03-07T07:43:33.854 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "container_image_loki", 2024-03-07T07:43:33.854 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.854 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.854 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.854 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-07T07:43:33.854 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.854 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.854 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.855 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Loki container image", 2024-03-07T07:43:33.855 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.855 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.855 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.855 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.855 INFO:teuthology.orchestra.run.smithi160.stdout: "container_image_node_exporter": { 2024-03-07T07:43:33.855 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "container_image_node_exporter", 2024-03-07T07:43:33.855 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.855 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.855 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.855 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-07T07:43:33.855 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.856 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.856 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.856 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Prometheus container image", 2024-03-07T07:43:33.856 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.856 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.856 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.856 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.856 INFO:teuthology.orchestra.run.smithi160.stdout: "container_image_nvmeof": { 2024-03-07T07:43:33.856 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "container_image_nvmeof", 2024-03-07T07:43:33.856 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.856 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.856 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.857 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-07T07:43:33.857 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.857 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.857 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.857 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Nvme-of container image", 2024-03-07T07:43:33.857 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.857 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.857 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.857 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.857 INFO:teuthology.orchestra.run.smithi160.stdout: "container_image_prometheus": { 2024-03-07T07:43:33.857 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "container_image_prometheus", 2024-03-07T07:43:33.857 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.858 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.858 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.858 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-07T07:43:33.858 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.858 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.858 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.858 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Prometheus container image", 2024-03-07T07:43:33.858 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.858 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.858 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.858 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.858 INFO:teuthology.orchestra.run.smithi160.stdout: "container_image_promtail": { 2024-03-07T07:43:33.859 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "container_image_promtail", 2024-03-07T07:43:33.859 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.859 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.859 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.859 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-07T07:43:33.859 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.859 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.859 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.859 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Promtail container image", 2024-03-07T07:43:33.859 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.859 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.859 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.860 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.860 INFO:teuthology.orchestra.run.smithi160.stdout: "container_image_snmp_gateway": { 2024-03-07T07:43:33.860 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "container_image_snmp_gateway", 2024-03-07T07:43:33.860 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.860 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.860 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.860 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-07T07:43:33.860 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.860 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.860 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.861 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "SNMP Gateway container image", 2024-03-07T07:43:33.861 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.861 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.861 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.861 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.861 INFO:teuthology.orchestra.run.smithi160.stdout: "container_init": { 2024-03-07T07:43:33.861 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "container_init", 2024-03-07T07:43:33.861 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.861 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.861 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.861 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.862 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.862 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.862 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.862 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Run podman/docker with `--init`", 2024-03-07T07:43:33.862 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.862 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.862 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.862 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.862 INFO:teuthology.orchestra.run.smithi160.stdout: "daemon_cache_timeout": { 2024-03-07T07:43:33.862 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "daemon_cache_timeout", 2024-03-07T07:43:33.863 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:33.863 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.863 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.863 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "600", 2024-03-07T07:43:33.863 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.863 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.863 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.863 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-07T07:43:33.863 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.863 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.864 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.864 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.864 INFO:teuthology.orchestra.run.smithi160.stdout: "default_cephadm_command_timeout": { 2024-03-07T07:43:33.864 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "default_cephadm_command_timeout", 2024-03-07T07:43:33.864 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.864 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.864 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.864 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "900", 2024-03-07T07:43:33.864 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.864 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.865 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.865 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-07T07:43:33.865 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.865 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.865 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.865 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.865 INFO:teuthology.orchestra.run.smithi160.stdout: "default_registry": { 2024-03-07T07:43:33.865 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "default_registry", 2024-03-07T07:43:33.865 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.865 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.865 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.866 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "docker.io", 2024-03-07T07:43:33.866 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.866 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.866 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.866 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-07T07:43:33.866 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.866 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.866 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.866 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.866 INFO:teuthology.orchestra.run.smithi160.stdout: "device_cache_timeout": { 2024-03-07T07:43:33.866 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "device_cache_timeout", 2024-03-07T07:43:33.866 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:33.867 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.867 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.867 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "1800", 2024-03-07T07:43:33.867 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.867 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.867 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.867 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "seconds to cache device inventory", 2024-03-07T07:43:33.867 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.867 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.867 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.867 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.867 INFO:teuthology.orchestra.run.smithi160.stdout: "device_enhanced_scan": { 2024-03-07T07:43:33.867 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "device_enhanced_scan", 2024-03-07T07:43:33.868 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.868 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.868 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.868 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.868 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.868 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.868 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.868 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-07T07:43:33.868 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.868 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.868 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.868 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.869 INFO:teuthology.orchestra.run.smithi160.stdout: "facts_cache_timeout": { 2024-03-07T07:43:33.869 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "facts_cache_timeout", 2024-03-07T07:43:33.869 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:33.869 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.869 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.869 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "60", 2024-03-07T07:43:33.869 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.869 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.869 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.869 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "seconds to cache host facts data", 2024-03-07T07:43:33.869 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.869 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.869 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.870 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.870 INFO:teuthology.orchestra.run.smithi160.stdout: "host_check_interval": { 2024-03-07T07:43:33.870 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "host_check_interval", 2024-03-07T07:43:33.870 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:33.870 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.870 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.870 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "600", 2024-03-07T07:43:33.870 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.870 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.870 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.870 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "how frequently to perform a host check", 2024-03-07T07:43:33.870 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.870 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.870 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.871 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.871 INFO:teuthology.orchestra.run.smithi160.stdout: "hw_monitoring": { 2024-03-07T07:43:33.871 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "hw_monitoring", 2024-03-07T07:43:33.871 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.871 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.871 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.871 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.871 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.871 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.871 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.871 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-07T07:43:33.871 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.872 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.872 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.872 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.872 INFO:teuthology.orchestra.run.smithi160.stdout: "inventory_list_all": { 2024-03-07T07:43:33.872 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "inventory_list_all", 2024-03-07T07:43:33.872 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.872 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.872 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.872 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.872 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.872 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.873 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.873 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-07T07:43:33.873 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.873 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.873 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.873 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.873 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.873 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.873 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.873 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.873 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.874 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.874 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.874 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.874 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.874 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.874 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.874 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.874 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.874 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.874 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.874 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.874 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.875 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.875 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.875 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.875 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.875 INFO:teuthology.orchestra.run.smithi160.stdout: "log_refresh_metadata": { 2024-03-07T07:43:33.875 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_refresh_metadata", 2024-03-07T07:43:33.875 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.875 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.875 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.875 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.875 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.876 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.876 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.876 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-07T07:43:33.876 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.876 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.876 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.876 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.876 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.876 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.876 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.876 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.876 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.877 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.877 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.877 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.877 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.877 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-07T07:43:33.877 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.877 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.877 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.877 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.877 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.877 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.877 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.878 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.878 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.878 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.878 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.878 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.878 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.878 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.878 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.878 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.879 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.879 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.879 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.879 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.879 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.879 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.879 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.879 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.879 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.879 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.879 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.879 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.879 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.880 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.880 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.880 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.880 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.880 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.880 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.880 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.880 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.880 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.880 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.880 INFO:teuthology.orchestra.run.smithi160.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-07T07:43:33.881 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-07T07:43:33.881 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.881 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.881 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.881 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.881 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.881 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.881 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.881 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-07T07:43:33.881 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.881 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.881 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.882 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.882 INFO:teuthology.orchestra.run.smithi160.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-07T07:43:33.882 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-07T07:43:33.882 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.882 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.882 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.882 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "*", 2024-03-07T07:43:33.882 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.882 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.882 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.882 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-07T07:43:33.882 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.883 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.883 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.883 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.883 INFO:teuthology.orchestra.run.smithi160.stdout: "max_count_per_host": { 2024-03-07T07:43:33.883 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "max_count_per_host", 2024-03-07T07:43:33.883 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.883 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.883 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.883 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "10", 2024-03-07T07:43:33.883 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.883 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.883 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.884 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "max number of daemons per service per host", 2024-03-07T07:43:33.884 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.884 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.884 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.884 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.884 INFO:teuthology.orchestra.run.smithi160.stdout: "max_osd_draining_count": { 2024-03-07T07:43:33.884 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "max_osd_draining_count", 2024-03-07T07:43:33.884 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.884 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.884 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.884 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "10", 2024-03-07T07:43:33.884 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.885 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.885 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.885 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-07T07:43:33.885 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.885 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.885 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.885 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.885 INFO:teuthology.orchestra.run.smithi160.stdout: "migration_current": { 2024-03-07T07:43:33.885 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "migration_current", 2024-03-07T07:43:33.885 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.885 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.885 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.886 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.886 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.886 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.886 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.886 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "internal - do not modify", 2024-03-07T07:43:33.886 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.886 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.886 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.886 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.886 INFO:teuthology.orchestra.run.smithi160.stdout: "mode": { 2024-03-07T07:43:33.886 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "mode", 2024-03-07T07:43:33.886 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.887 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.887 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.887 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "root", 2024-03-07T07:43:33.887 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.887 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.887 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.887 INFO:teuthology.orchestra.run.smithi160.stdout: "cephadm-package", 2024-03-07T07:43:33.887 INFO:teuthology.orchestra.run.smithi160.stdout: "root" 2024-03-07T07:43:33.887 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.887 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "mode for remote execution of cephadm", 2024-03-07T07:43:33.887 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.887 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.888 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.888 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.888 INFO:teuthology.orchestra.run.smithi160.stdout: "oob_default_addr": { 2024-03-07T07:43:33.888 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "oob_default_addr", 2024-03-07T07:43:33.888 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.888 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.888 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.888 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "169.254.1.1", 2024-03-07T07:43:33.888 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.888 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.888 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.888 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-07T07:43:33.889 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.889 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.889 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.889 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.889 INFO:teuthology.orchestra.run.smithi160.stdout: "prometheus_alerts_path": { 2024-03-07T07:43:33.889 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "prometheus_alerts_path", 2024-03-07T07:43:33.889 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.889 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.889 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.889 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-07T07:43:33.889 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.890 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.890 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.890 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-07T07:43:33.890 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.890 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.890 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.890 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.890 INFO:teuthology.orchestra.run.smithi160.stdout: "registry_insecure": { 2024-03-07T07:43:33.890 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "registry_insecure", 2024-03-07T07:43:33.890 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.890 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.891 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.891 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.891 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.891 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.891 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.891 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-07T07:43:33.891 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.891 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.891 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.891 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.891 INFO:teuthology.orchestra.run.smithi160.stdout: "registry_password": { 2024-03-07T07:43:33.891 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "registry_password", 2024-03-07T07:43:33.892 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.892 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.892 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.892 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.892 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.892 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.892 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.892 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-07T07:43:33.892 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.892 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.893 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.893 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.893 INFO:teuthology.orchestra.run.smithi160.stdout: "registry_url": { 2024-03-07T07:43:33.893 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "registry_url", 2024-03-07T07:43:33.893 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.893 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.893 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.893 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.893 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.893 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.893 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.894 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-07T07:43:33.894 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.894 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.894 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.894 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.894 INFO:teuthology.orchestra.run.smithi160.stdout: "registry_username": { 2024-03-07T07:43:33.894 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "registry_username", 2024-03-07T07:43:33.894 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.894 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.894 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.895 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.895 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.895 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.895 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.895 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-07T07:43:33.895 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.895 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.895 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.895 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.895 INFO:teuthology.orchestra.run.smithi160.stdout: "secure_monitoring_stack": { 2024-03-07T07:43:33.895 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "secure_monitoring_stack", 2024-03-07T07:43:33.895 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.896 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.896 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.896 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.896 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.896 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.896 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.896 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-07T07:43:33.896 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.896 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.896 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.896 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.896 INFO:teuthology.orchestra.run.smithi160.stdout: "service_discovery_port": { 2024-03-07T07:43:33.897 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "service_discovery_port", 2024-03-07T07:43:33.897 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.897 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.897 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.897 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "8765", 2024-03-07T07:43:33.897 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.897 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.897 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.897 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "cephadm service discovery port", 2024-03-07T07:43:33.897 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.897 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.897 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.897 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.898 INFO:teuthology.orchestra.run.smithi160.stdout: "ssh_config_file": { 2024-03-07T07:43:33.898 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "ssh_config_file", 2024-03-07T07:43:33.898 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.898 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.898 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.898 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.898 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.898 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.898 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.898 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-07T07:43:33.898 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.899 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.899 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.899 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.899 INFO:teuthology.orchestra.run.smithi160.stdout: "use_agent": { 2024-03-07T07:43:33.899 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "use_agent", 2024-03-07T07:43:33.899 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.899 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.899 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.899 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.899 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.899 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.899 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.899 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-07T07:43:33.900 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.900 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.900 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.900 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.900 INFO:teuthology.orchestra.run.smithi160.stdout: "use_repo_digest": { 2024-03-07T07:43:33.900 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "use_repo_digest", 2024-03-07T07:43:33.900 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.900 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.900 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.900 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.900 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.901 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.901 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.901 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-07T07:43:33.901 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.901 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.901 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.901 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.901 INFO:teuthology.orchestra.run.smithi160.stdout: "warn_on_failed_host_check": { 2024-03-07T07:43:33.901 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "warn_on_failed_host_check", 2024-03-07T07:43:33.902 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.902 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.902 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.902 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.902 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.902 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.902 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.902 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "raise a health warning if the host check fails", 2024-03-07T07:43:33.902 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.902 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.902 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.903 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.903 INFO:teuthology.orchestra.run.smithi160.stdout: "warn_on_stray_daemons": { 2024-03-07T07:43:33.903 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "warn_on_stray_daemons", 2024-03-07T07:43:33.903 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.903 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.903 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.903 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.903 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.903 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.903 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.904 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-07T07:43:33.904 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.904 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.904 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.904 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.904 INFO:teuthology.orchestra.run.smithi160.stdout: "warn_on_stray_hosts": { 2024-03-07T07:43:33.904 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "warn_on_stray_hosts", 2024-03-07T07:43:33.904 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.904 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.904 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.904 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.905 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.905 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.905 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.905 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-07T07:43:33.905 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.905 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.905 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.905 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.905 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.905 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.906 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.906 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "crash", 2024-03-07T07:43:33.906 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.906 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.906 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.906 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.906 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.906 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.906 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.906 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.906 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.907 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.907 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.907 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.907 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.907 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.907 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.907 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.907 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.907 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.907 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.907 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.908 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.908 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.908 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.908 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.908 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.908 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.908 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.908 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.908 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.909 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.909 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.909 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.909 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.909 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.909 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.909 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.909 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.909 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.909 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.909 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.910 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.910 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.910 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.910 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.910 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.910 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.910 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.910 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.910 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.910 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.910 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.910 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.910 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.911 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.911 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.911 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.911 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.911 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.911 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.911 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.911 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.911 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.911 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.911 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.912 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.912 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.912 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.912 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.912 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.912 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.912 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.912 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.912 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.912 INFO:teuthology.orchestra.run.smithi160.stdout: "retain_interval": { 2024-03-07T07:43:33.913 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "retain_interval", 2024-03-07T07:43:33.913 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:33.913 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.913 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.913 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "31536000", 2024-03-07T07:43:33.913 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.913 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.913 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.913 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-07T07:43:33.913 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.913 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.913 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.913 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.914 INFO:teuthology.orchestra.run.smithi160.stdout: "warn_recent_interval": { 2024-03-07T07:43:33.914 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "warn_recent_interval", 2024-03-07T07:43:33.914 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:33.914 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.914 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.914 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "1209600", 2024-03-07T07:43:33.914 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.914 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.914 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.914 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-07T07:43:33.914 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.914 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.915 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.915 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.915 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.915 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.915 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.915 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "dashboard", 2024-03-07T07:43:33.915 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.915 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.915 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.915 INFO:teuthology.orchestra.run.smithi160.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-07T07:43:33.915 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-07T07:43:33.915 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.916 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.916 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.916 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "10", 2024-03-07T07:43:33.916 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.916 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.916 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.916 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.916 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.916 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.916 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.916 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.916 INFO:teuthology.orchestra.run.smithi160.stdout: "ALERTMANAGER_API_HOST": { 2024-03-07T07:43:33.917 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-07T07:43:33.917 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.917 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.917 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.917 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.917 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.917 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.917 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.917 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.917 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.917 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.917 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.918 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.918 INFO:teuthology.orchestra.run.smithi160.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-07T07:43:33.918 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-07T07:43:33.918 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.918 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.918 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.918 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.918 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.918 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.918 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.918 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.918 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.919 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.919 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.919 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.919 INFO:teuthology.orchestra.run.smithi160.stdout: "AUDIT_API_ENABLED": { 2024-03-07T07:43:33.919 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "AUDIT_API_ENABLED", 2024-03-07T07:43:33.919 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.919 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.919 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.919 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.919 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.919 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.919 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.919 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.920 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.920 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.920 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.920 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.920 INFO:teuthology.orchestra.run.smithi160.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-07T07:43:33.920 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-07T07:43:33.920 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.920 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.920 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.920 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.920 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.920 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.921 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.921 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.921 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.921 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.921 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.921 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.921 INFO:teuthology.orchestra.run.smithi160.stdout: "ENABLE_BROWSABLE_API": { 2024-03-07T07:43:33.921 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-07T07:43:33.921 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.921 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.921 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.921 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.922 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.922 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.922 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.922 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.922 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.922 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.922 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.922 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.922 INFO:teuthology.orchestra.run.smithi160.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-07T07:43:33.922 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-07T07:43:33.922 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.922 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.922 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.922 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.923 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.923 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.923 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.923 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.923 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.923 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.923 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.923 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.923 INFO:teuthology.orchestra.run.smithi160.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-07T07:43:33.923 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-07T07:43:33.923 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.923 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.923 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.924 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.924 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.924 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.924 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.924 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.924 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.924 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.924 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.924 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.924 INFO:teuthology.orchestra.run.smithi160.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-07T07:43:33.924 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-07T07:43:33.925 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.925 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.925 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.925 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.925 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.925 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.925 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.925 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.925 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.925 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.925 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.925 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.926 INFO:teuthology.orchestra.run.smithi160.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-07T07:43:33.926 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-07T07:43:33.926 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.926 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.926 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.926 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.926 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.926 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.926 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.926 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.926 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.927 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.927 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.927 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.927 INFO:teuthology.orchestra.run.smithi160.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-07T07:43:33.927 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-07T07:43:33.927 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.927 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.927 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.927 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.927 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.927 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.928 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.928 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.928 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.928 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.928 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.928 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.928 INFO:teuthology.orchestra.run.smithi160.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-07T07:43:33.928 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-07T07:43:33.928 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.928 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.928 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.929 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.929 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.929 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.929 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.929 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.929 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.929 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.929 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.929 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.929 INFO:teuthology.orchestra.run.smithi160.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-07T07:43:33.929 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-07T07:43:33.929 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.929 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.930 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.930 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.930 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.930 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.930 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.930 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.930 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.930 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.930 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.930 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.930 INFO:teuthology.orchestra.run.smithi160.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-07T07:43:33.930 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-07T07:43:33.931 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.931 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.931 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.931 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.931 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.931 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.931 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.931 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.931 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.931 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.931 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.931 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.932 INFO:teuthology.orchestra.run.smithi160.stdout: "GRAFANA_API_PASSWORD": { 2024-03-07T07:43:33.932 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-07T07:43:33.932 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.932 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.932 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.932 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "admin", 2024-03-07T07:43:33.932 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.932 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.932 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.932 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.932 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.933 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.933 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.933 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.933 INFO:teuthology.orchestra.run.smithi160.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-07T07:43:33.933 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-07T07:43:33.933 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.933 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.933 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.933 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.933 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.933 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.933 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.933 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.934 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.934 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.934 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.934 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.934 INFO:teuthology.orchestra.run.smithi160.stdout: "GRAFANA_API_URL": { 2024-03-07T07:43:33.934 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "GRAFANA_API_URL", 2024-03-07T07:43:33.934 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.934 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.934 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.934 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.934 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.934 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.934 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.935 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.935 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.935 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.935 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.935 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.935 INFO:teuthology.orchestra.run.smithi160.stdout: "GRAFANA_API_USERNAME": { 2024-03-07T07:43:33.935 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-07T07:43:33.935 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.935 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.935 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.935 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "admin", 2024-03-07T07:43:33.935 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.935 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.936 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.936 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.936 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.936 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.936 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.936 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.936 INFO:teuthology.orchestra.run.smithi160.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-07T07:43:33.936 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-07T07:43:33.936 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.936 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.936 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.937 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.937 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.937 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.937 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.937 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.937 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.937 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.937 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.937 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.937 INFO:teuthology.orchestra.run.smithi160.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-07T07:43:33.937 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-07T07:43:33.938 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.938 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.938 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.938 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.938 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.938 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.938 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.938 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.938 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.938 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.938 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.938 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.939 INFO:teuthology.orchestra.run.smithi160.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-07T07:43:33.939 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-07T07:43:33.939 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.939 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.939 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.939 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.939 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.939 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.939 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.939 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.939 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.939 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.940 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.940 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.940 INFO:teuthology.orchestra.run.smithi160.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-07T07:43:33.940 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-07T07:43:33.940 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.940 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.940 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.940 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.940 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.940 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.940 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.940 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.941 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.941 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.941 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.941 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.941 INFO:teuthology.orchestra.run.smithi160.stdout: "PROMETHEUS_API_HOST": { 2024-03-07T07:43:33.941 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-07T07:43:33.941 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.941 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.941 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.941 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.941 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.942 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.942 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.942 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.942 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.942 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.942 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.942 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.942 INFO:teuthology.orchestra.run.smithi160.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-07T07:43:33.942 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-07T07:43:33.942 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.942 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.942 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.943 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.943 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.943 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.943 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.943 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.943 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.943 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.943 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.943 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.943 INFO:teuthology.orchestra.run.smithi160.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-07T07:43:33.943 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-07T07:43:33.944 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.944 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.944 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.944 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.944 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.944 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.944 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.944 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.944 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.944 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.944 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.944 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.945 INFO:teuthology.orchestra.run.smithi160.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-07T07:43:33.945 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-07T07:43:33.945 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.945 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.945 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.945 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.945 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.945 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.945 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.945 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.945 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.946 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.946 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.946 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.946 INFO:teuthology.orchestra.run.smithi160.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-07T07:43:33.946 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-07T07:43:33.946 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.946 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.946 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.946 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.946 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.947 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.947 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.947 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.947 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.947 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.947 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.947 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.947 INFO:teuthology.orchestra.run.smithi160.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-07T07:43:33.947 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-07T07:43:33.947 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.947 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.947 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.948 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.948 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.948 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.948 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.948 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.948 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.948 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.948 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.948 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.948 INFO:teuthology.orchestra.run.smithi160.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-07T07:43:33.948 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-07T07:43:33.948 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.949 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.949 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.949 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.949 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.949 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.949 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.949 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.949 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.949 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.949 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.949 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.950 INFO:teuthology.orchestra.run.smithi160.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-07T07:43:33.950 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-07T07:43:33.950 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.950 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.950 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.950 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.950 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.950 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.950 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.950 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.950 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.951 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.951 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.951 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.951 INFO:teuthology.orchestra.run.smithi160.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-07T07:43:33.951 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-07T07:43:33.951 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.951 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.951 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.951 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.951 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.951 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.952 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.952 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.952 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.952 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.952 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.952 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.952 INFO:teuthology.orchestra.run.smithi160.stdout: "PWD_POLICY_ENABLED": { 2024-03-07T07:43:33.952 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-07T07:43:33.952 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.952 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.952 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.953 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.953 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.953 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.953 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.953 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.953 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.953 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.953 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.953 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.953 INFO:teuthology.orchestra.run.smithi160.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-07T07:43:33.954 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-07T07:43:33.954 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.954 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.954 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.954 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-07T07:43:33.954 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.954 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.954 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.954 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.954 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.954 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.954 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.954 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.955 INFO:teuthology.orchestra.run.smithi160.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-07T07:43:33.955 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-07T07:43:33.955 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.955 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.955 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.955 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "10", 2024-03-07T07:43:33.955 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.955 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.955 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.955 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.956 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.956 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.956 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.956 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.956 INFO:teuthology.orchestra.run.smithi160.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-07T07:43:33.956 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-07T07:43:33.956 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.956 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.956 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.956 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "8", 2024-03-07T07:43:33.956 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.957 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.957 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.957 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.957 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.957 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.957 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.957 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.957 INFO:teuthology.orchestra.run.smithi160.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-07T07:43:33.957 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-07T07:43:33.957 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.957 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.957 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.957 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "45", 2024-03-07T07:43:33.958 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.958 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.958 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.958 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.958 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.958 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.958 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.958 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.958 INFO:teuthology.orchestra.run.smithi160.stdout: "RGW_API_ACCESS_KEY": { 2024-03-07T07:43:33.958 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-07T07:43:33.959 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.959 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.959 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.959 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.959 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.959 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.959 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.959 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.959 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.959 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.959 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.959 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.959 INFO:teuthology.orchestra.run.smithi160.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-07T07:43:33.960 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-07T07:43:33.960 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.960 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.960 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.960 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "admin", 2024-03-07T07:43:33.960 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.960 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.960 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.960 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.960 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.960 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.961 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.961 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.961 INFO:teuthology.orchestra.run.smithi160.stdout: "RGW_API_SECRET_KEY": { 2024-03-07T07:43:33.961 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-07T07:43:33.961 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.961 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.961 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.961 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.961 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.961 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.961 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.961 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.962 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.962 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.962 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.962 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.962 INFO:teuthology.orchestra.run.smithi160.stdout: "RGW_API_SSL_VERIFY": { 2024-03-07T07:43:33.962 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-07T07:43:33.962 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.962 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.962 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.962 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.962 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.962 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.963 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.963 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.963 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.963 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.963 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.963 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.963 INFO:teuthology.orchestra.run.smithi160.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-07T07:43:33.963 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-07T07:43:33.964 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.964 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.964 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.964 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "0", 2024-03-07T07:43:33.964 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.964 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.964 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.964 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.964 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.964 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.964 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.964 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.965 INFO:teuthology.orchestra.run.smithi160.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-07T07:43:33.965 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-07T07:43:33.965 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.965 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.965 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.965 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "10", 2024-03-07T07:43:33.965 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.965 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.965 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.965 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.966 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.966 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.966 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.966 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.966 INFO:teuthology.orchestra.run.smithi160.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-07T07:43:33.966 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-07T07:43:33.966 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.966 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.966 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.966 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "5", 2024-03-07T07:43:33.966 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.966 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.967 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.967 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.967 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.967 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.967 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.967 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.967 INFO:teuthology.orchestra.run.smithi160.stdout: "cross_origin_url": { 2024-03-07T07:43:33.967 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "cross_origin_url", 2024-03-07T07:43:33.967 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.967 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.968 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.968 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.968 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.968 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.968 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.968 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.968 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.968 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.968 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.968 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.969 INFO:teuthology.orchestra.run.smithi160.stdout: "crt_file": { 2024-03-07T07:43:33.969 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "crt_file", 2024-03-07T07:43:33.969 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.969 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.969 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.969 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.969 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.969 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.969 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.969 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.969 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.969 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.969 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.970 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.970 INFO:teuthology.orchestra.run.smithi160.stdout: "debug": { 2024-03-07T07:43:33.970 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "debug", 2024-03-07T07:43:33.970 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.970 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.970 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.970 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.970 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.970 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.970 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.970 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Enable/disable debug options", 2024-03-07T07:43:33.970 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.971 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.971 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.971 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.971 INFO:teuthology.orchestra.run.smithi160.stdout: "jwt_token_ttl": { 2024-03-07T07:43:33.971 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "jwt_token_ttl", 2024-03-07T07:43:33.971 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.971 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.971 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.971 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "28800", 2024-03-07T07:43:33.971 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.971 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.971 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.972 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.972 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.972 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.972 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.972 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.972 INFO:teuthology.orchestra.run.smithi160.stdout: "key_file": { 2024-03-07T07:43:33.972 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "key_file", 2024-03-07T07:43:33.972 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.972 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.972 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.972 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.973 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.973 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.973 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.973 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.973 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.973 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.973 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.973 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.973 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.973 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.973 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.973 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.973 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.974 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.974 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.974 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.974 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.974 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.974 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.974 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.974 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.974 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.974 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.974 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.974 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.975 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.975 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.975 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.975 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.975 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.975 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.975 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.975 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.975 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.975 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.975 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.975 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.975 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.976 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.976 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.976 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.976 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.976 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.976 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.976 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.976 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.976 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.976 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.976 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.977 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.977 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.977 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.977 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.977 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.977 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.977 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.977 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.977 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.977 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.978 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.978 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.978 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.978 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.978 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.978 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.978 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.978 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.978 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.978 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.978 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.979 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.979 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.979 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.979 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.979 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.979 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.979 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.979 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.979 INFO:teuthology.orchestra.run.smithi160.stdout: "motd": { 2024-03-07T07:43:33.979 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "motd", 2024-03-07T07:43:33.979 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.979 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.980 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.980 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.980 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.980 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.980 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.980 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "The message of the day", 2024-03-07T07:43:33.980 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.980 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.980 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.980 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.980 INFO:teuthology.orchestra.run.smithi160.stdout: "redirect_resolve_ip_addr": { 2024-03-07T07:43:33.980 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "redirect_resolve_ip_addr", 2024-03-07T07:43:33.981 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.981 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.981 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.981 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.981 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.981 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.981 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.981 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.981 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.981 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.981 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.981 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.982 INFO:teuthology.orchestra.run.smithi160.stdout: "server_addr": { 2024-03-07T07:43:33.982 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "server_addr", 2024-03-07T07:43:33.982 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.982 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.982 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.982 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "::", 2024-03-07T07:43:33.982 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.982 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.982 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.982 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.982 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.982 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.983 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.983 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.983 INFO:teuthology.orchestra.run.smithi160.stdout: "server_port": { 2024-03-07T07:43:33.983 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "server_port", 2024-03-07T07:43:33.983 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.983 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.983 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.983 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "8080", 2024-03-07T07:43:33.983 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.983 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.984 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.984 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.984 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.984 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.984 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.984 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.984 INFO:teuthology.orchestra.run.smithi160.stdout: "ssl": { 2024-03-07T07:43:33.984 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "ssl", 2024-03-07T07:43:33.984 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.984 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.984 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.984 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.985 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.985 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.985 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.985 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.985 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.985 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.985 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.985 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.985 INFO:teuthology.orchestra.run.smithi160.stdout: "ssl_server_port": { 2024-03-07T07:43:33.985 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "ssl_server_port", 2024-03-07T07:43:33.985 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.986 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.986 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.986 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "8443", 2024-03-07T07:43:33.986 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.986 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.986 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.986 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.986 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.986 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.986 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.986 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.986 INFO:teuthology.orchestra.run.smithi160.stdout: "standby_behaviour": { 2024-03-07T07:43:33.987 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "standby_behaviour", 2024-03-07T07:43:33.987 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.987 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.987 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.987 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "redirect", 2024-03-07T07:43:33.987 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.987 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.987 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.987 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.987 INFO:teuthology.orchestra.run.smithi160.stdout: "redirect" 2024-03-07T07:43:33.987 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.988 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.988 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.988 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.988 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.988 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.988 INFO:teuthology.orchestra.run.smithi160.stdout: "standby_error_status_code": { 2024-03-07T07:43:33.988 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "standby_error_status_code", 2024-03-07T07:43:33.988 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:33.988 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.988 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.988 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "500", 2024-03-07T07:43:33.989 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "400", 2024-03-07T07:43:33.989 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "599", 2024-03-07T07:43:33.989 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.989 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.989 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.989 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.989 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.989 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.989 INFO:teuthology.orchestra.run.smithi160.stdout: "url_prefix": { 2024-03-07T07:43:33.989 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "url_prefix", 2024-03-07T07:43:33.989 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.989 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.990 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:33.990 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.990 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.990 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.990 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.990 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.990 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.990 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.990 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.990 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.990 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:33.991 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.991 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:33.991 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "devicehealth", 2024-03-07T07:43:33.991 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:33.991 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:33.991 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:33.991 INFO:teuthology.orchestra.run.smithi160.stdout: "enable_monitoring": { 2024-03-07T07:43:33.991 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "enable_monitoring", 2024-03-07T07:43:33.991 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.991 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.991 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.992 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:33.992 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.992 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.992 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.992 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "monitor device health metrics", 2024-03-07T07:43:33.992 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.992 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.992 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.992 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.992 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:33.992 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:33.992 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.993 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.993 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.993 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:33.993 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.993 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.993 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.993 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.993 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.993 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.993 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.993 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.993 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.993 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.994 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.994 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.994 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.994 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.994 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.994 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:33.994 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:33.994 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.994 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.994 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.994 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.994 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.995 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.995 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.995 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.995 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.995 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.995 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.995 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.995 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:33.995 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:33.995 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.995 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.995 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.995 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:33.995 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.996 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.996 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:33.996 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:33.996 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:33.996 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:33.996 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:33.996 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:33.996 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:33.996 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:33.996 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.996 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.997 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.997 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.997 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.997 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:33.997 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:33.997 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:33.997 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.997 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.997 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:33.997 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.997 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.997 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.997 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:33.997 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.998 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.998 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.998 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.998 INFO:teuthology.orchestra.run.smithi160.stdout: "mark_out_threshold": { 2024-03-07T07:43:33.998 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "mark_out_threshold", 2024-03-07T07:43:33.998 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:33.998 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:33.998 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:33.998 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "2419200", 2024-03-07T07:43:33.998 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:33.999 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:33.999 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:33.999 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-07T07:43:33.999 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:33.999 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:33.999 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:33.999 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:33.999 INFO:teuthology.orchestra.run.smithi160.stdout: "pool_name": { 2024-03-07T07:43:33.999 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "pool_name", 2024-03-07T07:43:33.999 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:33.999 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.000 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.000 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "device_health_metrics", 2024-03-07T07:43:34.000 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.000 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.000 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.000 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-07T07:43:34.000 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.000 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.000 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.000 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.000 INFO:teuthology.orchestra.run.smithi160.stdout: "retention_period": { 2024-03-07T07:43:34.000 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "retention_period", 2024-03-07T07:43:34.000 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:34.001 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.001 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.001 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "15552000", 2024-03-07T07:43:34.001 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.001 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.001 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.001 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "how long to retain device health metrics", 2024-03-07T07:43:34.001 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.001 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.001 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.001 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.001 INFO:teuthology.orchestra.run.smithi160.stdout: "scrape_frequency": { 2024-03-07T07:43:34.002 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "scrape_frequency", 2024-03-07T07:43:34.002 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:34.002 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.002 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.002 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "86400", 2024-03-07T07:43:34.002 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.002 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.002 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.002 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-07T07:43:34.002 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.002 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.002 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.003 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.003 INFO:teuthology.orchestra.run.smithi160.stdout: "self_heal": { 2024-03-07T07:43:34.003 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "self_heal", 2024-03-07T07:43:34.003 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.003 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.003 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.003 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:34.003 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.003 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.003 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.003 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-07T07:43:34.004 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.004 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.004 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.004 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.004 INFO:teuthology.orchestra.run.smithi160.stdout: "sleep_interval": { 2024-03-07T07:43:34.004 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "sleep_interval", 2024-03-07T07:43:34.004 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:34.004 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.004 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.004 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "600", 2024-03-07T07:43:34.004 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.004 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.005 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.005 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "how frequently to wake up and check device health", 2024-03-07T07:43:34.005 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.005 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.005 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.005 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.005 INFO:teuthology.orchestra.run.smithi160.stdout: "warn_threshold": { 2024-03-07T07:43:34.005 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "warn_threshold", 2024-03-07T07:43:34.005 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:34.005 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.005 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.005 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "7257600", 2024-03-07T07:43:34.006 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.006 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.006 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.006 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-07T07:43:34.006 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.006 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.006 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.006 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.006 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.006 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.006 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:34.006 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "diskprediction_local", 2024-03-07T07:43:34.007 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:34.007 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:34.007 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:34.007 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:34.007 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:34.007 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.007 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.007 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.007 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.007 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.007 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.007 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.008 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.008 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.008 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.008 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.008 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.008 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.008 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.008 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.008 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.008 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.008 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.009 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.009 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:34.009 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:34.009 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.009 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.009 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.009 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.009 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.009 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.009 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.009 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.009 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.009 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.010 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.010 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.010 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:34.010 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:34.010 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.010 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.010 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.010 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:34.010 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.010 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.010 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.010 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.010 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.011 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.011 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.011 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.011 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.011 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.011 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.011 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.011 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.011 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.011 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.011 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:34.011 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:34.012 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.012 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.012 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.012 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.012 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.012 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.012 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.012 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.012 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.012 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.012 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.012 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.013 INFO:teuthology.orchestra.run.smithi160.stdout: "predict_interval": { 2024-03-07T07:43:34.013 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "predict_interval", 2024-03-07T07:43:34.013 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.013 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.013 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.013 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "86400", 2024-03-07T07:43:34.013 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.013 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.013 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.013 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.013 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.013 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.014 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.014 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.014 INFO:teuthology.orchestra.run.smithi160.stdout: "predictor_model": { 2024-03-07T07:43:34.014 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "predictor_model", 2024-03-07T07:43:34.014 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.014 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.014 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.014 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "prophetstor", 2024-03-07T07:43:34.014 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.014 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.014 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.014 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.015 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.015 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.015 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.015 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.015 INFO:teuthology.orchestra.run.smithi160.stdout: "sleep_interval": { 2024-03-07T07:43:34.015 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "sleep_interval", 2024-03-07T07:43:34.015 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.015 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.015 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.015 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "600", 2024-03-07T07:43:34.015 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.015 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.015 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.016 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.016 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.016 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.016 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.016 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.016 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.016 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.016 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:34.016 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "influx", 2024-03-07T07:43:34.016 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": false, 2024-03-07T07:43:34.016 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "influxdb python module not found", 2024-03-07T07:43:34.017 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:34.017 INFO:teuthology.orchestra.run.smithi160.stdout: "batch_size": { 2024-03-07T07:43:34.017 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "batch_size", 2024-03-07T07:43:34.017 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:34.017 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.017 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.017 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "5000", 2024-03-07T07:43:34.017 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.017 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.017 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.017 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-07T07:43:34.017 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.018 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.018 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.018 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.018 INFO:teuthology.orchestra.run.smithi160.stdout: "database": { 2024-03-07T07:43:34.018 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "database", 2024-03-07T07:43:34.018 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.018 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.018 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.018 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "ceph", 2024-03-07T07:43:34.018 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.018 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.018 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.019 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-07T07:43:34.019 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.019 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.019 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.019 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.019 INFO:teuthology.orchestra.run.smithi160.stdout: "hostname": { 2024-03-07T07:43:34.019 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "hostname", 2024-03-07T07:43:34.019 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.019 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.019 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.019 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.020 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.020 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.020 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.020 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "InfluxDB server hostname", 2024-03-07T07:43:34.020 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.020 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.020 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.020 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.020 INFO:teuthology.orchestra.run.smithi160.stdout: "interval": { 2024-03-07T07:43:34.020 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "interval", 2024-03-07T07:43:34.020 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:34.020 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.021 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.021 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "30", 2024-03-07T07:43:34.021 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "5", 2024-03-07T07:43:34.021 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.021 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.021 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-07T07:43:34.021 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.021 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.021 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.021 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.021 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:34.022 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:34.022 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.022 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.022 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.022 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.022 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.022 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.022 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.022 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.023 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.023 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.023 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.023 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.023 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.023 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.023 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.023 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.023 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.023 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.023 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.023 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:34.024 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:34.024 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.024 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.024 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.024 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.024 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.024 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.024 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.024 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.024 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.024 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.024 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.025 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.025 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:34.025 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:34.025 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.025 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.025 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.025 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:34.025 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.025 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.025 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.025 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.025 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.025 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.026 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.026 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.026 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.026 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.026 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.026 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.026 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.026 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.026 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.026 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:34.026 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:34.026 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.027 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.027 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.027 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.027 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.027 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.027 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.027 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.027 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.027 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.027 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.027 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.027 INFO:teuthology.orchestra.run.smithi160.stdout: "password": { 2024-03-07T07:43:34.028 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "password", 2024-03-07T07:43:34.028 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.028 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.028 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.028 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.028 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.028 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.028 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.028 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "password of InfluxDB server user", 2024-03-07T07:43:34.028 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.028 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.028 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.029 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.029 INFO:teuthology.orchestra.run.smithi160.stdout: "port": { 2024-03-07T07:43:34.029 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "port", 2024-03-07T07:43:34.029 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:34.029 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.029 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.029 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "8086", 2024-03-07T07:43:34.029 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.029 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.029 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.029 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "InfluxDB server port", 2024-03-07T07:43:34.029 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.030 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.030 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.030 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.030 INFO:teuthology.orchestra.run.smithi160.stdout: "ssl": { 2024-03-07T07:43:34.030 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "ssl", 2024-03-07T07:43:34.030 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.030 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.030 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.030 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "false", 2024-03-07T07:43:34.030 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.031 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.031 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.031 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-07T07:43:34.031 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.031 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.031 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.031 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.031 INFO:teuthology.orchestra.run.smithi160.stdout: "threads": { 2024-03-07T07:43:34.031 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "threads", 2024-03-07T07:43:34.031 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:34.031 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.031 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.032 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "5", 2024-03-07T07:43:34.032 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "1", 2024-03-07T07:43:34.032 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "32", 2024-03-07T07:43:34.032 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.032 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-07T07:43:34.032 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.032 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.032 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.032 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.032 INFO:teuthology.orchestra.run.smithi160.stdout: "username": { 2024-03-07T07:43:34.032 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "username", 2024-03-07T07:43:34.033 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.033 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.033 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.033 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.033 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.033 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.033 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.033 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "username of InfluxDB server user", 2024-03-07T07:43:34.033 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.033 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.033 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.033 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.033 INFO:teuthology.orchestra.run.smithi160.stdout: "verify_ssl": { 2024-03-07T07:43:34.034 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "verify_ssl", 2024-03-07T07:43:34.034 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.034 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.034 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.034 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "true", 2024-03-07T07:43:34.034 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.034 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.034 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.034 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-07T07:43:34.034 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.034 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.034 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.035 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.035 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.035 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.035 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:34.035 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "insights", 2024-03-07T07:43:34.035 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:34.035 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:34.035 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:34.035 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:34.035 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:34.035 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.035 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.035 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.036 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.036 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.036 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.036 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.036 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.036 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.036 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.036 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.036 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.036 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.036 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.036 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.037 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.037 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.037 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.037 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.037 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:34.037 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:34.037 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.037 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.037 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.037 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.037 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.037 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.037 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.038 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.038 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.038 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.038 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.038 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.038 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:34.038 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:34.038 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.038 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.038 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.038 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:34.038 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.039 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.039 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.039 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.039 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.039 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.039 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.039 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.039 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.039 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.039 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.039 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.039 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.039 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.040 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.040 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:34.040 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:34.040 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.040 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.040 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.040 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.040 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.040 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.041 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.041 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.041 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.041 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.041 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.041 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.041 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.041 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.041 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:34.041 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "iostat", 2024-03-07T07:43:34.041 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:34.041 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:34.042 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:34.042 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:34.042 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:34.042 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.042 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.042 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.042 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.042 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.042 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.043 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.043 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.043 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.043 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.043 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.043 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.043 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.043 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.043 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.043 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.043 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.043 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.044 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.044 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:34.044 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:34.044 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.044 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.044 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.044 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.044 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.044 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.044 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.045 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.045 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.045 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.045 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.045 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.045 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:34.045 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:34.045 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.045 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.045 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.046 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:34.046 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.046 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.046 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.046 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.046 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.046 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.046 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.046 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.046 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.047 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.047 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.047 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.047 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.047 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.047 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.047 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:34.047 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:34.047 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.047 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.047 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.048 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.048 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.048 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.048 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.048 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.048 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.048 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.048 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.048 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.048 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.049 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.049 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:34.049 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "k8sevents", 2024-03-07T07:43:34.049 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:34.049 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:34.049 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:34.049 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_event_retention_days": { 2024-03-07T07:43:34.049 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "ceph_event_retention_days", 2024-03-07T07:43:34.049 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:34.049 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.050 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.050 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "7", 2024-03-07T07:43:34.050 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.050 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.050 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.050 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-07T07:43:34.050 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.050 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.050 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.050 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.051 INFO:teuthology.orchestra.run.smithi160.stdout: "config_check_secs": { 2024-03-07T07:43:34.051 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "config_check_secs", 2024-03-07T07:43:34.051 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:34.051 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.051 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.051 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "10", 2024-03-07T07:43:34.051 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "10", 2024-03-07T07:43:34.051 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.051 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.051 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-07T07:43:34.052 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.052 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.052 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.052 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.052 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:34.052 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:34.052 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.052 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.052 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.052 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.052 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.052 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.053 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.053 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.053 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.053 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.053 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.053 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.053 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.053 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.053 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.053 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.053 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.054 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.054 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.054 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:34.054 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:34.054 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.054 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.054 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.054 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.054 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.054 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.054 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.054 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.055 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.055 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.055 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.055 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.055 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:34.055 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:34.055 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.055 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.055 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.055 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:34.055 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.055 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.056 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.056 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.056 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.056 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.056 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.056 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.056 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.056 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.056 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.056 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.056 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.057 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.057 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.057 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:34.057 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:34.057 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.057 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.057 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.057 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.057 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.057 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.057 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.057 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.057 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.058 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.058 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.058 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.058 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.058 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.058 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:34.058 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "localpool", 2024-03-07T07:43:34.058 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:34.058 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:34.058 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:34.058 INFO:teuthology.orchestra.run.smithi160.stdout: "failure_domain": { 2024-03-07T07:43:34.059 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "failure_domain", 2024-03-07T07:43:34.059 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.059 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.059 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.059 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "host", 2024-03-07T07:43:34.059 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.059 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.059 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.059 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "failure domain for any created local pool", 2024-03-07T07:43:34.059 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-07T07:43:34.059 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.060 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.060 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.060 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:34.060 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:34.060 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.060 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.060 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.060 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.060 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.060 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.060 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.060 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.061 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.061 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.061 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.061 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.061 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.061 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.061 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.061 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.061 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.061 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.061 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.061 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:34.062 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:34.062 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.062 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.062 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.062 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.062 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.062 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.062 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.062 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.063 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.063 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.063 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.063 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.063 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:34.063 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:34.063 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.063 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.063 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.063 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:34.063 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.064 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.064 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.064 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.064 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.064 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.064 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.064 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.064 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.064 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.064 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.065 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.065 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.065 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.065 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.065 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:34.065 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:34.065 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.065 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.065 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.065 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.066 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.066 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.066 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.066 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.066 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.066 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.066 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.066 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.066 INFO:teuthology.orchestra.run.smithi160.stdout: "min_size": { 2024-03-07T07:43:34.066 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "min_size", 2024-03-07T07:43:34.067 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:34.067 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.067 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.067 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.067 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.067 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.067 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.067 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "default min_size for any created local pool", 2024-03-07T07:43:34.068 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-07T07:43:34.068 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.068 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.068 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.068 INFO:teuthology.orchestra.run.smithi160.stdout: "num_rep": { 2024-03-07T07:43:34.068 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "num_rep", 2024-03-07T07:43:34.068 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:34.068 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.068 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.068 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "3", 2024-03-07T07:43:34.068 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.068 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.069 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.069 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "default replica count for any created local pool", 2024-03-07T07:43:34.069 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.069 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.069 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.069 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.069 INFO:teuthology.orchestra.run.smithi160.stdout: "pg_num": { 2024-03-07T07:43:34.069 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "pg_num", 2024-03-07T07:43:34.069 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:34.069 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.069 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.069 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "128", 2024-03-07T07:43:34.070 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.070 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.070 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.070 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "default pg_num for any created local pool", 2024-03-07T07:43:34.070 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.070 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.070 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.070 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.070 INFO:teuthology.orchestra.run.smithi160.stdout: "prefix": { 2024-03-07T07:43:34.070 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "prefix", 2024-03-07T07:43:34.070 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.071 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.071 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.071 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.071 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.071 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.071 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.071 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "name prefix for any created local pool", 2024-03-07T07:43:34.071 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.071 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.071 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.071 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.071 INFO:teuthology.orchestra.run.smithi160.stdout: "subtree": { 2024-03-07T07:43:34.072 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "subtree", 2024-03-07T07:43:34.072 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.072 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.072 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.072 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "rack", 2024-03-07T07:43:34.072 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.072 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.072 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.072 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-07T07:43:34.072 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-07T07:43:34.072 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.073 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.073 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.073 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.073 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.073 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:34.073 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "mds_autoscaler", 2024-03-07T07:43:34.073 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:34.073 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:34.073 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:34.073 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:34.073 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:34.073 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.074 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.074 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.074 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.074 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.074 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.074 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.074 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.074 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.074 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.074 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.074 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.074 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.074 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.075 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.075 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.075 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.075 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.075 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.075 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:34.075 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:34.075 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.075 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.075 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.075 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.075 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.075 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.075 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.076 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.076 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.076 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.076 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.076 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.076 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:34.076 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:34.076 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.076 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.076 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.076 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:34.076 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.076 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.077 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.077 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.077 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.077 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.077 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.077 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.077 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.077 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.077 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.077 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.077 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.078 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.078 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.078 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:34.078 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:34.078 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.078 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.078 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.078 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.078 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.078 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.078 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.079 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.079 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.079 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.079 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.079 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.079 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.079 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.079 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:34.079 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "mirroring", 2024-03-07T07:43:34.079 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:34.079 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:34.079 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:34.079 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:34.080 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:34.080 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.080 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.080 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.080 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.080 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.080 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.080 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.080 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.080 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.080 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.080 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.081 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.081 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.081 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.081 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.081 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.081 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.081 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.081 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.081 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:34.081 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:34.081 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.081 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.082 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.082 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.082 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.082 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.082 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.082 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.082 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.082 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.082 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.082 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.082 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:34.082 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:34.083 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.083 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.083 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.083 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:34.083 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.083 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.083 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.083 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.083 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.083 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.083 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.083 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.084 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.084 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.084 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.084 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.084 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.084 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.084 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.084 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:34.084 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:34.084 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.084 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.084 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.085 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.085 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.085 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.085 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.085 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.085 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.085 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.085 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.085 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.085 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.085 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.085 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:34.085 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "nfs", 2024-03-07T07:43:34.086 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:34.086 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:34.086 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:34.086 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:34.086 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:34.086 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.086 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.086 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.086 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.086 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.086 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.086 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.086 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.087 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.087 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.087 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.087 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.087 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.087 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.087 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.087 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.087 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.087 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.087 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.087 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:34.088 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:34.088 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.088 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.088 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.088 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.088 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.088 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.088 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.088 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.088 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.088 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.089 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.089 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.089 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:34.089 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:34.089 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.089 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.089 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.089 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:34.089 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.089 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.089 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.089 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.090 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.090 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.090 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.090 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.090 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.090 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.090 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.090 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.090 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.090 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.090 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.090 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:34.091 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:34.091 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.091 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.091 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.091 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.091 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.091 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.091 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.091 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.091 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.092 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.092 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.092 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.092 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.092 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.092 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:34.092 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "orchestrator", 2024-03-07T07:43:34.092 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:34.092 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:34.092 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:34.092 INFO:teuthology.orchestra.run.smithi160.stdout: "fail_fs": { 2024-03-07T07:43:34.092 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "fail_fs", 2024-03-07T07:43:34.093 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.093 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.093 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.093 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.093 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.093 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.093 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.093 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-07T07:43:34.093 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.093 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.093 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.093 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.094 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:34.094 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:34.094 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.094 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.094 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.094 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.094 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.094 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.094 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.094 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.094 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.094 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.095 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.095 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.095 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.095 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.095 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.095 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.095 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.095 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.095 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.095 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:34.096 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:34.096 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.096 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.096 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.096 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.096 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.096 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.096 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.096 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.096 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.096 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.096 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.097 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.097 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:34.097 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:34.097 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.097 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.097 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.097 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:34.097 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.097 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.097 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.097 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.098 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.098 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.098 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.098 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.098 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.098 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.098 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.098 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.098 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.098 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.098 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.098 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:34.099 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:34.099 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.099 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.099 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.099 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.099 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.099 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.099 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.099 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.099 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.099 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.099 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.100 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.100 INFO:teuthology.orchestra.run.smithi160.stdout: "orchestrator": { 2024-03-07T07:43:34.100 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "orchestrator", 2024-03-07T07:43:34.100 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.100 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.100 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.100 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.100 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.100 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.100 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.100 INFO:teuthology.orchestra.run.smithi160.stdout: "cephadm", 2024-03-07T07:43:34.101 INFO:teuthology.orchestra.run.smithi160.stdout: "rook", 2024-03-07T07:43:34.101 INFO:teuthology.orchestra.run.smithi160.stdout: "test_orchestrator" 2024-03-07T07:43:34.101 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.101 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Orchestrator backend", 2024-03-07T07:43:34.101 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.101 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.101 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.101 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.101 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.101 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.101 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:34.101 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "osd_perf_query", 2024-03-07T07:43:34.102 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:34.102 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:34.102 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:34.102 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:34.102 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:34.102 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.102 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.102 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.102 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.102 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.102 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.102 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.102 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.103 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.103 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.103 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.103 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.103 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.103 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.103 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.103 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.103 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.103 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.103 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.103 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:34.104 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:34.104 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.104 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.104 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.104 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.104 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.104 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.104 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.104 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.105 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.105 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.105 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.105 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.105 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:34.105 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:34.105 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.105 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.105 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.105 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:34.105 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.105 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.106 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.106 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.106 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.106 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.106 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.106 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.106 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.106 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.106 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.106 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.106 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.106 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.106 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.107 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:34.107 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:34.107 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.107 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.107 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.107 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.107 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.107 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.107 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.107 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.107 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.107 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.108 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.108 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.108 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.108 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.108 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:34.108 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "osd_support", 2024-03-07T07:43:34.108 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:34.108 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:34.108 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:34.108 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:34.109 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:34.109 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.109 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.109 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.109 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.109 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.109 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.109 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.109 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.109 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.109 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.109 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.110 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.110 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.110 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.110 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.110 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.110 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.110 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.110 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.110 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:34.110 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:34.110 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.110 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.111 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.111 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.111 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.111 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.111 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.111 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.111 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.111 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.111 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.111 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.111 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:34.111 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:34.112 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.112 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.112 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.112 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:34.112 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.112 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.112 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.112 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.112 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.112 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.112 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.112 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.113 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.113 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.113 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.113 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.113 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.113 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.113 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.113 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:34.113 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:34.113 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.113 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.113 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.114 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.114 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.114 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.114 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.114 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.114 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.114 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.114 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.114 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.114 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.114 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.114 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:34.115 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "pg_autoscaler", 2024-03-07T07:43:34.115 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:34.115 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:34.115 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:34.115 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:34.115 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:34.115 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.115 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.115 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.115 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.115 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.115 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.116 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.116 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.116 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.116 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.116 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.116 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.116 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.116 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.116 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.116 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.116 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.116 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.117 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.117 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:34.117 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:34.117 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.117 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.117 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.117 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.117 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.117 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.117 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.117 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.118 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.118 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.118 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.118 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.118 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:34.118 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:34.118 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.118 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.118 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.118 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:34.118 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.119 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.119 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.119 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.119 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.119 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.119 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.119 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.119 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.119 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.119 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.119 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.119 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.120 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.120 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.120 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:34.120 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:34.120 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.120 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.120 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.120 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.120 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.120 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.120 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.120 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.120 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.121 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.121 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.121 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.121 INFO:teuthology.orchestra.run.smithi160.stdout: "sleep_interval": { 2024-03-07T07:43:34.121 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "sleep_interval", 2024-03-07T07:43:34.121 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:34.121 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.121 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.121 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "60", 2024-03-07T07:43:34.121 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.121 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.122 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.122 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.122 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.122 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.122 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.122 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.122 INFO:teuthology.orchestra.run.smithi160.stdout: "threshold": { 2024-03-07T07:43:34.122 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "threshold", 2024-03-07T07:43:34.122 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "float", 2024-03-07T07:43:34.122 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.122 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.123 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "3.0", 2024-03-07T07:43:34.123 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "1.0", 2024-03-07T07:43:34.123 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.123 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.123 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "scaling threshold", 2024-03-07T07:43:34.123 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-07T07:43:34.123 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.123 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.123 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.123 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.123 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.123 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:34.124 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "progress", 2024-03-07T07:43:34.124 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:34.124 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:34.124 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:34.124 INFO:teuthology.orchestra.run.smithi160.stdout: "allow_pg_recovery_event": { 2024-03-07T07:43:34.124 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "allow_pg_recovery_event", 2024-03-07T07:43:34.124 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.124 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.124 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.124 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.124 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.124 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.125 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.125 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-07T07:43:34.125 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.125 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.125 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.125 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.125 INFO:teuthology.orchestra.run.smithi160.stdout: "enabled": { 2024-03-07T07:43:34.125 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "enabled", 2024-03-07T07:43:34.125 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.125 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.125 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.126 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:34.126 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.126 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.126 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.126 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.126 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.126 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.126 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.126 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.126 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:34.127 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:34.127 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.127 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.127 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.127 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.127 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.127 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.127 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.127 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.127 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.127 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.128 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.128 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.128 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.128 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.128 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.128 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.128 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.128 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.128 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.128 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:34.128 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:34.129 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.129 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.129 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.129 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.129 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.129 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.129 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.129 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.129 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.129 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.129 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.129 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.130 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:34.130 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:34.130 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.130 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.130 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.130 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:34.130 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.130 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.130 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.130 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.130 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.130 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.131 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.131 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.131 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.131 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.131 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.131 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.131 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.131 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.131 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.131 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:34.131 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:34.131 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.131 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.132 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.132 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.132 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.132 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.132 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.132 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.132 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.132 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.132 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.132 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.132 INFO:teuthology.orchestra.run.smithi160.stdout: "max_completed_events": { 2024-03-07T07:43:34.133 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "max_completed_events", 2024-03-07T07:43:34.133 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:34.133 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.133 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.133 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "50", 2024-03-07T07:43:34.133 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.133 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.133 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.133 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "number of past completed events to remember", 2024-03-07T07:43:34.133 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.133 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.133 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.134 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.134 INFO:teuthology.orchestra.run.smithi160.stdout: "sleep_interval": { 2024-03-07T07:43:34.134 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "sleep_interval", 2024-03-07T07:43:34.134 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:34.134 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.134 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.134 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "5", 2024-03-07T07:43:34.134 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.134 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.134 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.134 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "how long the module is going to sleep", 2024-03-07T07:43:34.134 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.134 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.135 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.135 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.135 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.135 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.135 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:34.135 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "prometheus", 2024-03-07T07:43:34.135 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:34.135 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:34.135 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:34.135 INFO:teuthology.orchestra.run.smithi160.stdout: "cache": { 2024-03-07T07:43:34.135 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "cache", 2024-03-07T07:43:34.135 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.136 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.136 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.136 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:34.136 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.136 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.136 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.136 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.136 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.136 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.136 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.136 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.136 INFO:teuthology.orchestra.run.smithi160.stdout: "exclude_perf_counters": { 2024-03-07T07:43:34.137 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "exclude_perf_counters", 2024-03-07T07:43:34.137 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.137 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.137 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.137 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:34.137 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.137 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.137 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.137 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-07T07:43:34.137 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-07T07:43:34.137 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.137 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.138 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.138 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:34.138 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:34.138 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.138 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.138 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.138 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.138 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.138 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.138 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.138 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.139 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.139 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.139 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.139 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.139 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.139 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.139 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.139 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.139 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.139 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.139 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.139 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:34.139 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:34.140 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.140 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.140 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.140 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.140 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.140 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.140 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.140 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.140 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.140 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.140 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.141 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.141 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:34.141 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:34.141 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.141 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.141 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.141 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:34.141 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.141 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.141 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.141 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.141 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.142 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.142 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.142 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.142 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.142 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.142 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.142 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.142 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.142 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.142 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.142 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:34.143 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:34.143 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.143 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.143 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.143 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.143 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.143 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.143 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.143 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.143 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.143 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.143 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.143 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.144 INFO:teuthology.orchestra.run.smithi160.stdout: "rbd_stats_pools": { 2024-03-07T07:43:34.144 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "rbd_stats_pools", 2024-03-07T07:43:34.144 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.144 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.144 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.144 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.144 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.144 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.144 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.145 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.145 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.145 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.145 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.145 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.145 INFO:teuthology.orchestra.run.smithi160.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-07T07:43:34.145 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-07T07:43:34.145 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:34.145 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.145 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.145 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "300", 2024-03-07T07:43:34.146 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.146 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.146 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.146 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.146 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.146 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.146 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.146 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.146 INFO:teuthology.orchestra.run.smithi160.stdout: "scrape_interval": { 2024-03-07T07:43:34.146 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "scrape_interval", 2024-03-07T07:43:34.146 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "float", 2024-03-07T07:43:34.146 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.147 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.147 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "15.0", 2024-03-07T07:43:34.147 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.147 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.147 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.147 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.147 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.147 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.147 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.147 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.147 INFO:teuthology.orchestra.run.smithi160.stdout: "server_addr": { 2024-03-07T07:43:34.147 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "server_addr", 2024-03-07T07:43:34.147 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.148 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.148 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.148 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "::", 2024-03-07T07:43:34.148 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.148 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.148 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.148 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-07T07:43:34.148 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.148 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.148 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.148 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.149 INFO:teuthology.orchestra.run.smithi160.stdout: "server_port": { 2024-03-07T07:43:34.149 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "server_port", 2024-03-07T07:43:34.149 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:34.149 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.149 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.149 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "9283", 2024-03-07T07:43:34.149 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.149 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.149 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.149 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-07T07:43:34.149 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.149 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.149 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.150 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.150 INFO:teuthology.orchestra.run.smithi160.stdout: "stale_cache_strategy": { 2024-03-07T07:43:34.150 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "stale_cache_strategy", 2024-03-07T07:43:34.150 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.150 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.150 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.150 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "log", 2024-03-07T07:43:34.150 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.150 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.150 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.150 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.150 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.151 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.151 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.151 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.151 INFO:teuthology.orchestra.run.smithi160.stdout: "standby_behaviour": { 2024-03-07T07:43:34.151 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "standby_behaviour", 2024-03-07T07:43:34.151 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.151 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.151 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.151 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "default", 2024-03-07T07:43:34.151 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.151 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.151 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.152 INFO:teuthology.orchestra.run.smithi160.stdout: "default", 2024-03-07T07:43:34.152 INFO:teuthology.orchestra.run.smithi160.stdout: "error" 2024-03-07T07:43:34.152 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.152 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.152 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.152 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.152 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.152 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.152 INFO:teuthology.orchestra.run.smithi160.stdout: "standby_error_status_code": { 2024-03-07T07:43:34.152 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "standby_error_status_code", 2024-03-07T07:43:34.152 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:34.153 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.153 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.153 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "500", 2024-03-07T07:43:34.153 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "400", 2024-03-07T07:43:34.153 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "599", 2024-03-07T07:43:34.153 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.153 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.153 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.153 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.153 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.153 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.153 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.154 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.154 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:34.154 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "rbd_support", 2024-03-07T07:43:34.154 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:34.154 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:34.154 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:34.154 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:34.154 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:34.154 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.154 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.154 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.154 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.155 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.155 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.155 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.155 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.155 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.155 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.155 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.155 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.155 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.155 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.155 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.155 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.155 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.156 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.156 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.156 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:34.156 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:34.156 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.156 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.156 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.156 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.156 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.156 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.157 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.157 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.157 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.157 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.157 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.157 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.157 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:34.157 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:34.157 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.157 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.157 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.158 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:34.158 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.158 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.158 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.158 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.158 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.158 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.158 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.158 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.158 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.158 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.159 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.159 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.159 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.159 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.159 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.159 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:34.159 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:34.159 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.159 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.159 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.159 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.159 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.160 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.160 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.160 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.160 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.160 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.160 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.160 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.160 INFO:teuthology.orchestra.run.smithi160.stdout: "max_concurrent_snap_create": { 2024-03-07T07:43:34.160 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "max_concurrent_snap_create", 2024-03-07T07:43:34.160 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:34.160 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.161 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.161 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "10", 2024-03-07T07:43:34.161 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.161 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.161 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.161 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.161 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.161 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.161 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.161 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.161 INFO:teuthology.orchestra.run.smithi160.stdout: "mirror_snapshot_schedule": { 2024-03-07T07:43:34.162 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "mirror_snapshot_schedule", 2024-03-07T07:43:34.162 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.162 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.162 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.162 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.162 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.162 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.162 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.162 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.162 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.162 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.162 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.162 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.163 INFO:teuthology.orchestra.run.smithi160.stdout: "trash_purge_schedule": { 2024-03-07T07:43:34.163 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "trash_purge_schedule", 2024-03-07T07:43:34.163 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.163 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.163 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.163 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.163 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.163 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.163 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.163 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.163 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.164 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.164 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.164 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.164 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.164 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.164 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:34.164 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "restful", 2024-03-07T07:43:34.164 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:34.164 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:34.164 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:34.164 INFO:teuthology.orchestra.run.smithi160.stdout: "enable_auth": { 2024-03-07T07:43:34.164 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "enable_auth", 2024-03-07T07:43:34.165 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.165 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.165 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.165 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:34.165 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.165 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.165 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.165 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.165 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.165 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.165 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.165 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.166 INFO:teuthology.orchestra.run.smithi160.stdout: "key_file": { 2024-03-07T07:43:34.166 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "key_file", 2024-03-07T07:43:34.166 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.166 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.166 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.166 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.166 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.166 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.166 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.166 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.166 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.167 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.167 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.167 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.167 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:34.167 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:34.167 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.167 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.167 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.167 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.167 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.167 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.167 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.168 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.168 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.168 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.168 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.168 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.168 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.168 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.168 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.168 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.168 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.169 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.169 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.169 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:34.169 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:34.169 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.169 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.169 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.169 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.169 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.169 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.169 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.170 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.170 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.170 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.170 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.170 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.170 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:34.170 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:34.170 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.170 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.170 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.170 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:34.170 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.170 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.171 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.171 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.171 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.171 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.171 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.171 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.171 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.171 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.171 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.171 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.171 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.171 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.172 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.172 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:34.172 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:34.172 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.172 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.172 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.172 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.172 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.172 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.172 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.172 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.172 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.173 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.173 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.173 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.173 INFO:teuthology.orchestra.run.smithi160.stdout: "server_addr": { 2024-03-07T07:43:34.173 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "server_addr", 2024-03-07T07:43:34.173 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.173 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.173 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.173 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.173 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.173 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.173 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.173 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.174 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.174 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.174 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.174 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.174 INFO:teuthology.orchestra.run.smithi160.stdout: "server_port": { 2024-03-07T07:43:34.174 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "server_port", 2024-03-07T07:43:34.174 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.174 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.174 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.174 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.174 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.174 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.174 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.175 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.175 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.175 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.175 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.175 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.175 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.175 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.175 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:34.175 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "rgw", 2024-03-07T07:43:34.175 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:34.175 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:34.175 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:34.175 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:34.175 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:34.176 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.176 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.176 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.176 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.176 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.176 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.176 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.176 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.176 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.176 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.176 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.177 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.177 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.177 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.177 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.177 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.177 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.177 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.177 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.177 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:34.177 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:34.177 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.177 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.178 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.178 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.178 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.178 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.178 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.178 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.178 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.178 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.178 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.178 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.178 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:34.179 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:34.179 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.179 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.179 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.179 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:34.179 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.179 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.179 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.179 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.179 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.179 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.179 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.180 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.180 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.180 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.180 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.180 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.180 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.180 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.180 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.180 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:34.180 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:34.181 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.181 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.181 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.181 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.181 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.181 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.181 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.181 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.181 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.181 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.181 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.181 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.181 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.182 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.182 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:34.182 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "rook", 2024-03-07T07:43:34.182 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:34.182 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:34.182 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:34.182 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:34.182 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:34.182 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.182 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.182 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.182 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.183 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.183 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.183 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.183 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.183 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.183 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.183 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.183 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.183 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.183 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.183 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.184 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.184 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.184 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.184 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.184 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:34.184 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:34.184 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.184 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.184 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.184 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.184 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.184 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.185 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.185 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.185 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.185 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.185 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.185 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.185 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:34.185 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:34.185 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.185 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.185 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.185 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:34.186 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.186 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.186 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.186 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.186 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.186 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.186 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.186 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.186 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.186 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.186 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.187 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.187 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.187 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.187 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.187 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:34.187 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:34.187 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.187 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.187 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.187 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.187 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.187 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.187 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.188 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.188 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.188 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.188 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.188 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.188 INFO:teuthology.orchestra.run.smithi160.stdout: "storage_class": { 2024-03-07T07:43:34.188 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "storage_class", 2024-03-07T07:43:34.188 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.188 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.188 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.188 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "local", 2024-03-07T07:43:34.189 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.189 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.189 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.189 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-07T07:43:34.189 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.189 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.189 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.189 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.189 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.189 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.189 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:34.189 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "selftest", 2024-03-07T07:43:34.190 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:34.190 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:34.190 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:34.190 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:34.190 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:34.190 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.190 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.190 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.190 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.191 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.191 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.191 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.191 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.191 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.191 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.191 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.191 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.191 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.191 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.191 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.191 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.192 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.192 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.192 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.192 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:34.192 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:34.192 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.192 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.192 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.192 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.192 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.192 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.193 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.193 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.193 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.193 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.193 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.193 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.193 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:34.193 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:34.193 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.193 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.193 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.194 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:34.194 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.194 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.194 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.194 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.194 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.194 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.194 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.194 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.194 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.194 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.194 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.194 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.194 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.195 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.195 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.195 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:34.195 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:34.195 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.195 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.195 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.195 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.195 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.195 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.195 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.195 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.195 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.196 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.196 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.196 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.196 INFO:teuthology.orchestra.run.smithi160.stdout: "roption1": { 2024-03-07T07:43:34.196 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "roption1", 2024-03-07T07:43:34.196 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.196 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.196 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.196 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.196 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.196 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.197 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.197 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.197 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.197 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.197 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.197 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.197 INFO:teuthology.orchestra.run.smithi160.stdout: "roption2": { 2024-03-07T07:43:34.197 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "roption2", 2024-03-07T07:43:34.197 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.197 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.197 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.197 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "xyz", 2024-03-07T07:43:34.198 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.198 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.198 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.198 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.198 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.198 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.198 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.198 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.198 INFO:teuthology.orchestra.run.smithi160.stdout: "rwoption1": { 2024-03-07T07:43:34.198 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "rwoption1", 2024-03-07T07:43:34.199 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.199 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.199 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.199 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.199 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.199 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.199 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.199 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.199 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.199 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.199 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.199 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.200 INFO:teuthology.orchestra.run.smithi160.stdout: "rwoption2": { 2024-03-07T07:43:34.200 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "rwoption2", 2024-03-07T07:43:34.200 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:34.200 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.200 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.200 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.200 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.200 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.200 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.200 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.200 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.200 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.201 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.201 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.201 INFO:teuthology.orchestra.run.smithi160.stdout: "rwoption3": { 2024-03-07T07:43:34.201 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "rwoption3", 2024-03-07T07:43:34.201 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "float", 2024-03-07T07:43:34.201 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.201 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.201 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.201 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.201 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.201 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.201 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.202 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.202 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.202 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.202 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.202 INFO:teuthology.orchestra.run.smithi160.stdout: "rwoption4": { 2024-03-07T07:43:34.202 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "rwoption4", 2024-03-07T07:43:34.202 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.202 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.202 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.202 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.202 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.202 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.203 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.203 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.203 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.203 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.203 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.203 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.203 INFO:teuthology.orchestra.run.smithi160.stdout: "rwoption5": { 2024-03-07T07:43:34.203 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "rwoption5", 2024-03-07T07:43:34.203 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.203 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.203 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.203 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.204 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.204 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.204 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.204 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.204 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.204 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.204 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.204 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.204 INFO:teuthology.orchestra.run.smithi160.stdout: "rwoption6": { 2024-03-07T07:43:34.204 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "rwoption6", 2024-03-07T07:43:34.204 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.204 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.205 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.205 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:34.205 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.205 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.205 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.205 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.205 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.205 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.205 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.205 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.205 INFO:teuthology.orchestra.run.smithi160.stdout: "rwoption7": { 2024-03-07T07:43:34.205 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "rwoption7", 2024-03-07T07:43:34.206 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:34.206 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.206 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.206 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.206 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "1", 2024-03-07T07:43:34.206 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "42", 2024-03-07T07:43:34.206 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.206 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.206 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.207 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.207 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.207 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.207 INFO:teuthology.orchestra.run.smithi160.stdout: "testkey": { 2024-03-07T07:43:34.207 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "testkey", 2024-03-07T07:43:34.207 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.207 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.207 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.207 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.207 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.207 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.208 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.208 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.208 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.208 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.208 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.208 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.208 INFO:teuthology.orchestra.run.smithi160.stdout: "testlkey": { 2024-03-07T07:43:34.208 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "testlkey", 2024-03-07T07:43:34.208 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.208 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.209 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.209 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.209 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.209 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.209 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.209 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.209 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.209 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.209 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.209 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.209 INFO:teuthology.orchestra.run.smithi160.stdout: "testnewline": { 2024-03-07T07:43:34.209 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "testnewline", 2024-03-07T07:43:34.210 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.210 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.210 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.210 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.210 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.210 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.210 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.210 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.210 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.210 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.210 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.211 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.211 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.211 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.211 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:34.211 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "snap_schedule", 2024-03-07T07:43:34.211 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:34.211 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:34.211 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:34.211 INFO:teuthology.orchestra.run.smithi160.stdout: "allow_m_granularity": { 2024-03-07T07:43:34.211 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "allow_m_granularity", 2024-03-07T07:43:34.211 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.212 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.212 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.212 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.212 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.212 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.212 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.212 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "allow minute scheduled snapshots", 2024-03-07T07:43:34.212 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.212 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.212 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.212 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.212 INFO:teuthology.orchestra.run.smithi160.stdout: "dump_on_update": { 2024-03-07T07:43:34.212 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "dump_on_update", 2024-03-07T07:43:34.213 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.213 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.213 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.213 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.213 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.213 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.213 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.213 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "dump database to debug log on update", 2024-03-07T07:43:34.213 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.213 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.213 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.214 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.214 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:34.214 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:34.214 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.214 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.214 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.214 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.214 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.214 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.215 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.215 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.215 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.215 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.215 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.215 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.215 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.215 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.215 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.215 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.215 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.216 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.216 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.216 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:34.216 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:34.216 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.216 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.216 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.216 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.216 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.216 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.216 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.217 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.217 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.217 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.217 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.217 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.217 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:34.217 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:34.217 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.217 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.217 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.217 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:34.218 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.218 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.218 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.218 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.218 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.218 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.218 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.218 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.218 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.218 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.218 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.218 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.219 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.219 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.219 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.219 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:34.219 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:34.219 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.219 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.219 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.219 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.219 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.219 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.219 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.220 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.220 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.220 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.220 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.220 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.220 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.220 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.220 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:34.220 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "stats", 2024-03-07T07:43:34.220 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:34.220 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:34.220 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:34.221 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:34.221 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:34.221 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.221 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.221 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.221 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.221 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.221 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.221 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.221 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.221 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.221 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.221 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.222 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.222 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.222 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.222 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.222 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.222 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.222 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.222 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.222 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:34.222 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:34.222 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.222 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.223 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.223 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.223 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.223 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.223 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.223 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.223 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.223 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.223 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.223 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.223 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:34.223 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:34.224 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.224 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.224 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.224 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:34.224 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.224 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.224 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.224 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.224 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.224 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.224 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.224 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.225 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.225 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.225 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.225 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.225 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.225 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.225 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.225 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:34.225 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:34.225 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.225 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.225 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.225 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.226 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.226 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.226 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.226 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.226 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.226 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.226 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.226 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.226 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.226 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.226 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:34.226 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "status", 2024-03-07T07:43:34.227 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:34.227 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:34.227 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:34.227 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:34.227 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:34.227 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.227 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.227 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.227 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.227 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.227 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.227 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.228 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.228 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.228 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.228 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.228 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.228 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.228 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.228 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.228 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.228 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.228 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.228 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.229 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:34.229 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:34.229 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.229 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.229 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.229 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.229 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.229 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.229 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.229 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.229 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.229 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.229 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.230 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.230 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:34.230 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:34.230 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.230 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.230 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.230 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:34.230 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.230 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.230 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.230 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.231 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.231 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.231 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.231 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.231 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.231 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.231 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.231 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.231 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.231 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.231 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.231 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:34.231 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:34.232 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.232 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.232 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.232 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.232 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.232 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.232 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.232 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.232 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.232 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.232 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.232 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.233 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.233 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.233 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:34.233 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "telegraf", 2024-03-07T07:43:34.233 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:34.233 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:34.233 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:34.233 INFO:teuthology.orchestra.run.smithi160.stdout: "address": { 2024-03-07T07:43:34.233 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "address", 2024-03-07T07:43:34.233 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.234 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.234 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.234 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-07T07:43:34.234 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.234 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.234 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.234 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.234 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.234 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.234 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.234 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.234 INFO:teuthology.orchestra.run.smithi160.stdout: "interval": { 2024-03-07T07:43:34.235 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "interval", 2024-03-07T07:43:34.235 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:34.235 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.235 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.235 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "15", 2024-03-07T07:43:34.235 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.235 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.235 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.235 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.235 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.235 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.235 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.236 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.236 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:34.236 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:34.236 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.236 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.236 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.236 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.236 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.236 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.236 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.236 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.236 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.237 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.237 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.237 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.237 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.237 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.237 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.237 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.237 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.237 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.237 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.237 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:34.237 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:34.237 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.238 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.238 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.238 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.238 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.238 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.238 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.238 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.238 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.238 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.238 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.238 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.238 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:34.239 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:34.239 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.239 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.239 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.239 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:34.239 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.239 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.239 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.239 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.239 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.239 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.239 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.240 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.240 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.240 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.240 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.240 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.240 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.240 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.240 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.240 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:34.240 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:34.240 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.241 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.241 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.241 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.241 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.241 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.241 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.241 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.241 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.241 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.241 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.241 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.241 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.242 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.242 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:34.242 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "telemetry", 2024-03-07T07:43:34.242 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:34.242 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:34.242 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:34.242 INFO:teuthology.orchestra.run.smithi160.stdout: "channel_basic": { 2024-03-07T07:43:34.242 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "channel_basic", 2024-03-07T07:43:34.242 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.242 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.243 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.243 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:34.243 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.243 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.243 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.243 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-07T07:43:34.243 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.243 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.243 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.243 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.243 INFO:teuthology.orchestra.run.smithi160.stdout: "channel_crash": { 2024-03-07T07:43:34.244 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "channel_crash", 2024-03-07T07:43:34.244 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.244 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.244 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.244 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:34.244 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.244 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.244 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.244 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-07T07:43:34.244 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.244 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.244 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.245 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.245 INFO:teuthology.orchestra.run.smithi160.stdout: "channel_device": { 2024-03-07T07:43:34.245 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "channel_device", 2024-03-07T07:43:34.245 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.245 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.245 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.245 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:34.245 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.245 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.245 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.245 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-07T07:43:34.245 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.246 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.246 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.246 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.246 INFO:teuthology.orchestra.run.smithi160.stdout: "channel_ident": { 2024-03-07T07:43:34.246 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "channel_ident", 2024-03-07T07:43:34.246 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.246 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.246 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.246 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.246 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.246 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.246 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.246 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-07T07:43:34.247 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.247 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.247 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.247 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.247 INFO:teuthology.orchestra.run.smithi160.stdout: "channel_perf": { 2024-03-07T07:43:34.247 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "channel_perf", 2024-03-07T07:43:34.247 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.247 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.247 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.247 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.247 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.247 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.248 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.248 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-07T07:43:34.248 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.248 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.248 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.248 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.248 INFO:teuthology.orchestra.run.smithi160.stdout: "contact": { 2024-03-07T07:43:34.248 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "contact", 2024-03-07T07:43:34.248 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.248 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.248 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.249 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.249 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.249 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.249 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.249 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.249 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.249 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.249 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.249 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.249 INFO:teuthology.orchestra.run.smithi160.stdout: "description": { 2024-03-07T07:43:34.249 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "description", 2024-03-07T07:43:34.249 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.250 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.250 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.250 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.250 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.250 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.250 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.250 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.250 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.250 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.250 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.250 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.250 INFO:teuthology.orchestra.run.smithi160.stdout: "device_url": { 2024-03-07T07:43:34.251 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "device_url", 2024-03-07T07:43:34.251 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.251 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.251 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.251 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-07T07:43:34.251 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.251 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.251 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.251 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.251 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.251 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.251 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.252 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.252 INFO:teuthology.orchestra.run.smithi160.stdout: "enabled": { 2024-03-07T07:43:34.252 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "enabled", 2024-03-07T07:43:34.252 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.252 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.252 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.252 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.252 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.252 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.252 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.252 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.252 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.253 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.253 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.256 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.256 INFO:teuthology.orchestra.run.smithi160.stdout: "interval": { 2024-03-07T07:43:34.256 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "interval", 2024-03-07T07:43:34.256 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:34.256 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.256 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.256 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "24", 2024-03-07T07:43:34.256 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "8", 2024-03-07T07:43:34.256 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.256 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.257 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.257 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.257 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.257 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.257 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.257 INFO:teuthology.orchestra.run.smithi160.stdout: "last_opt_revision": { 2024-03-07T07:43:34.257 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "last_opt_revision", 2024-03-07T07:43:34.257 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:34.257 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.257 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.257 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "1", 2024-03-07T07:43:34.257 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.258 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.258 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.258 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.258 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.258 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.258 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.258 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.258 INFO:teuthology.orchestra.run.smithi160.stdout: "leaderboard": { 2024-03-07T07:43:34.258 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "leaderboard", 2024-03-07T07:43:34.258 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.258 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.258 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.258 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.258 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.259 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.259 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.259 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.259 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.259 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.259 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.259 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.259 INFO:teuthology.orchestra.run.smithi160.stdout: "leaderboard_description": { 2024-03-07T07:43:34.259 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "leaderboard_description", 2024-03-07T07:43:34.259 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.259 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.259 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.259 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.259 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.260 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.260 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.260 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.260 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.260 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.260 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.260 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.260 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:34.260 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:34.260 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.260 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.260 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.260 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.261 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.261 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.261 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.261 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.261 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.261 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.261 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.261 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.261 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.261 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.261 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.261 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.261 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.261 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.262 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.262 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:34.262 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:34.262 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.262 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.262 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.262 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.262 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.262 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.262 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.262 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.262 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.262 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.262 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.263 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.263 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:34.263 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:34.263 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.263 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.263 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.263 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:34.263 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.263 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.263 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.263 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.263 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.263 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.263 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.264 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.264 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.264 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.264 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.264 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.264 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.264 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.264 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.264 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:34.264 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:34.264 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.264 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.264 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.264 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.265 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.265 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.265 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.265 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.265 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.265 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.265 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.265 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.265 INFO:teuthology.orchestra.run.smithi160.stdout: "organization": { 2024-03-07T07:43:34.265 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "organization", 2024-03-07T07:43:34.265 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.265 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.265 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.265 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.265 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.266 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.266 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.266 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.266 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.266 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.266 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.266 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.266 INFO:teuthology.orchestra.run.smithi160.stdout: "proxy": { 2024-03-07T07:43:34.266 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "proxy", 2024-03-07T07:43:34.266 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.266 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.266 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.266 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.266 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.267 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.267 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.267 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.267 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.267 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.267 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.267 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.267 INFO:teuthology.orchestra.run.smithi160.stdout: "url": { 2024-03-07T07:43:34.267 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "url", 2024-03-07T07:43:34.267 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.267 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.267 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.267 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-07T07:43:34.267 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.268 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.268 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.268 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.268 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.268 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.268 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.268 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.268 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.268 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.268 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:34.268 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "test_orchestrator", 2024-03-07T07:43:34.268 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:34.268 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:34.269 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:34.269 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:34.269 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:34.269 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.269 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.269 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.269 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.269 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.269 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.269 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.269 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.269 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.269 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.269 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.270 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.270 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.270 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.270 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.270 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.270 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.270 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.270 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.270 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:34.270 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:34.270 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.270 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.270 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.271 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.271 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.271 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.271 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.271 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.271 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.271 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.271 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.271 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.271 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:34.271 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:34.271 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.271 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.272 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.272 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:34.272 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.272 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.272 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.272 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.272 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.272 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.272 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.272 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.272 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.272 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.272 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.273 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.273 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.273 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.273 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.273 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:34.273 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:34.273 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.273 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.273 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.273 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.273 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.273 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.273 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.273 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.274 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.274 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.274 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.274 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.274 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.274 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.274 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:34.274 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "volumes", 2024-03-07T07:43:34.274 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:34.274 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:34.274 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:34.274 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:34.274 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:34.275 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.275 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.275 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.275 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.275 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.275 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.275 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.275 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.275 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.275 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.275 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.275 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.275 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.275 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.276 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.276 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.276 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.276 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.276 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.276 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:34.276 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:34.276 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.276 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.276 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.276 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.276 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.276 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.277 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.277 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.277 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.277 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.277 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.277 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.277 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:34.277 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:34.277 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.277 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.277 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.277 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:34.277 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.277 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.277 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.278 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.278 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.278 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.278 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.278 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.278 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.278 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.278 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.278 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.278 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.278 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.278 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.278 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:34.279 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:34.279 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.279 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.279 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.279 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.279 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.279 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.279 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.279 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.279 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.279 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.279 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.279 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.279 INFO:teuthology.orchestra.run.smithi160.stdout: "max_concurrent_clones": { 2024-03-07T07:43:34.280 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "max_concurrent_clones", 2024-03-07T07:43:34.280 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:34.280 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.280 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.280 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "4", 2024-03-07T07:43:34.280 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.280 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.280 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.280 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-07T07:43:34.280 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.280 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.280 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.280 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.281 INFO:teuthology.orchestra.run.smithi160.stdout: "periodic_async_work": { 2024-03-07T07:43:34.281 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "periodic_async_work", 2024-03-07T07:43:34.281 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.281 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.281 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.281 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.281 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.281 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.281 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.281 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Periodically check for async work", 2024-03-07T07:43:34.281 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.281 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.281 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.281 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.282 INFO:teuthology.orchestra.run.smithi160.stdout: "snapshot_clone_delay": { 2024-03-07T07:43:34.282 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "snapshot_clone_delay", 2024-03-07T07:43:34.282 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:34.282 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.282 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.282 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "0", 2024-03-07T07:43:34.282 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.282 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.282 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.282 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-07T07:43:34.282 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.282 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.282 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.282 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.283 INFO:teuthology.orchestra.run.smithi160.stdout: "snapshot_clone_no_wait": { 2024-03-07T07:43:34.283 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "snapshot_clone_no_wait", 2024-03-07T07:43:34.283 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.283 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.283 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.283 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2024-03-07T07:43:34.283 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.283 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.283 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.283 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-07T07:43:34.283 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.283 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.283 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.283 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.284 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.284 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.284 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:34.284 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "zabbix", 2024-03-07T07:43:34.284 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2024-03-07T07:43:34.284 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2024-03-07T07:43:34.284 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2024-03-07T07:43:34.284 INFO:teuthology.orchestra.run.smithi160.stdout: "discovery_interval": { 2024-03-07T07:43:34.284 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "discovery_interval", 2024-03-07T07:43:34.284 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "uint", 2024-03-07T07:43:34.284 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.284 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.284 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "100", 2024-03-07T07:43:34.284 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.285 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.285 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.285 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.285 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.285 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.285 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.285 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.285 INFO:teuthology.orchestra.run.smithi160.stdout: "identifier": { 2024-03-07T07:43:34.285 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "identifier", 2024-03-07T07:43:34.285 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.285 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.285 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.285 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.286 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.286 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.286 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.286 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.286 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.286 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.286 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.286 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.286 INFO:teuthology.orchestra.run.smithi160.stdout: "interval": { 2024-03-07T07:43:34.286 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "interval", 2024-03-07T07:43:34.286 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2024-03-07T07:43:34.286 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.286 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.286 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "60", 2024-03-07T07:43:34.287 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.287 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.287 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.287 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.287 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.287 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.287 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.287 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.287 INFO:teuthology.orchestra.run.smithi160.stdout: "log_level": { 2024-03-07T07:43:34.287 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_level", 2024-03-07T07:43:34.287 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.287 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.287 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.288 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.288 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.288 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.288 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.288 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.288 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.288 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.288 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.288 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.288 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.288 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.288 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.288 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.289 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.289 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.289 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.289 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster": { 2024-03-07T07:43:34.289 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster", 2024-03-07T07:43:34.289 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.289 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.289 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.289 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.289 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.289 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.289 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.289 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.290 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.290 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.290 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.290 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.290 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_cluster_level": { 2024-03-07T07:43:34.290 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_cluster_level", 2024-03-07T07:43:34.290 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.290 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.290 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.290 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "info", 2024-03-07T07:43:34.290 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.290 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.290 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2024-03-07T07:43:34.291 INFO:teuthology.orchestra.run.smithi160.stdout: "", 2024-03-07T07:43:34.291 INFO:teuthology.orchestra.run.smithi160.stdout: "critical", 2024-03-07T07:43:34.291 INFO:teuthology.orchestra.run.smithi160.stdout: "debug", 2024-03-07T07:43:34.291 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2024-03-07T07:43:34.291 INFO:teuthology.orchestra.run.smithi160.stdout: "info", 2024-03-07T07:43:34.291 INFO:teuthology.orchestra.run.smithi160.stdout: "warning" 2024-03-07T07:43:34.291 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.291 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.291 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.291 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.291 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.291 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.291 INFO:teuthology.orchestra.run.smithi160.stdout: "log_to_file": { 2024-03-07T07:43:34.291 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "log_to_file", 2024-03-07T07:43:34.292 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2024-03-07T07:43:34.292 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.292 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2024-03-07T07:43:34.292 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2024-03-07T07:43:34.292 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.292 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.292 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.292 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.292 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.292 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.292 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.292 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.292 INFO:teuthology.orchestra.run.smithi160.stdout: "zabbix_host": { 2024-03-07T07:43:34.293 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "zabbix_host", 2024-03-07T07:43:34.293 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.293 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.293 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.293 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2024-03-07T07:43:34.293 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.293 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.293 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.293 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.293 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.293 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.293 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.293 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.293 INFO:teuthology.orchestra.run.smithi160.stdout: "zabbix_port": { 2024-03-07T07:43:34.294 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "zabbix_port", 2024-03-07T07:43:34.294 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2024-03-07T07:43:34.294 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.294 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.294 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "10051", 2024-03-07T07:43:34.294 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.294 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.294 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.294 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.294 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.294 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.294 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.294 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.295 INFO:teuthology.orchestra.run.smithi160.stdout: "zabbix_sender": { 2024-03-07T07:43:34.295 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "zabbix_sender", 2024-03-07T07:43:34.295 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2024-03-07T07:43:34.295 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2024-03-07T07:43:34.295 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2024-03-07T07:43:34.295 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-07T07:43:34.295 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2024-03-07T07:43:34.295 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2024-03-07T07:43:34.295 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2024-03-07T07:43:34.295 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2024-03-07T07:43:34.295 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2024-03-07T07:43:34.295 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2024-03-07T07:43:34.295 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2024-03-07T07:43:34.295 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.295 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.296 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.296 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.296 INFO:teuthology.orchestra.run.smithi160.stdout: "services": { 2024-03-07T07:43:34.296 INFO:teuthology.orchestra.run.smithi160.stdout: "dashboard": "https://172.21.15.160:8443/" 2024-03-07T07:43:34.296 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.296 INFO:teuthology.orchestra.run.smithi160.stdout: "always_on_modules": { 2024-03-07T07:43:34.296 INFO:teuthology.orchestra.run.smithi160.stdout: "octopus": [ 2024-03-07T07:43:34.296 INFO:teuthology.orchestra.run.smithi160.stdout: "balancer", 2024-03-07T07:43:34.296 INFO:teuthology.orchestra.run.smithi160.stdout: "crash", 2024-03-07T07:43:34.296 INFO:teuthology.orchestra.run.smithi160.stdout: "devicehealth", 2024-03-07T07:43:34.296 INFO:teuthology.orchestra.run.smithi160.stdout: "orchestrator", 2024-03-07T07:43:34.296 INFO:teuthology.orchestra.run.smithi160.stdout: "pg_autoscaler", 2024-03-07T07:43:34.296 INFO:teuthology.orchestra.run.smithi160.stdout: "progress", 2024-03-07T07:43:34.297 INFO:teuthology.orchestra.run.smithi160.stdout: "rbd_support", 2024-03-07T07:43:34.297 INFO:teuthology.orchestra.run.smithi160.stdout: "status", 2024-03-07T07:43:34.297 INFO:teuthology.orchestra.run.smithi160.stdout: "telemetry", 2024-03-07T07:43:34.297 INFO:teuthology.orchestra.run.smithi160.stdout: "volumes" 2024-03-07T07:43:34.297 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.297 INFO:teuthology.orchestra.run.smithi160.stdout: "pacific": [ 2024-03-07T07:43:34.297 INFO:teuthology.orchestra.run.smithi160.stdout: "balancer", 2024-03-07T07:43:34.297 INFO:teuthology.orchestra.run.smithi160.stdout: "crash", 2024-03-07T07:43:34.297 INFO:teuthology.orchestra.run.smithi160.stdout: "devicehealth", 2024-03-07T07:43:34.297 INFO:teuthology.orchestra.run.smithi160.stdout: "orchestrator", 2024-03-07T07:43:34.297 INFO:teuthology.orchestra.run.smithi160.stdout: "pg_autoscaler", 2024-03-07T07:43:34.297 INFO:teuthology.orchestra.run.smithi160.stdout: "progress", 2024-03-07T07:43:34.297 INFO:teuthology.orchestra.run.smithi160.stdout: "rbd_support", 2024-03-07T07:43:34.297 INFO:teuthology.orchestra.run.smithi160.stdout: "status", 2024-03-07T07:43:34.298 INFO:teuthology.orchestra.run.smithi160.stdout: "telemetry", 2024-03-07T07:43:34.298 INFO:teuthology.orchestra.run.smithi160.stdout: "volumes" 2024-03-07T07:43:34.298 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.298 INFO:teuthology.orchestra.run.smithi160.stdout: "quincy": [ 2024-03-07T07:43:34.298 INFO:teuthology.orchestra.run.smithi160.stdout: "balancer", 2024-03-07T07:43:34.298 INFO:teuthology.orchestra.run.smithi160.stdout: "crash", 2024-03-07T07:43:34.298 INFO:teuthology.orchestra.run.smithi160.stdout: "devicehealth", 2024-03-07T07:43:34.298 INFO:teuthology.orchestra.run.smithi160.stdout: "orchestrator", 2024-03-07T07:43:34.298 INFO:teuthology.orchestra.run.smithi160.stdout: "pg_autoscaler", 2024-03-07T07:43:34.298 INFO:teuthology.orchestra.run.smithi160.stdout: "progress", 2024-03-07T07:43:34.298 INFO:teuthology.orchestra.run.smithi160.stdout: "rbd_support", 2024-03-07T07:43:34.298 INFO:teuthology.orchestra.run.smithi160.stdout: "status", 2024-03-07T07:43:34.298 INFO:teuthology.orchestra.run.smithi160.stdout: "telemetry", 2024-03-07T07:43:34.298 INFO:teuthology.orchestra.run.smithi160.stdout: "volumes" 2024-03-07T07:43:34.298 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.299 INFO:teuthology.orchestra.run.smithi160.stdout: "reef": [ 2024-03-07T07:43:34.299 INFO:teuthology.orchestra.run.smithi160.stdout: "balancer", 2024-03-07T07:43:34.299 INFO:teuthology.orchestra.run.smithi160.stdout: "crash", 2024-03-07T07:43:34.299 INFO:teuthology.orchestra.run.smithi160.stdout: "devicehealth", 2024-03-07T07:43:34.299 INFO:teuthology.orchestra.run.smithi160.stdout: "orchestrator", 2024-03-07T07:43:34.299 INFO:teuthology.orchestra.run.smithi160.stdout: "pg_autoscaler", 2024-03-07T07:43:34.299 INFO:teuthology.orchestra.run.smithi160.stdout: "progress", 2024-03-07T07:43:34.299 INFO:teuthology.orchestra.run.smithi160.stdout: "rbd_support", 2024-03-07T07:43:34.299 INFO:teuthology.orchestra.run.smithi160.stdout: "status", 2024-03-07T07:43:34.299 INFO:teuthology.orchestra.run.smithi160.stdout: "telemetry", 2024-03-07T07:43:34.299 INFO:teuthology.orchestra.run.smithi160.stdout: "volumes" 2024-03-07T07:43:34.299 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2024-03-07T07:43:34.299 INFO:teuthology.orchestra.run.smithi160.stdout: "squid": [ 2024-03-07T07:43:34.299 INFO:teuthology.orchestra.run.smithi160.stdout: "balancer", 2024-03-07T07:43:34.300 INFO:teuthology.orchestra.run.smithi160.stdout: "crash", 2024-03-07T07:43:34.300 INFO:teuthology.orchestra.run.smithi160.stdout: "devicehealth", 2024-03-07T07:43:34.300 INFO:teuthology.orchestra.run.smithi160.stdout: "orchestrator", 2024-03-07T07:43:34.300 INFO:teuthology.orchestra.run.smithi160.stdout: "pg_autoscaler", 2024-03-07T07:43:34.300 INFO:teuthology.orchestra.run.smithi160.stdout: "progress", 2024-03-07T07:43:34.300 INFO:teuthology.orchestra.run.smithi160.stdout: "rbd_support", 2024-03-07T07:43:34.300 INFO:teuthology.orchestra.run.smithi160.stdout: "status", 2024-03-07T07:43:34.300 INFO:teuthology.orchestra.run.smithi160.stdout: "telemetry", 2024-03-07T07:43:34.300 INFO:teuthology.orchestra.run.smithi160.stdout: "volumes" 2024-03-07T07:43:34.300 INFO:teuthology.orchestra.run.smithi160.stdout: ] 2024-03-07T07:43:34.300 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.300 INFO:teuthology.orchestra.run.smithi160.stdout: "last_failure_osd_epoch": 3, 2024-03-07T07:43:34.300 INFO:teuthology.orchestra.run.smithi160.stdout: "active_clients": [ 2024-03-07T07:43:34.301 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:34.301 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "devicehealth", 2024-03-07T07:43:34.301 INFO:teuthology.orchestra.run.smithi160.stdout: "addrvec": [ 2024-03-07T07:43:34.301 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:34.301 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "v2", 2024-03-07T07:43:34.301 INFO:teuthology.orchestra.run.smithi160.stdout: "addr": "172.21.15.160:0", 2024-03-07T07:43:34.301 INFO:teuthology.orchestra.run.smithi160.stdout: "nonce": 870342466 2024-03-07T07:43:34.301 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.301 INFO:teuthology.orchestra.run.smithi160.stdout: ] 2024-03-07T07:43:34.301 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.301 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:34.301 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "libcephsqlite", 2024-03-07T07:43:34.301 INFO:teuthology.orchestra.run.smithi160.stdout: "addrvec": [ 2024-03-07T07:43:34.301 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:34.302 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "v2", 2024-03-07T07:43:34.302 INFO:teuthology.orchestra.run.smithi160.stdout: "addr": "172.21.15.160:0", 2024-03-07T07:43:34.302 INFO:teuthology.orchestra.run.smithi160.stdout: "nonce": 1513978191 2024-03-07T07:43:34.302 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.302 INFO:teuthology.orchestra.run.smithi160.stdout: ] 2024-03-07T07:43:34.302 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.302 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:34.302 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "rbd_support", 2024-03-07T07:43:34.302 INFO:teuthology.orchestra.run.smithi160.stdout: "addrvec": [ 2024-03-07T07:43:34.302 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:34.302 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "v2", 2024-03-07T07:43:34.302 INFO:teuthology.orchestra.run.smithi160.stdout: "addr": "172.21.15.160:0", 2024-03-07T07:43:34.302 INFO:teuthology.orchestra.run.smithi160.stdout: "nonce": 3297345964 2024-03-07T07:43:34.302 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.302 INFO:teuthology.orchestra.run.smithi160.stdout: ] 2024-03-07T07:43:34.303 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:34.303 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:34.303 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "volumes", 2024-03-07T07:43:34.303 INFO:teuthology.orchestra.run.smithi160.stdout: "addrvec": [ 2024-03-07T07:43:34.303 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-03-07T07:43:34.303 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "v2", 2024-03-07T07:43:34.303 INFO:teuthology.orchestra.run.smithi160.stdout: "addr": "172.21.15.160:0", 2024-03-07T07:43:34.303 INFO:teuthology.orchestra.run.smithi160.stdout: "nonce": 1219726049 2024-03-07T07:43:34.303 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.303 INFO:teuthology.orchestra.run.smithi160.stdout: ] 2024-03-07T07:43:34.303 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:34.303 INFO:teuthology.orchestra.run.smithi160.stdout: ] 2024-03-07T07:43:34.303 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-03-07T07:43:34.305 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.827+0000 7f3240467640 1 -- 172.21.15.160:0/4252030292 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f32200a1d80 msgr2=0x7f32200a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:43:34.305 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.827+0000 7f3240467640 1 --2- 172.21.15.160:0/4252030292 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f32200a1d80 0x7f32200a4240 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f323002c170 tx=0x7f323002df90 comp rx=0 tx=0).stop 2024-03-07T07:43:34.305 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.827+0000 7f3240467640 1 -- 172.21.15.160:0/4252030292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3238102870 msgr2=0x7f3238197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:43:34.305 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.827+0000 7f3240467640 1 --2- 172.21.15.160:0/4252030292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3238102870 0x7f3238197ef0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f322c00eb90 tx=0x7f322c00ef90 comp rx=0 tx=0).stop 2024-03-07T07:43:34.305 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.827+0000 7f3240467640 1 -- 172.21.15.160:0/4252030292 shutdown_connections 2024-03-07T07:43:34.305 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.827+0000 7f3240467640 1 --2- 172.21.15.160:0/4252030292 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f32200a1d80 0x7f32200a4240 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:34.305 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.827+0000 7f3240467640 1 --2- 172.21.15.160:0/4252030292 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3238103230 0x7f323819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:34.305 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.827+0000 7f3240467640 1 --2- 172.21.15.160:0/4252030292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3238102870 0x7f3238197ef0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:34.306 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.827+0000 7f3240467640 1 --2- 172.21.15.160:0/4252030292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3238101680 0x7f32381979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:43:34.306 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.827+0000 7f3240467640 1 -- 172.21.15.160:0/4252030292 >> 172.21.15.160:0/4252030292 conn(0x7f32380fcc10 msgr2=0x7f3238105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:43:34.306 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.827+0000 7f3240467640 1 -- 172.21.15.160:0/4252030292 shutdown_connections 2024-03-07T07:43:34.306 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:43:32.827+0000 7f3240467640 1 -- 172.21.15.160:0/4252030292 wait complete. 2024-03-07T07:43:34.306 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:43:32 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/2600590607' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-07T07:43:34.306 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:43:32 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/1689018960' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-07T07:43:34.306 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:43:32 smithi160 ceph-mon[31963]: pgmap v419: 97 pgs: 97 active+clean; 258 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 861 KiB/s wr, 10 op/s 2024-03-07T07:43:34.306 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:43:32 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/2096591023' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-07T07:43:34.307 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:43:32 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/2600590607' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-07T07:43:34.307 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:43:32 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/1689018960' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-07T07:43:34.307 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:43:32 smithi177 ceph-mon[36578]: pgmap v419: 97 pgs: 97 active+clean; 258 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 861 KiB/s wr, 10 op/s 2024-03-07T07:43:34.307 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:43:32 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/2096591023' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-07T07:43:34.307 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:43:33 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/4252030292' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-07T07:43:34.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:43:32 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/2600590607' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-07T07:43:34.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:43:32 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/1689018960' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-07T07:43:34.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:43:32 smithi178 ceph-mon[36650]: pgmap v419: 97 pgs: 97 active+clean; 258 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 861 KiB/s wr, 10 op/s 2024-03-07T07:43:34.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:43:32 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/2096591023' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-07T07:43:34.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:43:33 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/4252030292' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-07T07:43:34.314 DEBUG:tasks.check_counter:Getting stats from a 2024-03-07T07:43:34.314 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph daemon mds.a perf dump 2024-03-07T07:43:34.314 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:43:33 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/4252030292' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-07T07:43:34.572 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:43:34.896 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:43:34 smithi160 ceph-mon[31963]: pgmap v420: 97 pgs: 97 active+clean; 258 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 946 KiB/s wr, 11 op/s 2024-03-07T07:43:35.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:43:34 smithi178 ceph-mon[36650]: pgmap v420: 97 pgs: 97 active+clean; 258 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 946 KiB/s wr, 11 op/s 2024-03-07T07:43:35.106 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:43:34 smithi177 ceph-mon[36578]: pgmap v420: 97 pgs: 97 active+clean; 258 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 946 KiB/s wr, 11 op/s 2024-03-07T07:43:35.970 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:43:35 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:43:36.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:43:35 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:43:36.106 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:43:35 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-07T07:43:36.354 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-03-07T07:43:36.355 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-0": { 2024-03-07T07:43:36.355 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 4, 2024-03-07T07:43:36.355 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 4, 2024-03-07T07:43:36.355 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 3072, 2024-03-07T07:43:36.355 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 448, 2024-03-07T07:43:36.355 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 4, 2024-03-07T07:43:36.355 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 0, 2024-03-07T07:43:36.355 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.001586216, 2024-03-07T07:43:36.355 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.000036512, 2024-03-07T07:43:36.355 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.000629390, 2024-03-07T07:43:36.355 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-07T07:43:36.355 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-03-07T07:43:36.355 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 3, 2024-03-07T07:43:36.355 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000806173, 2024-03-07T07:43:36.356 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000268724 2024-03-07T07:43:36.356 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:36.356 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-03-07T07:43:36.356 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-03-07T07:43:36.356 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-03-07T07:43:36.356 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-03-07T07:43:36.356 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:36.356 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-03-07T07:43:36.356 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 448 2024-03-07T07:43:36.356 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:36.356 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-1": { 2024-03-07T07:43:36.356 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 1, 2024-03-07T07:43:36.356 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 58, 2024-03-07T07:43:36.357 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 160, 2024-03-07T07:43:36.357 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 87744, 2024-03-07T07:43:36.357 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 3, 2024-03-07T07:43:36.357 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 2, 2024-03-07T07:43:36.357 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.005695902, 2024-03-07T07:43:36.357 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.003680981, 2024-03-07T07:43:36.357 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.001490089, 2024-03-07T07:43:36.357 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-07T07:43:36.357 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-03-07T07:43:36.357 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 56, 2024-03-07T07:43:36.357 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.002012839, 2024-03-07T07:43:36.357 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000035943 2024-03-07T07:43:36.357 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:36.357 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-03-07T07:43:36.358 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-03-07T07:43:36.358 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-03-07T07:43:36.358 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-03-07T07:43:36.358 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:36.358 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-07T07:43:36.358 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 87744 2024-03-07T07:43:36.358 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:36.358 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-2": { 2024-03-07T07:43:36.358 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 81, 2024-03-07T07:43:36.358 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 76, 2024-03-07T07:43:36.358 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 708672, 2024-03-07T07:43:36.358 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 44528, 2024-03-07T07:43:36.358 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 2, 2024-03-07T07:43:36.358 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 1, 2024-03-07T07:43:36.359 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.014501515, 2024-03-07T07:43:36.359 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.004396919, 2024-03-07T07:43:36.359 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.013017173, 2024-03-07T07:43:36.359 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.001311297, 2024-03-07T07:43:36.359 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-03-07T07:43:36.359 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 75, 2024-03-07T07:43:36.359 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.004016649, 2024-03-07T07:43:36.359 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000053555 2024-03-07T07:43:36.359 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:36.359 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-03-07T07:43:36.359 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-03-07T07:43:36.359 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-03-07T07:43:36.359 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-03-07T07:43:36.359 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:36.359 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 708672, 2024-03-07T07:43:36.360 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 44528 2024-03-07T07:43:36.360 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:36.360 INFO:teuthology.orchestra.run.smithi160.stdout: "cct": { 2024-03-07T07:43:36.360 INFO:teuthology.orchestra.run.smithi160.stdout: "total_workers": 0, 2024-03-07T07:43:36.360 INFO:teuthology.orchestra.run.smithi160.stdout: "unhealthy_workers": 0 2024-03-07T07:43:36.360 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:36.360 INFO:teuthology.orchestra.run.smithi160.stdout: "mempool": { 2024-03-07T07:43:36.360 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_bytes": 0, 2024-03-07T07:43:36.360 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_items": 0, 2024-03-07T07:43:36.360 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_bytes": 0, 2024-03-07T07:43:36.360 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_items": 0, 2024-03-07T07:43:36.360 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_bytes": 0, 2024-03-07T07:43:36.360 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_items": 0, 2024-03-07T07:43:36.360 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-07T07:43:36.361 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_items": 0, 2024-03-07T07:43:36.361 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-07T07:43:36.361 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_items": 0, 2024-03-07T07:43:36.361 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_bytes": 0, 2024-03-07T07:43:36.361 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_items": 0, 2024-03-07T07:43:36.361 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-07T07:43:36.361 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_items": 0, 2024-03-07T07:43:36.361 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_bytes": 0, 2024-03-07T07:43:36.361 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_items": 0, 2024-03-07T07:43:36.361 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_bytes": 0, 2024-03-07T07:43:36.361 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_items": 0, 2024-03-07T07:43:36.361 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-07T07:43:36.361 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_items": 0, 2024-03-07T07:43:36.361 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-07T07:43:36.362 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_items": 0, 2024-03-07T07:43:36.362 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_bytes": 0, 2024-03-07T07:43:36.362 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_items": 0, 2024-03-07T07:43:36.362 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_bytes": 0, 2024-03-07T07:43:36.362 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_items": 0, 2024-03-07T07:43:36.362 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-07T07:43:36.362 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_items": 0, 2024-03-07T07:43:36.362 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_bytes": 0, 2024-03-07T07:43:36.362 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_items": 0, 2024-03-07T07:43:36.362 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_bytes": 0, 2024-03-07T07:43:36.362 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_items": 0, 2024-03-07T07:43:36.362 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_bytes": 0, 2024-03-07T07:43:36.362 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_items": 0, 2024-03-07T07:43:36.362 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_bytes": 0, 2024-03-07T07:43:36.362 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_items": 0, 2024-03-07T07:43:36.363 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_bytes": 148823, 2024-03-07T07:43:36.363 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_items": 27, 2024-03-07T07:43:36.363 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_bytes": 0, 2024-03-07T07:43:36.363 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_items": 0, 2024-03-07T07:43:36.363 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_bytes": 0, 2024-03-07T07:43:36.363 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_items": 0, 2024-03-07T07:43:36.363 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_bytes": 0, 2024-03-07T07:43:36.363 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_items": 0, 2024-03-07T07:43:36.363 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_bytes": 0, 2024-03-07T07:43:36.363 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_items": 0, 2024-03-07T07:43:36.363 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_bytes": 0, 2024-03-07T07:43:36.363 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_items": 0, 2024-03-07T07:43:36.363 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_bytes": 0, 2024-03-07T07:43:36.363 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_items": 0, 2024-03-07T07:43:36.363 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_bytes": 0, 2024-03-07T07:43:36.364 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_items": 0, 2024-03-07T07:43:36.364 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_bytes": 928, 2024-03-07T07:43:36.364 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_items": 2, 2024-03-07T07:43:36.364 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_bytes": 0, 2024-03-07T07:43:36.364 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_items": 0, 2024-03-07T07:43:36.364 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_bytes": 0, 2024-03-07T07:43:36.364 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_items": 0 2024-03-07T07:43:36.364 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:36.364 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-07T07:43:36.364 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-03-07T07:43:36.364 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 104857600, 2024-03-07T07:43:36.364 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-03-07T07:43:36.364 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 82, 2024-03-07T07:43:36.364 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 697264, 2024-03-07T07:43:36.365 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-03-07T07:43:36.365 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 82, 2024-03-07T07:43:36.365 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-03-07T07:43:36.365 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-03-07T07:43:36.365 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 82, 2024-03-07T07:43:36.365 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 697264, 2024-03-07T07:43:36.365 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-03-07T07:43:36.365 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-03-07T07:43:36.365 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-03-07T07:43:36.365 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-03-07T07:43:36.365 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:36.365 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:36.365 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-03-07T07:43:37.068 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-03-07T07:43:37.068 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-03-07T07:43:37.068 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-03-07T07:43:37.068 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-03-07T07:43:37.068 DEBUG:tasks.check_counter:Getting stats from d 2024-03-07T07:43:37.068 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph daemon mds.d perf dump 2024-03-07T07:43:37.319 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:43:37.427 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:43:37 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:43:37.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:43:37 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:43:37.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:43:37 smithi160 ceph-mon[31963]: pgmap v421: 97 pgs: 97 active+clean; 258 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 522 KiB/s wr, 12 op/s 2024-03-07T07:43:37.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:43:37 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:43:37.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:43:37 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:43:37.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:43:37 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:43:37.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:43:37 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:43:37.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:43:37 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:43:37.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:43:37 smithi178 ceph-mon[36650]: pgmap v421: 97 pgs: 97 active+clean; 258 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 522 KiB/s wr, 12 op/s 2024-03-07T07:43:37.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:43:37 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:43:37.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:43:37 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:43:37.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:43:37 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:43:37.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:43:37 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:43:37.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:43:37 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:43:37.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:43:37 smithi177 ceph-mon[36578]: pgmap v421: 97 pgs: 97 active+clean; 258 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 522 KiB/s wr, 12 op/s 2024-03-07T07:43:37.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:43:37 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-07T07:43:37.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:43:37 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-07T07:43:37.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:43:37 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' 2024-03-07T07:43:39.068 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-03-07T07:43:39.068 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-0": { 2024-03-07T07:43:39.068 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 91, 2024-03-07T07:43:39.068 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 84, 2024-03-07T07:43:39.068 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 713376, 2024-03-07T07:43:39.068 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 48720, 2024-03-07T07:43:39.068 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 4, 2024-03-07T07:43:39.068 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 0, 2024-03-07T07:43:39.068 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.017925609, 2024-03-07T07:43:39.068 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.005173954, 2024-03-07T07:43:39.069 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.015224319, 2024-03-07T07:43:39.069 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.001455943, 2024-03-07T07:43:39.069 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-03-07T07:43:39.069 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 84, 2024-03-07T07:43:39.069 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.004453661, 2024-03-07T07:43:39.069 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000053019 2024-03-07T07:43:39.069 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:39.069 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-03-07T07:43:39.069 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-03-07T07:43:39.069 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-03-07T07:43:39.069 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-03-07T07:43:39.069 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:39.069 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 713376, 2024-03-07T07:43:39.070 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 48720 2024-03-07T07:43:39.070 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:39.070 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-1": { 2024-03-07T07:43:39.070 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 1, 2024-03-07T07:43:39.070 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 62, 2024-03-07T07:43:39.070 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 160, 2024-03-07T07:43:39.070 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 88448, 2024-03-07T07:43:39.070 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 3, 2024-03-07T07:43:39.070 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 2, 2024-03-07T07:43:39.070 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.005416580, 2024-03-07T07:43:39.070 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.003631715, 2024-03-07T07:43:39.070 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.001112839, 2024-03-07T07:43:39.070 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-07T07:43:39.070 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-03-07T07:43:39.071 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 60, 2024-03-07T07:43:39.071 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.003148396, 2024-03-07T07:43:39.071 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000052473 2024-03-07T07:43:39.071 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:39.071 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-03-07T07:43:39.071 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-03-07T07:43:39.071 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-03-07T07:43:39.071 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-03-07T07:43:39.071 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:39.071 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-07T07:43:39.071 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 88448 2024-03-07T07:43:39.071 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:39.071 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-2": { 2024-03-07T07:43:39.071 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 0, 2024-03-07T07:43:39.071 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 2, 2024-03-07T07:43:39.072 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 0, 2024-03-07T07:43:39.072 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 0, 2024-03-07T07:43:39.072 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 2, 2024-03-07T07:43:39.072 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 1, 2024-03-07T07:43:39.072 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.000301985, 2024-03-07T07:43:39.072 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.000000000, 2024-03-07T07:43:39.072 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.000238961, 2024-03-07T07:43:39.072 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-07T07:43:39.072 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-03-07T07:43:39.072 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-03-07T07:43:39.072 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-03-07T07:43:39.072 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-03-07T07:43:39.072 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:39.073 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-03-07T07:43:39.073 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-03-07T07:43:39.073 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-03-07T07:43:39.073 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-03-07T07:43:39.073 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:39.073 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-07T07:43:39.073 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 0 2024-03-07T07:43:39.073 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:39.073 INFO:teuthology.orchestra.run.smithi160.stdout: "cct": { 2024-03-07T07:43:39.073 INFO:teuthology.orchestra.run.smithi160.stdout: "total_workers": 0, 2024-03-07T07:43:39.073 INFO:teuthology.orchestra.run.smithi160.stdout: "unhealthy_workers": 0 2024-03-07T07:43:39.073 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:39.073 INFO:teuthology.orchestra.run.smithi160.stdout: "mempool": { 2024-03-07T07:43:39.073 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_bytes": 0, 2024-03-07T07:43:39.074 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_items": 0, 2024-03-07T07:43:39.074 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_bytes": 0, 2024-03-07T07:43:39.074 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_items": 0, 2024-03-07T07:43:39.074 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_bytes": 0, 2024-03-07T07:43:39.074 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_items": 0, 2024-03-07T07:43:39.074 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-07T07:43:39.074 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_items": 0, 2024-03-07T07:43:39.074 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-07T07:43:39.074 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_items": 0, 2024-03-07T07:43:39.074 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_bytes": 0, 2024-03-07T07:43:39.074 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_items": 0, 2024-03-07T07:43:39.074 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-07T07:43:39.074 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_items": 0, 2024-03-07T07:43:39.074 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_bytes": 0, 2024-03-07T07:43:39.075 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_items": 0, 2024-03-07T07:43:39.075 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_bytes": 0, 2024-03-07T07:43:39.075 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_items": 0, 2024-03-07T07:43:39.075 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-07T07:43:39.075 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_items": 0, 2024-03-07T07:43:39.075 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-07T07:43:39.075 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_items": 0, 2024-03-07T07:43:39.075 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_bytes": 0, 2024-03-07T07:43:39.075 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_items": 0, 2024-03-07T07:43:39.075 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_bytes": 0, 2024-03-07T07:43:39.075 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_items": 0, 2024-03-07T07:43:39.075 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-07T07:43:39.075 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_items": 0, 2024-03-07T07:43:39.075 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_bytes": 0, 2024-03-07T07:43:39.076 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_items": 0, 2024-03-07T07:43:39.076 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_bytes": 0, 2024-03-07T07:43:39.076 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_items": 0, 2024-03-07T07:43:39.076 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_bytes": 0, 2024-03-07T07:43:39.076 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_items": 0, 2024-03-07T07:43:39.076 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_bytes": 0, 2024-03-07T07:43:39.076 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_items": 0, 2024-03-07T07:43:39.076 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_bytes": 148880, 2024-03-07T07:43:39.076 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_items": 29, 2024-03-07T07:43:39.076 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_bytes": 0, 2024-03-07T07:43:39.076 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_items": 0, 2024-03-07T07:43:39.076 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_bytes": 0, 2024-03-07T07:43:39.076 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_items": 0, 2024-03-07T07:43:39.076 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_bytes": 0, 2024-03-07T07:43:39.077 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_items": 0, 2024-03-07T07:43:39.077 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_bytes": 0, 2024-03-07T07:43:39.077 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_items": 0, 2024-03-07T07:43:39.077 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_bytes": 0, 2024-03-07T07:43:39.077 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_items": 0, 2024-03-07T07:43:39.077 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_bytes": 0, 2024-03-07T07:43:39.077 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_items": 0, 2024-03-07T07:43:39.077 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_bytes": 0, 2024-03-07T07:43:39.077 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_items": 0, 2024-03-07T07:43:39.077 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_bytes": 928, 2024-03-07T07:43:39.077 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_items": 2, 2024-03-07T07:43:39.077 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_bytes": 0, 2024-03-07T07:43:39.077 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_items": 0, 2024-03-07T07:43:39.077 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_bytes": 0, 2024-03-07T07:43:39.078 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_items": 0 2024-03-07T07:43:39.078 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:39.078 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-07T07:43:39.078 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-03-07T07:43:39.078 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 104857600, 2024-03-07T07:43:39.078 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-03-07T07:43:39.078 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 88, 2024-03-07T07:43:39.078 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 698044, 2024-03-07T07:43:39.078 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-03-07T07:43:39.078 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 88, 2024-03-07T07:43:39.078 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-03-07T07:43:39.078 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-03-07T07:43:39.078 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 88, 2024-03-07T07:43:39.078 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 698044, 2024-03-07T07:43:39.079 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-03-07T07:43:39.079 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-03-07T07:43:39.079 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-03-07T07:43:39.079 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-03-07T07:43:39.079 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:39.079 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:39.079 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-03-07T07:43:39.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:43:39 smithi160 ceph-mon[31963]: pgmap v422: 97 pgs: 97 active+clean; 258 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 2.8 MiB/s wr, 8 op/s 2024-03-07T07:43:39.764 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-03-07T07:43:39.764 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-03-07T07:43:39.764 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-03-07T07:43:39.764 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-03-07T07:43:39.764 DEBUG:tasks.check_counter:Getting stats from g 2024-03-07T07:43:39.764 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph daemon mds.g perf dump 2024-03-07T07:43:39.805 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:43:39 smithi178 ceph-mon[36650]: pgmap v422: 97 pgs: 97 active+clean; 258 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 2.8 MiB/s wr, 8 op/s 2024-03-07T07:43:39.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:43:39 smithi177 ceph-mon[36578]: pgmap v422: 97 pgs: 97 active+clean; 258 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 2.8 MiB/s wr, 8 op/s 2024-03-07T07:43:40.010 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:43:41.518 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:43:41 smithi160 ceph-mon[31963]: pgmap v423: 97 pgs: 97 active+clean; 257 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 14 op/s 2024-03-07T07:43:41.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:43:41 smithi178 ceph-mon[36650]: pgmap v423: 97 pgs: 97 active+clean; 257 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 14 op/s 2024-03-07T07:43:41.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:43:41 smithi177 ceph-mon[36578]: pgmap v423: 97 pgs: 97 active+clean; 257 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 14 op/s 2024-03-07T07:43:41.950 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-03-07T07:43:41.951 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-0": { 2024-03-07T07:43:41.951 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 4, 2024-03-07T07:43:41.951 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 4, 2024-03-07T07:43:41.951 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 3072, 2024-03-07T07:43:41.951 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 448, 2024-03-07T07:43:41.951 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 4, 2024-03-07T07:43:41.951 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 0, 2024-03-07T07:43:41.951 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.001378998, 2024-03-07T07:43:41.951 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.000040860, 2024-03-07T07:43:41.951 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.000497457, 2024-03-07T07:43:41.951 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-07T07:43:41.951 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-03-07T07:43:41.951 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 3, 2024-03-07T07:43:41.951 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000744516, 2024-03-07T07:43:41.951 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000248172 2024-03-07T07:43:41.952 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:41.952 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-03-07T07:43:41.952 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-03-07T07:43:41.952 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-03-07T07:43:41.952 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-03-07T07:43:41.952 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:41.952 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-03-07T07:43:41.952 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 448 2024-03-07T07:43:41.952 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:41.952 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-1": { 2024-03-07T07:43:41.952 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 83, 2024-03-07T07:43:41.952 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 78, 2024-03-07T07:43:41.952 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 709216, 2024-03-07T07:43:41.952 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 45776, 2024-03-07T07:43:41.953 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 2, 2024-03-07T07:43:41.953 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 1, 2024-03-07T07:43:41.953 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.016008138, 2024-03-07T07:43:41.953 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.005002807, 2024-03-07T07:43:41.953 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.013406860, 2024-03-07T07:43:41.953 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.001434130, 2024-03-07T07:43:41.953 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-03-07T07:43:41.953 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 77, 2024-03-07T07:43:41.953 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.003436620, 2024-03-07T07:43:41.953 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000044631 2024-03-07T07:43:41.953 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:41.953 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-03-07T07:43:41.953 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-03-07T07:43:41.953 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-03-07T07:43:41.954 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-03-07T07:43:41.954 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:41.954 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 709216, 2024-03-07T07:43:41.954 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 45776 2024-03-07T07:43:41.954 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:41.954 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-2": { 2024-03-07T07:43:41.954 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 1, 2024-03-07T07:43:41.954 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 59, 2024-03-07T07:43:41.954 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 160, 2024-03-07T07:43:41.954 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 87920, 2024-03-07T07:43:41.954 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 3, 2024-03-07T07:43:41.954 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 2, 2024-03-07T07:43:41.954 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.005521970, 2024-03-07T07:43:41.954 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.003769335, 2024-03-07T07:43:41.954 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.001121541, 2024-03-07T07:43:41.955 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-07T07:43:41.955 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-03-07T07:43:41.955 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 57, 2024-03-07T07:43:41.955 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.002912811, 2024-03-07T07:43:41.955 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000051101 2024-03-07T07:43:41.955 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:41.955 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-03-07T07:43:41.955 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-03-07T07:43:41.955 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-03-07T07:43:41.955 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-03-07T07:43:41.955 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:41.955 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-07T07:43:41.955 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 87920 2024-03-07T07:43:41.955 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:41.956 INFO:teuthology.orchestra.run.smithi160.stdout: "cct": { 2024-03-07T07:43:41.956 INFO:teuthology.orchestra.run.smithi160.stdout: "total_workers": 0, 2024-03-07T07:43:41.956 INFO:teuthology.orchestra.run.smithi160.stdout: "unhealthy_workers": 0 2024-03-07T07:43:41.956 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:41.956 INFO:teuthology.orchestra.run.smithi160.stdout: "mempool": { 2024-03-07T07:43:41.956 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_bytes": 0, 2024-03-07T07:43:41.956 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_items": 0, 2024-03-07T07:43:41.956 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_bytes": 0, 2024-03-07T07:43:41.956 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_items": 0, 2024-03-07T07:43:41.956 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_bytes": 0, 2024-03-07T07:43:41.956 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_items": 0, 2024-03-07T07:43:41.956 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-07T07:43:41.956 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_items": 0, 2024-03-07T07:43:41.956 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-07T07:43:41.957 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_items": 0, 2024-03-07T07:43:41.957 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_bytes": 0, 2024-03-07T07:43:41.957 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_items": 0, 2024-03-07T07:43:41.957 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-07T07:43:41.957 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_items": 0, 2024-03-07T07:43:41.957 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_bytes": 0, 2024-03-07T07:43:41.957 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_items": 0, 2024-03-07T07:43:41.957 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_bytes": 0, 2024-03-07T07:43:41.957 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_items": 0, 2024-03-07T07:43:41.957 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-07T07:43:41.957 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_items": 0, 2024-03-07T07:43:41.957 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-07T07:43:41.957 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_items": 0, 2024-03-07T07:43:41.958 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_bytes": 0, 2024-03-07T07:43:41.958 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_items": 0, 2024-03-07T07:43:41.958 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_bytes": 0, 2024-03-07T07:43:41.958 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_items": 0, 2024-03-07T07:43:41.958 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-07T07:43:41.958 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_items": 0, 2024-03-07T07:43:41.958 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_bytes": 0, 2024-03-07T07:43:41.958 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_items": 0, 2024-03-07T07:43:41.958 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_bytes": 0, 2024-03-07T07:43:41.958 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_items": 0, 2024-03-07T07:43:41.958 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_bytes": 0, 2024-03-07T07:43:41.958 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_items": 0, 2024-03-07T07:43:41.958 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_bytes": 0, 2024-03-07T07:43:41.958 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_items": 0, 2024-03-07T07:43:41.959 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_bytes": 148841, 2024-03-07T07:43:41.959 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_items": 28, 2024-03-07T07:43:41.959 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_bytes": 0, 2024-03-07T07:43:41.959 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_items": 0, 2024-03-07T07:43:41.959 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_bytes": 0, 2024-03-07T07:43:41.959 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_items": 0, 2024-03-07T07:43:41.959 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_bytes": 0, 2024-03-07T07:43:41.959 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_items": 0, 2024-03-07T07:43:41.959 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_bytes": 0, 2024-03-07T07:43:41.959 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_items": 0, 2024-03-07T07:43:41.959 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_bytes": 0, 2024-03-07T07:43:41.959 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_items": 0, 2024-03-07T07:43:41.959 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_bytes": 0, 2024-03-07T07:43:41.959 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_items": 0, 2024-03-07T07:43:41.959 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_bytes": 0, 2024-03-07T07:43:41.959 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_items": 0, 2024-03-07T07:43:41.960 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_bytes": 928, 2024-03-07T07:43:41.960 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_items": 2, 2024-03-07T07:43:41.960 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_bytes": 0, 2024-03-07T07:43:41.960 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_items": 0, 2024-03-07T07:43:41.960 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_bytes": 0, 2024-03-07T07:43:41.960 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_items": 0 2024-03-07T07:43:41.960 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:41.960 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-07T07:43:41.960 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-03-07T07:43:41.960 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 104857600, 2024-03-07T07:43:41.960 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-03-07T07:43:41.960 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 84, 2024-03-07T07:43:41.960 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 697524, 2024-03-07T07:43:41.960 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-03-07T07:43:41.961 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 84, 2024-03-07T07:43:41.961 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-03-07T07:43:41.961 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-03-07T07:43:41.961 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 84, 2024-03-07T07:43:41.961 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 697524, 2024-03-07T07:43:41.961 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-03-07T07:43:41.961 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-03-07T07:43:41.961 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-03-07T07:43:41.961 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-03-07T07:43:41.961 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:41.961 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:41.961 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-03-07T07:43:42.665 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-03-07T07:43:42.666 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-03-07T07:43:42.666 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-03-07T07:43:42.666 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-03-07T07:43:42.666 DEBUG:tasks.check_counter:Getting stats from j 2024-03-07T07:43:42.666 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph daemon mds.j perf dump 2024-03-07T07:43:42.908 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.a/config 2024-03-07T07:43:43.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:43:43 smithi160 ceph-mon[31963]: pgmap v424: 97 pgs: 97 active+clean; 257 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 17 MiB/s wr, 11 op/s 2024-03-07T07:43:43.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:43:43 smithi178 ceph-mon[36650]: pgmap v424: 97 pgs: 97 active+clean; 257 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 17 MiB/s wr, 11 op/s 2024-03-07T07:43:43.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:43:43 smithi177 ceph-mon[36578]: pgmap v424: 97 pgs: 97 active+clean; 257 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 17 MiB/s wr, 11 op/s 2024-03-07T07:43:44.803 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-03-07T07:43:44.803 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-0": { 2024-03-07T07:43:44.803 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 4, 2024-03-07T07:43:44.804 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 4, 2024-03-07T07:43:44.804 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 3072, 2024-03-07T07:43:44.804 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 448, 2024-03-07T07:43:44.804 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 4, 2024-03-07T07:43:44.804 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 0, 2024-03-07T07:43:44.804 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.001448281, 2024-03-07T07:43:44.804 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.000030180, 2024-03-07T07:43:44.804 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.000631970, 2024-03-07T07:43:44.804 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-07T07:43:44.804 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-03-07T07:43:44.804 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 3, 2024-03-07T07:43:44.804 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000742419, 2024-03-07T07:43:44.804 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000247473 2024-03-07T07:43:44.804 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:44.805 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-03-07T07:43:44.805 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-03-07T07:43:44.805 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-03-07T07:43:44.805 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-03-07T07:43:44.805 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:44.805 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-03-07T07:43:44.805 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 448 2024-03-07T07:43:44.805 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:44.805 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-1": { 2024-03-07T07:43:44.805 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 90, 2024-03-07T07:43:44.805 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 85, 2024-03-07T07:43:44.805 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 711184, 2024-03-07T07:43:44.805 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 49680, 2024-03-07T07:43:44.805 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 2, 2024-03-07T07:43:44.806 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 1, 2024-03-07T07:43:44.806 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.017013944, 2024-03-07T07:43:44.806 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.005063612, 2024-03-07T07:43:44.806 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.015353207, 2024-03-07T07:43:44.806 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.001479531, 2024-03-07T07:43:44.806 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-03-07T07:43:44.806 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 84, 2024-03-07T07:43:44.806 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.003965072, 2024-03-07T07:43:44.806 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000047203 2024-03-07T07:43:44.806 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:44.806 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-03-07T07:43:44.806 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-03-07T07:43:44.806 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-03-07T07:43:44.806 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-03-07T07:43:44.807 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:44.807 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 711184, 2024-03-07T07:43:44.807 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 49680 2024-03-07T07:43:44.807 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:44.807 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-2": { 2024-03-07T07:43:44.807 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 1, 2024-03-07T07:43:44.807 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 64, 2024-03-07T07:43:44.807 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 160, 2024-03-07T07:43:44.807 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 88800, 2024-03-07T07:43:44.807 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 3, 2024-03-07T07:43:44.807 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 2, 2024-03-07T07:43:44.807 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.005281842, 2024-03-07T07:43:44.807 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.003403880, 2024-03-07T07:43:44.808 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.001212835, 2024-03-07T07:43:44.808 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-07T07:43:44.808 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-03-07T07:43:44.808 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 62, 2024-03-07T07:43:44.808 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.003036759, 2024-03-07T07:43:44.808 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000048979 2024-03-07T07:43:44.808 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:44.808 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-03-07T07:43:44.808 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-03-07T07:43:44.808 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-03-07T07:43:44.808 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-03-07T07:43:44.808 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:44.808 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-07T07:43:44.808 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 88800 2024-03-07T07:43:44.809 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:44.809 INFO:teuthology.orchestra.run.smithi160.stdout: "cct": { 2024-03-07T07:43:44.809 INFO:teuthology.orchestra.run.smithi160.stdout: "total_workers": 0, 2024-03-07T07:43:44.809 INFO:teuthology.orchestra.run.smithi160.stdout: "unhealthy_workers": 0 2024-03-07T07:43:44.809 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:44.809 INFO:teuthology.orchestra.run.smithi160.stdout: "mempool": { 2024-03-07T07:43:44.809 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_bytes": 0, 2024-03-07T07:43:44.809 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_items": 0, 2024-03-07T07:43:44.809 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_bytes": 0, 2024-03-07T07:43:44.809 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_items": 0, 2024-03-07T07:43:44.809 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_bytes": 0, 2024-03-07T07:43:44.809 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_items": 0, 2024-03-07T07:43:44.809 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-07T07:43:44.809 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_items": 0, 2024-03-07T07:43:44.809 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-07T07:43:44.810 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_items": 0, 2024-03-07T07:43:44.810 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_bytes": 0, 2024-03-07T07:43:44.810 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_items": 0, 2024-03-07T07:43:44.810 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-07T07:43:44.810 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_items": 0, 2024-03-07T07:43:44.810 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_bytes": 0, 2024-03-07T07:43:44.810 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_items": 0, 2024-03-07T07:43:44.810 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_bytes": 0, 2024-03-07T07:43:44.810 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_items": 0, 2024-03-07T07:43:44.810 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-07T07:43:44.810 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_items": 0, 2024-03-07T07:43:44.810 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-07T07:43:44.810 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_items": 0, 2024-03-07T07:43:44.810 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_bytes": 0, 2024-03-07T07:43:44.811 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_items": 0, 2024-03-07T07:43:44.811 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_bytes": 0, 2024-03-07T07:43:44.811 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_items": 0, 2024-03-07T07:43:44.811 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-07T07:43:44.811 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_items": 0, 2024-03-07T07:43:44.811 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_bytes": 0, 2024-03-07T07:43:44.811 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_items": 0, 2024-03-07T07:43:44.811 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_bytes": 0, 2024-03-07T07:43:44.811 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_items": 0, 2024-03-07T07:43:44.811 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_bytes": 0, 2024-03-07T07:43:44.811 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_items": 0, 2024-03-07T07:43:44.811 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_bytes": 0, 2024-03-07T07:43:44.811 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_items": 0, 2024-03-07T07:43:44.811 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_bytes": 148903, 2024-03-07T07:43:44.811 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_items": 29, 2024-03-07T07:43:44.812 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_bytes": 0, 2024-03-07T07:43:44.812 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_items": 0, 2024-03-07T07:43:44.812 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_bytes": 0, 2024-03-07T07:43:44.812 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_items": 0, 2024-03-07T07:43:44.812 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_bytes": 0, 2024-03-07T07:43:44.812 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_items": 0, 2024-03-07T07:43:44.812 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_bytes": 0, 2024-03-07T07:43:44.812 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_items": 0, 2024-03-07T07:43:44.812 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_bytes": 0, 2024-03-07T07:43:44.812 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_items": 0, 2024-03-07T07:43:44.812 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_bytes": 0, 2024-03-07T07:43:44.812 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_items": 0, 2024-03-07T07:43:44.812 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_bytes": 0, 2024-03-07T07:43:44.812 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_items": 0, 2024-03-07T07:43:44.813 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_bytes": 928, 2024-03-07T07:43:44.813 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_items": 2, 2024-03-07T07:43:44.813 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_bytes": 0, 2024-03-07T07:43:44.813 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_items": 0, 2024-03-07T07:43:44.813 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_bytes": 0, 2024-03-07T07:43:44.813 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_items": 0 2024-03-07T07:43:44.813 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-03-07T07:43:44.813 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-07T07:43:44.813 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-03-07T07:43:44.813 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 104857600, 2024-03-07T07:43:44.813 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-03-07T07:43:44.813 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 91, 2024-03-07T07:43:44.813 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 698498, 2024-03-07T07:43:44.813 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-03-07T07:43:44.814 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 91, 2024-03-07T07:43:44.814 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-03-07T07:43:44.814 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-03-07T07:43:44.814 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 91, 2024-03-07T07:43:44.814 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 698498, 2024-03-07T07:43:44.814 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-03-07T07:43:44.814 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-03-07T07:43:44.814 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-03-07T07:43:44.814 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-03-07T07:43:44.814 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:44.814 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-03-07T07:43:44.814 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-03-07T07:43:45.468 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-03-07T07:43:45.468 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-03-07T07:43:45.468 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-03-07T07:43:45.468 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-03-07T07:43:45.468 DEBUG:tasks.check_counter:Getting stats from b 2024-03-07T07:43:45.468 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph daemon mds.b perf dump 2024-03-07T07:43:45.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:43:45 smithi160 ceph-mon[31963]: pgmap v425: 97 pgs: 97 active+clean; 257 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 20 MiB/s wr, 13 op/s 2024-03-07T07:43:45.716 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.b/config 2024-03-07T07:43:45.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:43:45 smithi178 ceph-mon[36650]: pgmap v425: 97 pgs: 97 active+clean; 257 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 20 MiB/s wr, 13 op/s 2024-03-07T07:43:45.837 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:43:45 smithi177 ceph-mon[36578]: pgmap v425: 97 pgs: 97 active+clean; 257 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 20 MiB/s wr, 13 op/s 2024-03-07T07:43:47.448 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:43:47 smithi177 ceph-mon[36578]: pgmap v426: 97 pgs: 97 active+clean; 237 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 20 MiB/s wr, 15 op/s 2024-03-07T07:43:47.574 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-03-07T07:43:47.574 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-0": { 2024-03-07T07:43:47.574 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 143, 2024-03-07T07:43:47.574 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 133, 2024-03-07T07:43:47.574 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 745513, 2024-03-07T07:43:47.574 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 103486, 2024-03-07T07:43:47.574 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 12, 2024-03-07T07:43:47.574 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 4, 2024-03-07T07:43:47.574 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.020504899, 2024-03-07T07:43:47.574 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.003839951, 2024-03-07T07:43:47.575 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.018246207, 2024-03-07T07:43:47.575 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.002830293, 2024-03-07T07:43:47.575 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2024-03-07T07:43:47.575 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 132, 2024-03-07T07:43:47.575 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.012092404, 2024-03-07T07:43:47.575 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000091609 2024-03-07T07:43:47.575 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.575 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2024-03-07T07:43:47.575 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 11, 2024-03-07T07:43:47.575 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000001256, 2024-03-07T07:43:47.575 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000114 2024-03-07T07:43:47.575 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.575 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 741568, 2024-03-07T07:43:47.575 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 52464 2024-03-07T07:43:47.575 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.575 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-1": { 2024-03-07T07:43:47.576 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 218, 2024-03-07T07:43:47.576 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 280, 2024-03-07T07:43:47.576 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 69016, 2024-03-07T07:43:47.576 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 253389, 2024-03-07T07:43:47.576 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 7, 2024-03-07T07:43:47.576 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 5, 2024-03-07T07:43:47.576 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.017833441, 2024-03-07T07:43:47.576 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.006962406, 2024-03-07T07:43:47.576 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.009571333, 2024-03-07T07:43:47.576 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.001082801, 2024-03-07T07:43:47.576 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2024-03-07T07:43:47.576 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 279, 2024-03-07T07:43:47.576 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.021831042, 2024-03-07T07:43:47.576 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000078247 2024-03-07T07:43:47.577 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.577 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2024-03-07T07:43:47.577 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 399, 2024-03-07T07:43:47.577 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000070928, 2024-03-07T07:43:47.577 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000177 2024-03-07T07:43:47.577 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.577 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-03-07T07:43:47.577 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 183872 2024-03-07T07:43:47.577 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.577 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-2": { 2024-03-07T07:43:47.577 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 14, 2024-03-07T07:43:47.577 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 16, 2024-03-07T07:43:47.577 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 3871, 2024-03-07T07:43:47.577 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 12055, 2024-03-07T07:43:47.578 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 6, 2024-03-07T07:43:47.578 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 4, 2024-03-07T07:43:47.578 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.002866651, 2024-03-07T07:43:47.578 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.000195039, 2024-03-07T07:43:47.578 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.002093947, 2024-03-07T07:43:47.578 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.000432763, 2024-03-07T07:43:47.578 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2024-03-07T07:43:47.578 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 14, 2024-03-07T07:43:47.578 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.014091019, 2024-03-07T07:43:47.578 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.001006501 2024-03-07T07:43:47.578 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.578 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2024-03-07T07:43:47.578 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:47.579 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:47.579 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:47.579 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.579 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-07T07:43:47.579 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 0 2024-03-07T07:43:47.579 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.579 INFO:teuthology.orchestra.run.smithi177.stdout: "cct": { 2024-03-07T07:43:47.579 INFO:teuthology.orchestra.run.smithi177.stdout: "total_workers": 1, 2024-03-07T07:43:47.579 INFO:teuthology.orchestra.run.smithi177.stdout: "unhealthy_workers": 0 2024-03-07T07:43:47.579 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.579 INFO:teuthology.orchestra.run.smithi177.stdout: "finisher-MDSRank": { 2024-03-07T07:43:47.579 INFO:teuthology.orchestra.run.smithi177.stdout: "queue_len": 0, 2024-03-07T07:43:47.579 INFO:teuthology.orchestra.run.smithi177.stdout: "complete_latency": { 2024-03-07T07:43:47.579 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 31, 2024-03-07T07:43:47.579 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.001931371, 2024-03-07T07:43:47.579 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000062302 2024-03-07T07:43:47.580 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-07T07:43:47.580 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.580 INFO:teuthology.orchestra.run.smithi177.stdout: "finisher-PurgeQueue": { 2024-03-07T07:43:47.580 INFO:teuthology.orchestra.run.smithi177.stdout: "queue_len": 0, 2024-03-07T07:43:47.580 INFO:teuthology.orchestra.run.smithi177.stdout: "complete_latency": { 2024-03-07T07:43:47.580 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 2, 2024-03-07T07:43:47.580 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000009282, 2024-03-07T07:43:47.580 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000004641 2024-03-07T07:43:47.580 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-07T07:43:47.580 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.580 INFO:teuthology.orchestra.run.smithi177.stdout: "mds": { 2024-03-07T07:43:47.580 INFO:teuthology.orchestra.run.smithi177.stdout: "request": 1, 2024-03-07T07:43:47.580 INFO:teuthology.orchestra.run.smithi177.stdout: "reply": 0, 2024-03-07T07:43:47.581 INFO:teuthology.orchestra.run.smithi177.stdout: "reply_latency": { 2024-03-07T07:43:47.581 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:47.581 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:47.581 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:47.581 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.581 INFO:teuthology.orchestra.run.smithi177.stdout: "slow_reply": 0, 2024-03-07T07:43:47.581 INFO:teuthology.orchestra.run.smithi177.stdout: "forward": 1, 2024-03-07T07:43:47.581 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_fetch_complete": 0, 2024-03-07T07:43:47.581 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_fetch_keys": 0, 2024-03-07T07:43:47.581 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_commit": 11, 2024-03-07T07:43:47.581 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_split": 0, 2024-03-07T07:43:47.581 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_merge": 0, 2024-03-07T07:43:47.581 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes": 16, 2024-03-07T07:43:47.581 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_top": 0, 2024-03-07T07:43:47.581 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_bottom": 16, 2024-03-07T07:43:47.581 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_pin_tail": 0, 2024-03-07T07:43:47.582 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_pinned": 13, 2024-03-07T07:43:47.582 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_expired": 4, 2024-03-07T07:43:47.582 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_with_caps": 0, 2024-03-07T07:43:47.582 INFO:teuthology.orchestra.run.smithi177.stdout: "caps": 0, 2024-03-07T07:43:47.582 INFO:teuthology.orchestra.run.smithi177.stdout: "subtrees": 3, 2024-03-07T07:43:47.582 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse": 7, 2024-03-07T07:43:47.582 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_hit": 3, 2024-03-07T07:43:47.582 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_forward": 1, 2024-03-07T07:43:47.582 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_discover": 3, 2024-03-07T07:43:47.582 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_dir_fetch": 0, 2024-03-07T07:43:47.582 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_remote_ino": 0, 2024-03-07T07:43:47.582 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_lock": 0, 2024-03-07T07:43:47.582 INFO:teuthology.orchestra.run.smithi177.stdout: "load_cent": 0, 2024-03-07T07:43:47.582 INFO:teuthology.orchestra.run.smithi177.stdout: "q": 0, 2024-03-07T07:43:47.582 INFO:teuthology.orchestra.run.smithi177.stdout: "exported": 0, 2024-03-07T07:43:47.582 INFO:teuthology.orchestra.run.smithi177.stdout: "exported_inodes": 0, 2024-03-07T07:43:47.583 INFO:teuthology.orchestra.run.smithi177.stdout: "imported": 0, 2024-03-07T07:43:47.583 INFO:teuthology.orchestra.run.smithi177.stdout: "imported_inodes": 0, 2024-03-07T07:43:47.583 INFO:teuthology.orchestra.run.smithi177.stdout: "openino_dir_fetch": 0, 2024-03-07T07:43:47.583 INFO:teuthology.orchestra.run.smithi177.stdout: "openino_backtrace_fetch": 0, 2024-03-07T07:43:47.583 INFO:teuthology.orchestra.run.smithi177.stdout: "openino_peer_discover": 0, 2024-03-07T07:43:47.583 INFO:teuthology.orchestra.run.smithi177.stdout: "root_rfiles": 7, 2024-03-07T07:43:47.583 INFO:teuthology.orchestra.run.smithi177.stdout: "root_rbytes": 269487290, 2024-03-07T07:43:47.583 INFO:teuthology.orchestra.run.smithi177.stdout: "root_rsnaps": 0, 2024-03-07T07:43:47.583 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_backtrace_fetch": 0, 2024-03-07T07:43:47.583 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_set_tag": 0, 2024-03-07T07:43:47.583 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_backtrace_repaired": 0, 2024-03-07T07:43:47.583 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_inotable_repaired": 0, 2024-03-07T07:43:47.583 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_dir_inodes": 0, 2024-03-07T07:43:47.583 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_dir_base_inodes": 0, 2024-03-07T07:43:47.583 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_dirfrag_rstats": 0, 2024-03-07T07:43:47.583 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_file_inodes": 0, 2024-03-07T07:43:47.584 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_inode_file_caps": 0, 2024-03-07T07:43:47.584 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_revoke": 0, 2024-03-07T07:43:47.584 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_grant": 0, 2024-03-07T07:43:47.584 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_trunc": 0, 2024-03-07T07:43:47.584 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-07T07:43:47.584 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-07T07:43:47.584 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_caps": 0, 2024-03-07T07:43:47.584 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_caps_dirty": 0, 2024-03-07T07:43:47.584 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_cap_release": 0, 2024-03-07T07:43:47.584 INFO:teuthology.orchestra.run.smithi177.stdout: "process_request_cap_release": 0 2024-03-07T07:43:47.584 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.584 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_cache": { 2024-03-07T07:43:47.584 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_update": 0, 2024-03-07T07:43:47.584 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_update_receipt": 9, 2024-03-07T07:43:47.584 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_try_discover": 6, 2024-03-07T07:43:47.584 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_send_discover": 7, 2024-03-07T07:43:47.585 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_handle_discover": 0, 2024-03-07T07:43:47.585 INFO:teuthology.orchestra.run.smithi177.stdout: "num_strays": 0, 2024-03-07T07:43:47.585 INFO:teuthology.orchestra.run.smithi177.stdout: "num_strays_delayed": 0, 2024-03-07T07:43:47.585 INFO:teuthology.orchestra.run.smithi177.stdout: "num_strays_enqueuing": 0, 2024-03-07T07:43:47.585 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_created": 0, 2024-03-07T07:43:47.585 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_enqueued": 0, 2024-03-07T07:43:47.585 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_reintegrated": 0, 2024-03-07T07:43:47.585 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_migrated": 0, 2024-03-07T07:43:47.585 INFO:teuthology.orchestra.run.smithi177.stdout: "num_recovering_processing": 0, 2024-03-07T07:43:47.585 INFO:teuthology.orchestra.run.smithi177.stdout: "num_recovering_enqueued": 0, 2024-03-07T07:43:47.585 INFO:teuthology.orchestra.run.smithi177.stdout: "num_recovering_prioritized": 0, 2024-03-07T07:43:47.585 INFO:teuthology.orchestra.run.smithi177.stdout: "recovery_started": 0, 2024-03-07T07:43:47.585 INFO:teuthology.orchestra.run.smithi177.stdout: "recovery_completed": 0, 2024-03-07T07:43:47.585 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_enqueue_scrub": 0, 2024-03-07T07:43:47.585 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_exportdir": 0, 2024-03-07T07:43:47.585 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_flush": 0, 2024-03-07T07:43:47.586 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_fragmentdir": 0, 2024-03-07T07:43:47.586 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_fragstats": 0, 2024-03-07T07:43:47.586 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_inodestats": 0 2024-03-07T07:43:47.586 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.586 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_log": { 2024-03-07T07:43:47.586 INFO:teuthology.orchestra.run.smithi177.stdout: "evlrg": 0, 2024-03-07T07:43:47.586 INFO:teuthology.orchestra.run.smithi177.stdout: "evadd": 3, 2024-03-07T07:43:47.586 INFO:teuthology.orchestra.run.smithi177.stdout: "evex": 0, 2024-03-07T07:43:47.586 INFO:teuthology.orchestra.run.smithi177.stdout: "evtrm": 0, 2024-03-07T07:43:47.586 INFO:teuthology.orchestra.run.smithi177.stdout: "ev": 3, 2024-03-07T07:43:47.586 INFO:teuthology.orchestra.run.smithi177.stdout: "evexg": 0, 2024-03-07T07:43:47.586 INFO:teuthology.orchestra.run.smithi177.stdout: "evexd": 0, 2024-03-07T07:43:47.586 INFO:teuthology.orchestra.run.smithi177.stdout: "segadd": 2, 2024-03-07T07:43:47.586 INFO:teuthology.orchestra.run.smithi177.stdout: "segex": 0, 2024-03-07T07:43:47.586 INFO:teuthology.orchestra.run.smithi177.stdout: "segtrm": 0, 2024-03-07T07:43:47.586 INFO:teuthology.orchestra.run.smithi177.stdout: "seg": 2, 2024-03-07T07:43:47.587 INFO:teuthology.orchestra.run.smithi177.stdout: "segmjr": 2, 2024-03-07T07:43:47.587 INFO:teuthology.orchestra.run.smithi177.stdout: "segexg": 0, 2024-03-07T07:43:47.587 INFO:teuthology.orchestra.run.smithi177.stdout: "segexd": 0, 2024-03-07T07:43:47.587 INFO:teuthology.orchestra.run.smithi177.stdout: "expos": 4194304, 2024-03-07T07:43:47.587 INFO:teuthology.orchestra.run.smithi177.stdout: "wrpos": 4195200, 2024-03-07T07:43:47.587 INFO:teuthology.orchestra.run.smithi177.stdout: "rdpos": 0, 2024-03-07T07:43:47.587 INFO:teuthology.orchestra.run.smithi177.stdout: "jlat": { 2024-03-07T07:43:47.587 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 2, 2024-03-07T07:43:47.587 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.002897242, 2024-03-07T07:43:47.587 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.001448621 2024-03-07T07:43:47.587 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.587 INFO:teuthology.orchestra.run.smithi177.stdout: "replayed": 0 2024-03-07T07:43:47.587 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.587 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_mem": { 2024-03-07T07:43:47.588 INFO:teuthology.orchestra.run.smithi177.stdout: "ino": 20, 2024-03-07T07:43:47.588 INFO:teuthology.orchestra.run.smithi177.stdout: "ino+": 22, 2024-03-07T07:43:47.588 INFO:teuthology.orchestra.run.smithi177.stdout: "ino-": 2, 2024-03-07T07:43:47.588 INFO:teuthology.orchestra.run.smithi177.stdout: "dir": 19, 2024-03-07T07:43:47.588 INFO:teuthology.orchestra.run.smithi177.stdout: "dir+": 22, 2024-03-07T07:43:47.588 INFO:teuthology.orchestra.run.smithi177.stdout: "dir-": 3, 2024-03-07T07:43:47.588 INFO:teuthology.orchestra.run.smithi177.stdout: "dn": 16, 2024-03-07T07:43:47.588 INFO:teuthology.orchestra.run.smithi177.stdout: "dn+": 20, 2024-03-07T07:43:47.588 INFO:teuthology.orchestra.run.smithi177.stdout: "dn-": 4, 2024-03-07T07:43:47.588 INFO:teuthology.orchestra.run.smithi177.stdout: "cap": 0, 2024-03-07T07:43:47.588 INFO:teuthology.orchestra.run.smithi177.stdout: "cap+": 0, 2024-03-07T07:43:47.588 INFO:teuthology.orchestra.run.smithi177.stdout: "cap-": 0, 2024-03-07T07:43:47.588 INFO:teuthology.orchestra.run.smithi177.stdout: "rss": 48604, 2024-03-07T07:43:47.588 INFO:teuthology.orchestra.run.smithi177.stdout: "heap": 223612 2024-03-07T07:43:47.588 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.589 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_server": { 2024-03-07T07:43:47.589 INFO:teuthology.orchestra.run.smithi177.stdout: "dispatch_client_request": 1, 2024-03-07T07:43:47.589 INFO:teuthology.orchestra.run.smithi177.stdout: "dispatch_server_request": 0, 2024-03-07T07:43:47.589 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_request": 1, 2024-03-07T07:43:47.589 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_session": 3, 2024-03-07T07:43:47.589 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_peer_request": 0, 2024-03-07T07:43:47.589 INFO:teuthology.orchestra.run.smithi177.stdout: "req_create_latency": { 2024-03-07T07:43:47.589 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:47.589 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:47.589 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:47.589 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.589 INFO:teuthology.orchestra.run.smithi177.stdout: "req_getattr_latency": { 2024-03-07T07:43:47.589 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:47.589 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:47.589 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:47.589 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.590 INFO:teuthology.orchestra.run.smithi177.stdout: "req_getfilelock_latency": { 2024-03-07T07:43:47.590 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:47.590 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:47.590 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:47.590 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.590 INFO:teuthology.orchestra.run.smithi177.stdout: "req_link_latency": { 2024-03-07T07:43:47.590 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:47.590 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:47.590 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:47.590 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.590 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookup_latency": { 2024-03-07T07:43:47.590 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:47.590 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:47.591 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:47.591 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.591 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookuphash_latency": { 2024-03-07T07:43:47.591 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:47.591 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:47.591 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:47.591 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.591 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupino_latency": { 2024-03-07T07:43:47.591 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:47.591 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:47.591 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:47.591 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.591 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupname_latency": { 2024-03-07T07:43:47.591 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:47.591 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:47.591 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:47.592 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.592 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupparent_latency": { 2024-03-07T07:43:47.592 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:47.592 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:47.592 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:47.592 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.592 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupsnap_latency": { 2024-03-07T07:43:47.592 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:47.592 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:47.592 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:47.592 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.592 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lssnap_latency": { 2024-03-07T07:43:47.592 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:47.592 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:47.592 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:47.593 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.593 INFO:teuthology.orchestra.run.smithi177.stdout: "req_mkdir_latency": { 2024-03-07T07:43:47.593 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:47.593 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:47.593 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:47.593 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.593 INFO:teuthology.orchestra.run.smithi177.stdout: "req_mknod_latency": { 2024-03-07T07:43:47.593 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:47.594 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:47.594 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:47.594 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.594 INFO:teuthology.orchestra.run.smithi177.stdout: "req_mksnap_latency": { 2024-03-07T07:43:47.594 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:47.594 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:47.594 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:47.594 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.594 INFO:teuthology.orchestra.run.smithi177.stdout: "req_open_latency": { 2024-03-07T07:43:47.594 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:47.594 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:47.594 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:47.594 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.594 INFO:teuthology.orchestra.run.smithi177.stdout: "req_readdir_latency": { 2024-03-07T07:43:47.594 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:47.595 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:47.595 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:47.595 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.595 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rename_latency": { 2024-03-07T07:43:47.595 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:47.595 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:47.595 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:47.595 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.595 INFO:teuthology.orchestra.run.smithi177.stdout: "req_renamesnap_latency": { 2024-03-07T07:43:47.595 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:47.595 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:47.595 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:47.595 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.595 INFO:teuthology.orchestra.run.smithi177.stdout: "req_snapdiff_latency": { 2024-03-07T07:43:47.595 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:47.595 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:47.596 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:47.596 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.596 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rmdir_latency": { 2024-03-07T07:43:47.596 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:47.596 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:47.596 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:47.596 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.596 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rmsnap_latency": { 2024-03-07T07:43:47.596 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:47.596 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:47.596 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:47.596 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.596 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rmxattr_latency": { 2024-03-07T07:43:47.596 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:47.596 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:47.597 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:47.597 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.597 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setattr_latency": { 2024-03-07T07:43:47.597 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:47.597 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:47.597 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:47.597 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.597 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setdirlayout_latency": { 2024-03-07T07:43:47.597 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:47.597 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:47.597 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:47.597 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.597 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setfilelock_latency": { 2024-03-07T07:43:47.597 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:47.597 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:47.598 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:47.598 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.598 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setlayout_latency": { 2024-03-07T07:43:47.598 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:47.598 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:47.598 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:47.598 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.598 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setxattr_latency": { 2024-03-07T07:43:47.598 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:47.598 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:47.598 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:47.598 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.598 INFO:teuthology.orchestra.run.smithi177.stdout: "req_symlink_latency": { 2024-03-07T07:43:47.598 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:47.599 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:47.599 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:47.599 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.599 INFO:teuthology.orchestra.run.smithi177.stdout: "req_unlink_latency": { 2024-03-07T07:43:47.599 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:47.599 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:47.599 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:47.599 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.599 INFO:teuthology.orchestra.run.smithi177.stdout: "cap_revoke_eviction": 0, 2024-03-07T07:43:47.599 INFO:teuthology.orchestra.run.smithi177.stdout: "cap_acquisition_throttle": 0, 2024-03-07T07:43:47.599 INFO:teuthology.orchestra.run.smithi177.stdout: "req_getvxattr_latency": { 2024-03-07T07:43:47.599 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:47.599 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:47.599 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:47.599 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-07T07:43:47.600 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.600 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_sessions": { 2024-03-07T07:43:47.600 INFO:teuthology.orchestra.run.smithi177.stdout: "session_count": 1, 2024-03-07T07:43:47.600 INFO:teuthology.orchestra.run.smithi177.stdout: "session_add": 1, 2024-03-07T07:43:47.600 INFO:teuthology.orchestra.run.smithi177.stdout: "session_remove": 0, 2024-03-07T07:43:47.600 INFO:teuthology.orchestra.run.smithi177.stdout: "sessions_open": 1, 2024-03-07T07:43:47.600 INFO:teuthology.orchestra.run.smithi177.stdout: "sessions_stale": 0, 2024-03-07T07:43:47.600 INFO:teuthology.orchestra.run.smithi177.stdout: "total_load": 0, 2024-03-07T07:43:47.600 INFO:teuthology.orchestra.run.smithi177.stdout: "average_load": 0, 2024-03-07T07:43:47.600 INFO:teuthology.orchestra.run.smithi177.stdout: "avg_session_uptime": 152, 2024-03-07T07:43:47.600 INFO:teuthology.orchestra.run.smithi177.stdout: "mdthresh_evicted": 0 2024-03-07T07:43:47.600 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.601 INFO:teuthology.orchestra.run.smithi177.stdout: "mempool": { 2024-03-07T07:43:47.601 INFO:teuthology.orchestra.run.smithi177.stdout: "bloom_filter_bytes": 0, 2024-03-07T07:43:47.601 INFO:teuthology.orchestra.run.smithi177.stdout: "bloom_filter_items": 0, 2024-03-07T07:43:47.601 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_alloc_bytes": 0, 2024-03-07T07:43:47.601 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_alloc_items": 0, 2024-03-07T07:43:47.601 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_data_bytes": 0, 2024-03-07T07:43:47.601 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_data_items": 0, 2024-03-07T07:43:47.601 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-07T07:43:47.601 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_onode_items": 0, 2024-03-07T07:43:47.601 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-07T07:43:47.601 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_meta_items": 0, 2024-03-07T07:43:47.601 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_other_bytes": 0, 2024-03-07T07:43:47.601 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_other_items": 0, 2024-03-07T07:43:47.602 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-07T07:43:47.602 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_buffer_items": 0, 2024-03-07T07:43:47.602 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_extent_bytes": 0, 2024-03-07T07:43:47.602 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_extent_items": 0, 2024-03-07T07:43:47.602 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_blob_bytes": 0, 2024-03-07T07:43:47.602 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_blob_items": 0, 2024-03-07T07:43:47.602 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-07T07:43:47.602 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_shared_blob_items": 0, 2024-03-07T07:43:47.602 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-07T07:43:47.602 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_inline_bl_items": 0, 2024-03-07T07:43:47.602 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_bytes": 0, 2024-03-07T07:43:47.602 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_items": 0, 2024-03-07T07:43:47.602 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_txc_bytes": 0, 2024-03-07T07:43:47.602 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_txc_items": 0, 2024-03-07T07:43:47.603 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-07T07:43:47.603 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_deferred_items": 0, 2024-03-07T07:43:47.603 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_bytes": 0, 2024-03-07T07:43:47.603 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_items": 0, 2024-03-07T07:43:47.603 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_bytes": 0, 2024-03-07T07:43:47.603 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_items": 0, 2024-03-07T07:43:47.603 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_reader_bytes": 0, 2024-03-07T07:43:47.603 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_reader_items": 0, 2024-03-07T07:43:47.603 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_writer_bytes": 0, 2024-03-07T07:43:47.603 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_writer_items": 0, 2024-03-07T07:43:47.603 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_anon_bytes": 238327, 2024-03-07T07:43:47.603 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_anon_items": 77, 2024-03-07T07:43:47.603 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_meta_bytes": 0, 2024-03-07T07:43:47.603 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_meta_items": 0, 2024-03-07T07:43:47.603 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_bytes": 0, 2024-03-07T07:43:47.604 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_items": 0, 2024-03-07T07:43:47.604 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mapbl_bytes": 0, 2024-03-07T07:43:47.604 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mapbl_items": 0, 2024-03-07T07:43:47.604 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pglog_bytes": 0, 2024-03-07T07:43:47.604 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pglog_items": 0, 2024-03-07T07:43:47.604 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_bytes": 7624, 2024-03-07T07:43:47.604 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_items": 160, 2024-03-07T07:43:47.604 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_mapping_bytes": 0, 2024-03-07T07:43:47.604 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_mapping_items": 0, 2024-03-07T07:43:47.604 INFO:teuthology.orchestra.run.smithi177.stdout: "pgmap_bytes": 0, 2024-03-07T07:43:47.604 INFO:teuthology.orchestra.run.smithi177.stdout: "pgmap_items": 0, 2024-03-07T07:43:47.604 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_co_bytes": 79653, 2024-03-07T07:43:47.604 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_co_items": 323, 2024-03-07T07:43:47.604 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_1_bytes": 0, 2024-03-07T07:43:47.605 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_1_items": 0, 2024-03-07T07:43:47.605 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_2_bytes": 0, 2024-03-07T07:43:47.605 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_2_items": 0 2024-03-07T07:43:47.605 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.605 INFO:teuthology.orchestra.run.smithi177.stdout: "objecter": { 2024-03-07T07:43:47.605 INFO:teuthology.orchestra.run.smithi177.stdout: "op_active": 0, 2024-03-07T07:43:47.605 INFO:teuthology.orchestra.run.smithi177.stdout: "op_laggy": 0, 2024-03-07T07:43:47.605 INFO:teuthology.orchestra.run.smithi177.stdout: "op_send": 37, 2024-03-07T07:43:47.605 INFO:teuthology.orchestra.run.smithi177.stdout: "op_send_bytes": 11108, 2024-03-07T07:43:47.605 INFO:teuthology.orchestra.run.smithi177.stdout: "op_resend": 28, 2024-03-07T07:43:47.605 INFO:teuthology.orchestra.run.smithi177.stdout: "op_reply": 37, 2024-03-07T07:43:47.605 INFO:teuthology.orchestra.run.smithi177.stdout: "op_latency": { 2024-03-07T07:43:47.605 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 37, 2024-03-07T07:43:47.605 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.216994141, 2024-03-07T07:43:47.605 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.005864706 2024-03-07T07:43:47.606 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.606 INFO:teuthology.orchestra.run.smithi177.stdout: "op_inflight": 0, 2024-03-07T07:43:47.606 INFO:teuthology.orchestra.run.smithi177.stdout: "oplen_avg": { 2024-03-07T07:43:47.606 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 37, 2024-03-07T07:43:47.606 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 59 2024-03-07T07:43:47.606 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.606 INFO:teuthology.orchestra.run.smithi177.stdout: "op": 37, 2024-03-07T07:43:47.606 INFO:teuthology.orchestra.run.smithi177.stdout: "op_r": 0, 2024-03-07T07:43:47.606 INFO:teuthology.orchestra.run.smithi177.stdout: "op_w": 37, 2024-03-07T07:43:47.606 INFO:teuthology.orchestra.run.smithi177.stdout: "op_rmw": 0, 2024-03-07T07:43:47.606 INFO:teuthology.orchestra.run.smithi177.stdout: "op_pg": 0, 2024-03-07T07:43:47.606 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_stat": 0, 2024-03-07T07:43:47.606 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_create": 10, 2024-03-07T07:43:47.606 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_read": 0, 2024-03-07T07:43:47.606 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_write": 2, 2024-03-07T07:43:47.607 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_writefull": 6, 2024-03-07T07:43:47.607 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_writesame": 0, 2024-03-07T07:43:47.607 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_append": 0, 2024-03-07T07:43:47.607 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_zero": 0, 2024-03-07T07:43:47.607 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_truncate": 0, 2024-03-07T07:43:47.607 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_delete": 6, 2024-03-07T07:43:47.607 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_mapext": 0, 2024-03-07T07:43:47.607 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_sparse_read": 0, 2024-03-07T07:43:47.607 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_clonerange": 0, 2024-03-07T07:43:47.607 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_getxattr": 0, 2024-03-07T07:43:47.607 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_setxattr": 20, 2024-03-07T07:43:47.607 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_cmpxattr": 0, 2024-03-07T07:43:47.607 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_rmxattr": 0, 2024-03-07T07:43:47.607 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_resetxattrs": 0, 2024-03-07T07:43:47.607 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_call": 0, 2024-03-07T07:43:47.607 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_watch": 0, 2024-03-07T07:43:47.608 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_notify": 0, 2024-03-07T07:43:47.608 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_src_cmpxattr": 0, 2024-03-07T07:43:47.608 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_pgls": 0, 2024-03-07T07:43:47.608 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_pgls_filter": 0, 2024-03-07T07:43:47.608 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_other": 0, 2024-03-07T07:43:47.608 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_active": 0, 2024-03-07T07:43:47.608 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_send": 0, 2024-03-07T07:43:47.608 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_resend": 0, 2024-03-07T07:43:47.608 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_ping": 0, 2024-03-07T07:43:47.608 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_active": 0, 2024-03-07T07:43:47.608 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_send": 0, 2024-03-07T07:43:47.608 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_resend": 0, 2024-03-07T07:43:47.608 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_active": 0, 2024-03-07T07:43:47.608 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_send": 0, 2024-03-07T07:43:47.609 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_resend": 0, 2024-03-07T07:43:47.609 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_active": 0, 2024-03-07T07:43:47.609 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_send": 0, 2024-03-07T07:43:47.609 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_resend": 0, 2024-03-07T07:43:47.609 INFO:teuthology.orchestra.run.smithi177.stdout: "command_active": 0, 2024-03-07T07:43:47.609 INFO:teuthology.orchestra.run.smithi177.stdout: "command_send": 0, 2024-03-07T07:43:47.609 INFO:teuthology.orchestra.run.smithi177.stdout: "command_resend": 0, 2024-03-07T07:43:47.609 INFO:teuthology.orchestra.run.smithi177.stdout: "map_epoch": 87, 2024-03-07T07:43:47.609 INFO:teuthology.orchestra.run.smithi177.stdout: "map_full": 0, 2024-03-07T07:43:47.609 INFO:teuthology.orchestra.run.smithi177.stdout: "map_inc": 2, 2024-03-07T07:43:47.610 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_sessions": 9, 2024-03-07T07:43:47.610 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_session_open": 9, 2024-03-07T07:43:47.610 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_session_close": 0, 2024-03-07T07:43:47.610 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_laggy": 0, 2024-03-07T07:43:47.610 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_wr": 14, 2024-03-07T07:43:47.610 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_rd": 0, 2024-03-07T07:43:47.610 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_del": 1 2024-03-07T07:43:47.610 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.610 INFO:teuthology.orchestra.run.smithi177.stdout: "oft": { 2024-03-07T07:43:47.610 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_objs": 1, 2024-03-07T07:43:47.610 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_kv_pairs": 0, 2024-03-07T07:43:47.611 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_updates": 0, 2024-03-07T07:43:47.611 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_removes": 0 2024-03-07T07:43:47.611 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.611 INFO:teuthology.orchestra.run.smithi177.stdout: "osd-slow-ops": { 2024-03-07T07:43:47.611 INFO:teuthology.orchestra.run.smithi177.stdout: "slow_ops_count": 0 2024-03-07T07:43:47.611 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.611 INFO:teuthology.orchestra.run.smithi177.stdout: "purge_queue": { 2024-03-07T07:43:47.611 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing_ops": 0, 2024-03-07T07:43:47.611 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing_ops_high_water": 0, 2024-03-07T07:43:47.611 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing": 0, 2024-03-07T07:43:47.611 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing_high_water": 0, 2024-03-07T07:43:47.611 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executed_ops": 0, 2024-03-07T07:43:47.611 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executed": 0, 2024-03-07T07:43:47.611 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_item_in_journal": 0 2024-03-07T07:43:47.612 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.612 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-07T07:43:47.612 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-03-07T07:43:47.612 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2024-03-07T07:43:47.612 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-03-07T07:43:47.612 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 371, 2024-03-07T07:43:47.612 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 775673, 2024-03-07T07:43:47.612 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-03-07T07:43:47.612 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 371, 2024-03-07T07:43:47.612 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2024-03-07T07:43:47.612 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2024-03-07T07:43:47.612 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 371, 2024-03-07T07:43:47.612 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 775673, 2024-03-07T07:43:47.612 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-03-07T07:43:47.612 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:47.613 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:47.613 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:47.613 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-07T07:43:47.613 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.613 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-objecter_bytes": { 2024-03-07T07:43:47.613 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-03-07T07:43:47.613 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2024-03-07T07:43:47.613 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-03-07T07:43:47.613 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2024-03-07T07:43:47.613 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2024-03-07T07:43:47.613 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-03-07T07:43:47.613 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2024-03-07T07:43:47.613 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 37, 2024-03-07T07:43:47.613 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 11108, 2024-03-07T07:43:47.613 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 37, 2024-03-07T07:43:47.614 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 11108, 2024-03-07T07:43:47.614 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-03-07T07:43:47.614 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:47.614 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:47.614 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:47.614 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-07T07:43:47.614 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.614 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-objecter_ops": { 2024-03-07T07:43:47.614 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-03-07T07:43:47.614 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 1024, 2024-03-07T07:43:47.614 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-03-07T07:43:47.614 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2024-03-07T07:43:47.614 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2024-03-07T07:43:47.614 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-03-07T07:43:47.614 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2024-03-07T07:43:47.614 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 37, 2024-03-07T07:43:47.615 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 37, 2024-03-07T07:43:47.615 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 37, 2024-03-07T07:43:47.615 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 37, 2024-03-07T07:43:47.615 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-03-07T07:43:47.615 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:47.615 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:47.615 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:47.615 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-07T07:43:47.615 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.615 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-write_buf_throttle": { 2024-03-07T07:43:47.615 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-03-07T07:43:47.615 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 3758096384, 2024-03-07T07:43:47.615 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-03-07T07:43:47.615 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2024-03-07T07:43:47.615 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2024-03-07T07:43:47.616 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-03-07T07:43:47.616 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2024-03-07T07:43:47.616 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2024-03-07T07:43:47.616 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2024-03-07T07:43:47.616 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 0, 2024-03-07T07:43:47.616 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 0, 2024-03-07T07:43:47.616 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-03-07T07:43:47.616 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:47.616 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:47.616 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:47.616 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-07T07:43:47.616 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:47.616 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-write_buf_throttle-0x55dc43da06e0": { 2024-03-07T07:43:47.616 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-03-07T07:43:47.616 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 3758096384, 2024-03-07T07:43:47.617 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-03-07T07:43:47.617 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 3, 2024-03-07T07:43:47.617 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 896, 2024-03-07T07:43:47.617 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-03-07T07:43:47.617 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 3, 2024-03-07T07:43:47.617 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2024-03-07T07:43:47.617 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2024-03-07T07:43:47.617 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 2, 2024-03-07T07:43:47.617 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 896, 2024-03-07T07:43:47.617 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-03-07T07:43:47.617 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:47.617 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:47.617 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:47.617 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-07T07:43:47.618 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-07T07:43:47.618 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-03-07T07:43:47.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:43:47 smithi160 ceph-mon[31963]: pgmap v426: 97 pgs: 97 active+clean; 237 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 20 MiB/s wr, 15 op/s 2024-03-07T07:43:47.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:43:47 smithi178 ceph-mon[36650]: pgmap v426: 97 pgs: 97 active+clean; 237 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 20 MiB/s wr, 15 op/s 2024-03-07T07:43:48.362 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-03-07T07:43:48.362 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-03-07T07:43:48.362 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=0 2024-03-07T07:43:48.363 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=9 2024-03-07T07:43:48.363 DEBUG:tasks.check_counter:Getting stats from e 2024-03-07T07:43:48.363 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph daemon mds.e perf dump 2024-03-07T07:43:48.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:43:48 smithi177 ceph-mon[36578]: pgmap v427: 97 pgs: 97 active+clean; 153 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 21 MiB/s wr, 14 op/s 2024-03-07T07:43:48.666 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.b/config 2024-03-07T07:43:48.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:43:48 smithi160 ceph-mon[31963]: pgmap v427: 97 pgs: 97 active+clean; 153 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 21 MiB/s wr, 14 op/s 2024-03-07T07:43:48.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:43:48 smithi178 ceph-mon[36650]: pgmap v427: 97 pgs: 97 active+clean; 153 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 21 MiB/s wr, 14 op/s 2024-03-07T07:43:50.418 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-03-07T07:43:50.418 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-0": { 2024-03-07T07:43:50.419 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 90, 2024-03-07T07:43:50.419 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 85, 2024-03-07T07:43:50.419 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 711184, 2024-03-07T07:43:50.419 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 49680, 2024-03-07T07:43:50.419 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 4, 2024-03-07T07:43:50.419 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 0, 2024-03-07T07:43:50.419 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.014895829, 2024-03-07T07:43:50.419 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.003359622, 2024-03-07T07:43:50.419 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.014057200, 2024-03-07T07:43:50.419 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.001351083, 2024-03-07T07:43:50.419 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2024-03-07T07:43:50.419 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 84, 2024-03-07T07:43:50.419 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.003425468, 2024-03-07T07:43:50.419 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000040779 2024-03-07T07:43:50.420 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:50.420 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2024-03-07T07:43:50.420 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:50.420 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:50.420 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:50.420 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:50.420 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 711184, 2024-03-07T07:43:50.420 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 49680 2024-03-07T07:43:50.420 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:50.420 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-1": { 2024-03-07T07:43:50.420 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 1, 2024-03-07T07:43:50.420 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 64, 2024-03-07T07:43:50.420 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 160, 2024-03-07T07:43:50.421 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 88800, 2024-03-07T07:43:50.421 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 3, 2024-03-07T07:43:50.421 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 2, 2024-03-07T07:43:50.421 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.003082000, 2024-03-07T07:43:50.421 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.001768847, 2024-03-07T07:43:50.421 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.000781362, 2024-03-07T07:43:50.421 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-07T07:43:50.421 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2024-03-07T07:43:50.421 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 62, 2024-03-07T07:43:50.421 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.002902894, 2024-03-07T07:43:50.421 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000046820 2024-03-07T07:43:50.421 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:50.421 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2024-03-07T07:43:50.421 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:50.421 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:50.422 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:50.422 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:50.422 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-07T07:43:50.422 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 88800 2024-03-07T07:43:50.422 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:50.422 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-2": { 2024-03-07T07:43:50.422 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 4, 2024-03-07T07:43:50.422 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 4, 2024-03-07T07:43:50.422 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 3072, 2024-03-07T07:43:50.422 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 448, 2024-03-07T07:43:50.422 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 2, 2024-03-07T07:43:50.422 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 1, 2024-03-07T07:43:50.422 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.000747866, 2024-03-07T07:43:50.422 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.000045730, 2024-03-07T07:43:50.422 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.000694164, 2024-03-07T07:43:50.423 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-07T07:43:50.423 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2024-03-07T07:43:50.423 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 3, 2024-03-07T07:43:50.423 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000687948, 2024-03-07T07:43:50.423 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000229316 2024-03-07T07:43:50.423 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:50.423 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2024-03-07T07:43:50.423 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:50.423 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:50.423 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:50.423 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:50.423 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-03-07T07:43:50.423 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 448 2024-03-07T07:43:50.423 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:50.423 INFO:teuthology.orchestra.run.smithi177.stdout: "cct": { 2024-03-07T07:43:50.424 INFO:teuthology.orchestra.run.smithi177.stdout: "total_workers": 0, 2024-03-07T07:43:50.424 INFO:teuthology.orchestra.run.smithi177.stdout: "unhealthy_workers": 0 2024-03-07T07:43:50.424 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:50.424 INFO:teuthology.orchestra.run.smithi177.stdout: "mempool": { 2024-03-07T07:43:50.424 INFO:teuthology.orchestra.run.smithi177.stdout: "bloom_filter_bytes": 0, 2024-03-07T07:43:50.424 INFO:teuthology.orchestra.run.smithi177.stdout: "bloom_filter_items": 0, 2024-03-07T07:43:50.424 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_alloc_bytes": 0, 2024-03-07T07:43:50.424 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_alloc_items": 0, 2024-03-07T07:43:50.424 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_data_bytes": 0, 2024-03-07T07:43:50.424 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_data_items": 0, 2024-03-07T07:43:50.424 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-07T07:43:50.424 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_onode_items": 0, 2024-03-07T07:43:50.424 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-07T07:43:50.424 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_meta_items": 0, 2024-03-07T07:43:50.425 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_other_bytes": 0, 2024-03-07T07:43:50.425 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_other_items": 0, 2024-03-07T07:43:50.425 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-07T07:43:50.425 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_buffer_items": 0, 2024-03-07T07:43:50.425 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_extent_bytes": 0, 2024-03-07T07:43:50.425 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_extent_items": 0, 2024-03-07T07:43:50.425 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_blob_bytes": 0, 2024-03-07T07:43:50.425 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_blob_items": 0, 2024-03-07T07:43:50.425 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-07T07:43:50.425 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_shared_blob_items": 0, 2024-03-07T07:43:50.425 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-07T07:43:50.425 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_inline_bl_items": 0, 2024-03-07T07:43:50.425 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_bytes": 0, 2024-03-07T07:43:50.425 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_items": 0, 2024-03-07T07:43:50.425 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_txc_bytes": 0, 2024-03-07T07:43:50.425 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_txc_items": 0, 2024-03-07T07:43:50.426 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-07T07:43:50.426 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_deferred_items": 0, 2024-03-07T07:43:50.426 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_bytes": 0, 2024-03-07T07:43:50.426 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_items": 0, 2024-03-07T07:43:50.426 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_bytes": 0, 2024-03-07T07:43:50.426 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_items": 0, 2024-03-07T07:43:50.426 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_reader_bytes": 0, 2024-03-07T07:43:50.426 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_reader_items": 0, 2024-03-07T07:43:50.426 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_writer_bytes": 0, 2024-03-07T07:43:50.426 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_writer_items": 0, 2024-03-07T07:43:50.426 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_anon_bytes": 148903, 2024-03-07T07:43:50.426 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_anon_items": 29, 2024-03-07T07:43:50.426 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_meta_bytes": 0, 2024-03-07T07:43:50.426 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_meta_items": 0, 2024-03-07T07:43:50.426 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_bytes": 0, 2024-03-07T07:43:50.427 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_items": 0, 2024-03-07T07:43:50.427 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mapbl_bytes": 0, 2024-03-07T07:43:50.427 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mapbl_items": 0, 2024-03-07T07:43:50.427 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pglog_bytes": 0, 2024-03-07T07:43:50.427 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pglog_items": 0, 2024-03-07T07:43:50.427 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_bytes": 0, 2024-03-07T07:43:50.427 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_items": 0, 2024-03-07T07:43:50.427 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_mapping_bytes": 0, 2024-03-07T07:43:50.427 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_mapping_items": 0, 2024-03-07T07:43:50.427 INFO:teuthology.orchestra.run.smithi177.stdout: "pgmap_bytes": 0, 2024-03-07T07:43:50.427 INFO:teuthology.orchestra.run.smithi177.stdout: "pgmap_items": 0, 2024-03-07T07:43:50.427 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_co_bytes": 928, 2024-03-07T07:43:50.427 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_co_items": 2, 2024-03-07T07:43:50.427 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_1_bytes": 0, 2024-03-07T07:43:50.427 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_1_items": 0, 2024-03-07T07:43:50.428 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_2_bytes": 0, 2024-03-07T07:43:50.428 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_2_items": 0 2024-03-07T07:43:50.428 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:50.428 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-07T07:43:50.428 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-03-07T07:43:50.428 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2024-03-07T07:43:50.428 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-03-07T07:43:50.428 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 91, 2024-03-07T07:43:50.428 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 698498, 2024-03-07T07:43:50.428 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-03-07T07:43:50.428 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 91, 2024-03-07T07:43:50.428 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2024-03-07T07:43:50.428 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2024-03-07T07:43:50.428 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 91, 2024-03-07T07:43:50.428 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 698498, 2024-03-07T07:43:50.429 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-03-07T07:43:50.429 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:50.429 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:50.429 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:50.429 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-07T07:43:50.429 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-07T07:43:50.429 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-03-07T07:43:51.255 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.e 2024-03-07T07:43:51.255 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.e 2024-03-07T07:43:51.256 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.e 2024-03-07T07:43:51.256 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.e 2024-03-07T07:43:51.256 DEBUG:tasks.check_counter:Getting stats from h 2024-03-07T07:43:51.256 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph daemon mds.h perf dump 2024-03-07T07:43:51.508 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.b/config 2024-03-07T07:43:51.656 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:43:51 smithi177 ceph-mon[36578]: pgmap v428: 97 pgs: 97 active+clean; 1.2 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 19 MiB/s wr, 19 op/s 2024-03-07T07:43:51.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:43:51 smithi160 ceph-mon[31963]: pgmap v428: 97 pgs: 97 active+clean; 1.2 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 19 MiB/s wr, 19 op/s 2024-03-07T07:43:51.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:43:51 smithi178 ceph-mon[36650]: pgmap v428: 97 pgs: 97 active+clean; 1.2 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 19 MiB/s wr, 19 op/s 2024-03-07T07:43:53.233 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-03-07T07:43:53.233 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-0": { 2024-03-07T07:43:53.233 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 376, 2024-03-07T07:43:53.233 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 368, 2024-03-07T07:43:53.233 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 824368, 2024-03-07T07:43:53.233 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 138138, 2024-03-07T07:43:53.233 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 5, 2024-03-07T07:43:53.234 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 1, 2024-03-07T07:43:53.234 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.035197905, 2024-03-07T07:43:53.234 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.010351356, 2024-03-07T07:43:53.234 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.019712500, 2024-03-07T07:43:53.234 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.007586785, 2024-03-07T07:43:53.234 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2024-03-07T07:43:53.234 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 368, 2024-03-07T07:43:53.234 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.013592422, 2024-03-07T07:43:53.234 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000036935 2024-03-07T07:43:53.234 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.235 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2024-03-07T07:43:53.235 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:53.235 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:53.235 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:53.235 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.235 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 731968, 2024-03-07T07:43:53.235 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 50688 2024-03-07T07:43:53.235 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.235 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-1": { 2024-03-07T07:43:53.235 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 3, 2024-03-07T07:43:53.235 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 65, 2024-03-07T07:43:53.235 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 716, 2024-03-07T07:43:53.235 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 186426, 2024-03-07T07:43:53.235 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 4, 2024-03-07T07:43:53.236 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 3, 2024-03-07T07:43:53.236 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.003500259, 2024-03-07T07:43:53.236 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.001777521, 2024-03-07T07:43:53.236 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.001217810, 2024-03-07T07:43:53.236 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.000057678, 2024-03-07T07:43:53.236 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2024-03-07T07:43:53.236 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 63, 2024-03-07T07:43:53.236 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.002978328, 2024-03-07T07:43:53.236 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000047275 2024-03-07T07:43:53.236 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.236 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2024-03-07T07:43:53.236 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:53.236 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:53.237 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:53.237 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.237 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-07T07:43:53.237 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 185728 2024-03-07T07:43:53.237 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.237 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-2": { 2024-03-07T07:43:53.237 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 444, 2024-03-07T07:43:53.237 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 446, 2024-03-07T07:43:53.237 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 115165, 2024-03-07T07:43:53.237 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 141199, 2024-03-07T07:43:53.237 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 4, 2024-03-07T07:43:53.237 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 3, 2024-03-07T07:43:53.237 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.020380308, 2024-03-07T07:43:53.238 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.005334868, 2024-03-07T07:43:53.238 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.005865858, 2024-03-07T07:43:53.238 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.007677665, 2024-03-07T07:43:53.238 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2024-03-07T07:43:53.238 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 444, 2024-03-07T07:43:53.238 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.008368745, 2024-03-07T07:43:53.238 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000018848 2024-03-07T07:43:53.238 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.238 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2024-03-07T07:43:53.238 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:53.238 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:53.238 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:53.239 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.239 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-07T07:43:53.239 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 0 2024-03-07T07:43:53.239 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.239 INFO:teuthology.orchestra.run.smithi177.stdout: "cct": { 2024-03-07T07:43:53.239 INFO:teuthology.orchestra.run.smithi177.stdout: "total_workers": 1, 2024-03-07T07:43:53.239 INFO:teuthology.orchestra.run.smithi177.stdout: "unhealthy_workers": 0 2024-03-07T07:43:53.239 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.239 INFO:teuthology.orchestra.run.smithi177.stdout: "finisher-MDSRank": { 2024-03-07T07:43:53.239 INFO:teuthology.orchestra.run.smithi177.stdout: "queue_len": 0, 2024-03-07T07:43:53.239 INFO:teuthology.orchestra.run.smithi177.stdout: "complete_latency": { 2024-03-07T07:43:53.239 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 1269, 2024-03-07T07:43:53.239 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.023745980, 2024-03-07T07:43:53.240 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000018712 2024-03-07T07:43:53.240 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-07T07:43:53.240 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.240 INFO:teuthology.orchestra.run.smithi177.stdout: "finisher-PurgeQueue": { 2024-03-07T07:43:53.240 INFO:teuthology.orchestra.run.smithi177.stdout: "queue_len": 0, 2024-03-07T07:43:53.240 INFO:teuthology.orchestra.run.smithi177.stdout: "complete_latency": { 2024-03-07T07:43:53.240 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:53.240 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:53.240 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:53.240 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-07T07:43:53.240 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.240 INFO:teuthology.orchestra.run.smithi177.stdout: "mds": { 2024-03-07T07:43:53.240 INFO:teuthology.orchestra.run.smithi177.stdout: "request": 0, 2024-03-07T07:43:53.240 INFO:teuthology.orchestra.run.smithi177.stdout: "reply": 0, 2024-03-07T07:43:53.241 INFO:teuthology.orchestra.run.smithi177.stdout: "reply_latency": { 2024-03-07T07:43:53.241 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:53.241 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:53.241 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:53.241 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.241 INFO:teuthology.orchestra.run.smithi177.stdout: "slow_reply": 0, 2024-03-07T07:43:53.241 INFO:teuthology.orchestra.run.smithi177.stdout: "forward": 0, 2024-03-07T07:43:53.241 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_fetch_complete": 0, 2024-03-07T07:43:53.241 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_fetch_keys": 0, 2024-03-07T07:43:53.241 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_commit": 0, 2024-03-07T07:43:53.241 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_split": 0, 2024-03-07T07:43:53.241 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_merge": 0, 2024-03-07T07:43:53.241 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes": 0, 2024-03-07T07:43:53.242 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_top": 0, 2024-03-07T07:43:53.242 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_bottom": 0, 2024-03-07T07:43:53.242 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_pin_tail": 0, 2024-03-07T07:43:53.242 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_pinned": 0, 2024-03-07T07:43:53.242 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_expired": 0, 2024-03-07T07:43:53.242 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_with_caps": 0, 2024-03-07T07:43:53.242 INFO:teuthology.orchestra.run.smithi177.stdout: "caps": 0, 2024-03-07T07:43:53.242 INFO:teuthology.orchestra.run.smithi177.stdout: "subtrees": 1, 2024-03-07T07:43:53.242 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse": 0, 2024-03-07T07:43:53.242 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_hit": 0, 2024-03-07T07:43:53.242 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_forward": 0, 2024-03-07T07:43:53.242 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_discover": 0, 2024-03-07T07:43:53.242 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_dir_fetch": 0, 2024-03-07T07:43:53.243 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_remote_ino": 0, 2024-03-07T07:43:53.243 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_lock": 0, 2024-03-07T07:43:53.243 INFO:teuthology.orchestra.run.smithi177.stdout: "load_cent": 0, 2024-03-07T07:43:53.243 INFO:teuthology.orchestra.run.smithi177.stdout: "q": 0, 2024-03-07T07:43:53.243 INFO:teuthology.orchestra.run.smithi177.stdout: "exported": 0, 2024-03-07T07:43:53.243 INFO:teuthology.orchestra.run.smithi177.stdout: "exported_inodes": 0, 2024-03-07T07:43:53.243 INFO:teuthology.orchestra.run.smithi177.stdout: "imported": 0, 2024-03-07T07:43:53.243 INFO:teuthology.orchestra.run.smithi177.stdout: "imported_inodes": 0, 2024-03-07T07:43:53.243 INFO:teuthology.orchestra.run.smithi177.stdout: "openino_dir_fetch": 0, 2024-03-07T07:43:53.243 INFO:teuthology.orchestra.run.smithi177.stdout: "openino_backtrace_fetch": 0, 2024-03-07T07:43:53.243 INFO:teuthology.orchestra.run.smithi177.stdout: "openino_peer_discover": 0, 2024-03-07T07:43:53.243 INFO:teuthology.orchestra.run.smithi177.stdout: "root_rfiles": 0, 2024-03-07T07:43:53.243 INFO:teuthology.orchestra.run.smithi177.stdout: "root_rbytes": 0, 2024-03-07T07:43:53.243 INFO:teuthology.orchestra.run.smithi177.stdout: "root_rsnaps": 0, 2024-03-07T07:43:53.244 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_backtrace_fetch": 0, 2024-03-07T07:43:53.244 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_set_tag": 0, 2024-03-07T07:43:53.244 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_backtrace_repaired": 0, 2024-03-07T07:43:53.244 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_inotable_repaired": 0, 2024-03-07T07:43:53.244 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_dir_inodes": 0, 2024-03-07T07:43:53.244 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_dir_base_inodes": 0, 2024-03-07T07:43:53.244 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_dirfrag_rstats": 0, 2024-03-07T07:43:53.244 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_file_inodes": 0, 2024-03-07T07:43:53.244 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_inode_file_caps": 0, 2024-03-07T07:43:53.244 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_revoke": 0, 2024-03-07T07:43:53.244 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_grant": 0, 2024-03-07T07:43:53.244 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_trunc": 0, 2024-03-07T07:43:53.244 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-07T07:43:53.244 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-07T07:43:53.245 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_caps": 0, 2024-03-07T07:43:53.245 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_caps_dirty": 0, 2024-03-07T07:43:53.245 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_cap_release": 0, 2024-03-07T07:43:53.245 INFO:teuthology.orchestra.run.smithi177.stdout: "process_request_cap_release": 0 2024-03-07T07:43:53.245 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.245 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_cache": { 2024-03-07T07:43:53.245 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_update": 0, 2024-03-07T07:43:53.245 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_update_receipt": 0, 2024-03-07T07:43:53.245 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_try_discover": 0, 2024-03-07T07:43:53.245 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_send_discover": 0, 2024-03-07T07:43:53.245 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_handle_discover": 0, 2024-03-07T07:43:53.245 INFO:teuthology.orchestra.run.smithi177.stdout: "num_strays": 0, 2024-03-07T07:43:53.245 INFO:teuthology.orchestra.run.smithi177.stdout: "num_strays_delayed": 0, 2024-03-07T07:43:53.246 INFO:teuthology.orchestra.run.smithi177.stdout: "num_strays_enqueuing": 0, 2024-03-07T07:43:53.246 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_created": 0, 2024-03-07T07:43:53.246 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_enqueued": 0, 2024-03-07T07:43:53.246 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_reintegrated": 0, 2024-03-07T07:43:53.246 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_migrated": 0, 2024-03-07T07:43:53.246 INFO:teuthology.orchestra.run.smithi177.stdout: "num_recovering_processing": 0, 2024-03-07T07:43:53.246 INFO:teuthology.orchestra.run.smithi177.stdout: "num_recovering_enqueued": 0, 2024-03-07T07:43:53.246 INFO:teuthology.orchestra.run.smithi177.stdout: "num_recovering_prioritized": 0, 2024-03-07T07:43:53.246 INFO:teuthology.orchestra.run.smithi177.stdout: "recovery_started": 0, 2024-03-07T07:43:53.246 INFO:teuthology.orchestra.run.smithi177.stdout: "recovery_completed": 0, 2024-03-07T07:43:53.246 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_enqueue_scrub": 0, 2024-03-07T07:43:53.246 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_exportdir": 0, 2024-03-07T07:43:53.246 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_flush": 0, 2024-03-07T07:43:53.247 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_fragmentdir": 0, 2024-03-07T07:43:53.247 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_fragstats": 0, 2024-03-07T07:43:53.247 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_inodestats": 0 2024-03-07T07:43:53.247 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.247 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_log": { 2024-03-07T07:43:53.247 INFO:teuthology.orchestra.run.smithi177.stdout: "evlrg": 0, 2024-03-07T07:43:53.247 INFO:teuthology.orchestra.run.smithi177.stdout: "evadd": 0, 2024-03-07T07:43:53.247 INFO:teuthology.orchestra.run.smithi177.stdout: "evex": 0, 2024-03-07T07:43:53.247 INFO:teuthology.orchestra.run.smithi177.stdout: "evtrm": 0, 2024-03-07T07:43:53.247 INFO:teuthology.orchestra.run.smithi177.stdout: "ev": 3, 2024-03-07T07:43:53.247 INFO:teuthology.orchestra.run.smithi177.stdout: "evexg": 0, 2024-03-07T07:43:53.247 INFO:teuthology.orchestra.run.smithi177.stdout: "evexd": 0, 2024-03-07T07:43:53.247 INFO:teuthology.orchestra.run.smithi177.stdout: "segadd": 0, 2024-03-07T07:43:53.248 INFO:teuthology.orchestra.run.smithi177.stdout: "segex": 0, 2024-03-07T07:43:53.248 INFO:teuthology.orchestra.run.smithi177.stdout: "segtrm": 0, 2024-03-07T07:43:53.248 INFO:teuthology.orchestra.run.smithi177.stdout: "seg": 2, 2024-03-07T07:43:53.248 INFO:teuthology.orchestra.run.smithi177.stdout: "segmjr": 2, 2024-03-07T07:43:53.248 INFO:teuthology.orchestra.run.smithi177.stdout: "segexg": 0, 2024-03-07T07:43:53.248 INFO:teuthology.orchestra.run.smithi177.stdout: "segexd": 0, 2024-03-07T07:43:53.248 INFO:teuthology.orchestra.run.smithi177.stdout: "expos": 4194304, 2024-03-07T07:43:53.248 INFO:teuthology.orchestra.run.smithi177.stdout: "wrpos": 4195200, 2024-03-07T07:43:53.248 INFO:teuthology.orchestra.run.smithi177.stdout: "rdpos": 4194892, 2024-03-07T07:43:53.248 INFO:teuthology.orchestra.run.smithi177.stdout: "jlat": { 2024-03-07T07:43:53.248 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:53.248 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:53.248 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:53.249 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.249 INFO:teuthology.orchestra.run.smithi177.stdout: "replayed": 3 2024-03-07T07:43:53.249 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.249 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_mem": { 2024-03-07T07:43:53.249 INFO:teuthology.orchestra.run.smithi177.stdout: "ino": 3, 2024-03-07T07:43:53.249 INFO:teuthology.orchestra.run.smithi177.stdout: "ino+": 3, 2024-03-07T07:43:53.249 INFO:teuthology.orchestra.run.smithi177.stdout: "ino-": 0, 2024-03-07T07:43:53.249 INFO:teuthology.orchestra.run.smithi177.stdout: "dir": 1, 2024-03-07T07:43:53.249 INFO:teuthology.orchestra.run.smithi177.stdout: "dir+": 1, 2024-03-07T07:43:53.249 INFO:teuthology.orchestra.run.smithi177.stdout: "dir-": 0, 2024-03-07T07:43:53.249 INFO:teuthology.orchestra.run.smithi177.stdout: "dn": 0, 2024-03-07T07:43:53.249 INFO:teuthology.orchestra.run.smithi177.stdout: "dn+": 0, 2024-03-07T07:43:53.249 INFO:teuthology.orchestra.run.smithi177.stdout: "dn-": 0, 2024-03-07T07:43:53.249 INFO:teuthology.orchestra.run.smithi177.stdout: "cap": 0, 2024-03-07T07:43:53.250 INFO:teuthology.orchestra.run.smithi177.stdout: "cap+": 0, 2024-03-07T07:43:53.250 INFO:teuthology.orchestra.run.smithi177.stdout: "cap-": 0, 2024-03-07T07:43:53.250 INFO:teuthology.orchestra.run.smithi177.stdout: "rss": 47740, 2024-03-07T07:43:53.250 INFO:teuthology.orchestra.run.smithi177.stdout: "heap": 215420 2024-03-07T07:43:53.250 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.250 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_server": { 2024-03-07T07:43:53.250 INFO:teuthology.orchestra.run.smithi177.stdout: "dispatch_client_request": 0, 2024-03-07T07:43:53.250 INFO:teuthology.orchestra.run.smithi177.stdout: "dispatch_server_request": 0, 2024-03-07T07:43:53.250 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_request": 0, 2024-03-07T07:43:53.250 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_session": 0, 2024-03-07T07:43:53.250 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_peer_request": 0, 2024-03-07T07:43:53.250 INFO:teuthology.orchestra.run.smithi177.stdout: "req_create_latency": { 2024-03-07T07:43:53.250 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:53.250 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:53.251 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:53.251 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.251 INFO:teuthology.orchestra.run.smithi177.stdout: "req_getattr_latency": { 2024-03-07T07:43:53.251 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:53.251 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:53.251 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:53.251 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.251 INFO:teuthology.orchestra.run.smithi177.stdout: "req_getfilelock_latency": { 2024-03-07T07:43:53.251 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:53.251 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:53.251 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:53.251 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.251 INFO:teuthology.orchestra.run.smithi177.stdout: "req_link_latency": { 2024-03-07T07:43:53.251 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:53.252 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:53.252 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:53.252 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.252 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookup_latency": { 2024-03-07T07:43:53.252 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:53.252 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:53.252 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:53.252 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.252 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookuphash_latency": { 2024-03-07T07:43:53.252 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:53.252 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:53.252 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:53.252 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.253 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupino_latency": { 2024-03-07T07:43:53.253 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:53.253 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:53.253 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:53.253 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.253 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupname_latency": { 2024-03-07T07:43:53.253 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:53.253 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:53.253 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:53.253 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.254 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupparent_latency": { 2024-03-07T07:43:53.254 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:53.254 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:53.254 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:53.254 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.254 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupsnap_latency": { 2024-03-07T07:43:53.254 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:53.254 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:53.254 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:53.254 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.254 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lssnap_latency": { 2024-03-07T07:43:53.255 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:53.255 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:53.255 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:53.255 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.255 INFO:teuthology.orchestra.run.smithi177.stdout: "req_mkdir_latency": { 2024-03-07T07:43:53.255 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:53.255 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:53.255 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:53.255 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.256 INFO:teuthology.orchestra.run.smithi177.stdout: "req_mknod_latency": { 2024-03-07T07:43:53.256 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:53.256 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:53.256 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:53.256 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.256 INFO:teuthology.orchestra.run.smithi177.stdout: "req_mksnap_latency": { 2024-03-07T07:43:53.256 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:53.256 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:53.256 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:53.257 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.257 INFO:teuthology.orchestra.run.smithi177.stdout: "req_open_latency": { 2024-03-07T07:43:53.257 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:53.257 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:53.257 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:53.257 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.257 INFO:teuthology.orchestra.run.smithi177.stdout: "req_readdir_latency": { 2024-03-07T07:43:53.257 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:53.257 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:53.257 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:53.257 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.257 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rename_latency": { 2024-03-07T07:43:53.257 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:53.257 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:53.258 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:53.258 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.258 INFO:teuthology.orchestra.run.smithi177.stdout: "req_renamesnap_latency": { 2024-03-07T07:43:53.258 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:53.258 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:53.258 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:53.258 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.258 INFO:teuthology.orchestra.run.smithi177.stdout: "req_snapdiff_latency": { 2024-03-07T07:43:53.258 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:53.258 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:53.258 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:53.258 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.258 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rmdir_latency": { 2024-03-07T07:43:53.258 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:53.259 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:53.259 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:53.259 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.259 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rmsnap_latency": { 2024-03-07T07:43:53.259 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:53.259 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:53.259 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:53.259 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.259 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rmxattr_latency": { 2024-03-07T07:43:53.259 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:53.259 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:53.259 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:53.259 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.259 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setattr_latency": { 2024-03-07T07:43:53.260 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:53.260 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:53.260 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:53.260 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.260 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setdirlayout_latency": { 2024-03-07T07:43:53.260 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:53.260 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:53.260 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:53.260 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.260 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setfilelock_latency": { 2024-03-07T07:43:53.260 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:53.260 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:53.260 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:53.260 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.261 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setlayout_latency": { 2024-03-07T07:43:53.261 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:53.261 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:53.261 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:53.261 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.261 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setxattr_latency": { 2024-03-07T07:43:53.261 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:53.261 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:53.261 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:53.261 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.261 INFO:teuthology.orchestra.run.smithi177.stdout: "req_symlink_latency": { 2024-03-07T07:43:53.261 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:53.261 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:53.261 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:53.262 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.262 INFO:teuthology.orchestra.run.smithi177.stdout: "req_unlink_latency": { 2024-03-07T07:43:53.262 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:53.262 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:53.262 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:53.262 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.262 INFO:teuthology.orchestra.run.smithi177.stdout: "cap_revoke_eviction": 0, 2024-03-07T07:43:53.262 INFO:teuthology.orchestra.run.smithi177.stdout: "cap_acquisition_throttle": 0, 2024-03-07T07:43:53.262 INFO:teuthology.orchestra.run.smithi177.stdout: "req_getvxattr_latency": { 2024-03-07T07:43:53.262 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:53.262 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:53.262 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:53.262 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-07T07:43:53.262 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.263 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_sessions": { 2024-03-07T07:43:53.263 INFO:teuthology.orchestra.run.smithi177.stdout: "session_count": 1, 2024-03-07T07:43:53.263 INFO:teuthology.orchestra.run.smithi177.stdout: "session_add": 1, 2024-03-07T07:43:53.263 INFO:teuthology.orchestra.run.smithi177.stdout: "session_remove": 0, 2024-03-07T07:43:53.263 INFO:teuthology.orchestra.run.smithi177.stdout: "sessions_open": 1, 2024-03-07T07:43:53.263 INFO:teuthology.orchestra.run.smithi177.stdout: "sessions_stale": 0, 2024-03-07T07:43:53.263 INFO:teuthology.orchestra.run.smithi177.stdout: "total_load": 0, 2024-03-07T07:43:53.263 INFO:teuthology.orchestra.run.smithi177.stdout: "average_load": 0, 2024-03-07T07:43:53.263 INFO:teuthology.orchestra.run.smithi177.stdout: "avg_session_uptime": 1584, 2024-03-07T07:43:53.263 INFO:teuthology.orchestra.run.smithi177.stdout: "mdthresh_evicted": 0 2024-03-07T07:43:53.263 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.263 INFO:teuthology.orchestra.run.smithi177.stdout: "mempool": { 2024-03-07T07:43:53.263 INFO:teuthology.orchestra.run.smithi177.stdout: "bloom_filter_bytes": 0, 2024-03-07T07:43:53.263 INFO:teuthology.orchestra.run.smithi177.stdout: "bloom_filter_items": 0, 2024-03-07T07:43:53.264 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_alloc_bytes": 0, 2024-03-07T07:43:53.264 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_alloc_items": 0, 2024-03-07T07:43:53.264 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_data_bytes": 0, 2024-03-07T07:43:53.264 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_data_items": 0, 2024-03-07T07:43:53.264 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-07T07:43:53.264 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_onode_items": 0, 2024-03-07T07:43:53.264 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-07T07:43:53.264 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_meta_items": 0, 2024-03-07T07:43:53.264 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_other_bytes": 0, 2024-03-07T07:43:53.264 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_other_items": 0, 2024-03-07T07:43:53.264 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-07T07:43:53.264 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_buffer_items": 0, 2024-03-07T07:43:53.264 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_extent_bytes": 0, 2024-03-07T07:43:53.264 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_extent_items": 0, 2024-03-07T07:43:53.265 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_blob_bytes": 0, 2024-03-07T07:43:53.265 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_blob_items": 0, 2024-03-07T07:43:53.265 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-07T07:43:53.265 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_shared_blob_items": 0, 2024-03-07T07:43:53.265 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-07T07:43:53.265 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_inline_bl_items": 0, 2024-03-07T07:43:53.265 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_bytes": 0, 2024-03-07T07:43:53.265 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_items": 0, 2024-03-07T07:43:53.265 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_txc_bytes": 0, 2024-03-07T07:43:53.265 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_txc_items": 0, 2024-03-07T07:43:53.265 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-07T07:43:53.265 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_deferred_items": 0, 2024-03-07T07:43:53.265 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_bytes": 0, 2024-03-07T07:43:53.265 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_items": 0, 2024-03-07T07:43:53.266 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_bytes": 0, 2024-03-07T07:43:53.266 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_items": 0, 2024-03-07T07:43:53.266 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_reader_bytes": 0, 2024-03-07T07:43:53.266 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_reader_items": 0, 2024-03-07T07:43:53.266 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_writer_bytes": 0, 2024-03-07T07:43:53.266 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_writer_items": 0, 2024-03-07T07:43:53.266 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_anon_bytes": 181199, 2024-03-07T07:43:53.266 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_anon_items": 45, 2024-03-07T07:43:53.266 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_meta_bytes": 0, 2024-03-07T07:43:53.266 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_meta_items": 0, 2024-03-07T07:43:53.266 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_bytes": 0, 2024-03-07T07:43:53.266 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_items": 0, 2024-03-07T07:43:53.266 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mapbl_bytes": 0, 2024-03-07T07:43:53.266 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mapbl_items": 0, 2024-03-07T07:43:53.267 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pglog_bytes": 0, 2024-03-07T07:43:53.267 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pglog_items": 0, 2024-03-07T07:43:53.267 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_bytes": 7624, 2024-03-07T07:43:53.267 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_items": 160, 2024-03-07T07:43:53.267 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_mapping_bytes": 0, 2024-03-07T07:43:53.267 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_mapping_items": 0, 2024-03-07T07:43:53.267 INFO:teuthology.orchestra.run.smithi177.stdout: "pgmap_bytes": 0, 2024-03-07T07:43:53.267 INFO:teuthology.orchestra.run.smithi177.stdout: "pgmap_items": 0, 2024-03-07T07:43:53.267 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_co_bytes": 8320, 2024-03-07T07:43:53.267 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_co_items": 13, 2024-03-07T07:43:53.267 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_1_bytes": 0, 2024-03-07T07:43:53.267 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_1_items": 0, 2024-03-07T07:43:53.267 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_2_bytes": 0, 2024-03-07T07:43:53.267 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_2_items": 0 2024-03-07T07:43:53.267 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.268 INFO:teuthology.orchestra.run.smithi177.stdout: "objecter": { 2024-03-07T07:43:53.268 INFO:teuthology.orchestra.run.smithi177.stdout: "op_active": 0, 2024-03-07T07:43:53.268 INFO:teuthology.orchestra.run.smithi177.stdout: "op_laggy": 0, 2024-03-07T07:43:53.268 INFO:teuthology.orchestra.run.smithi177.stdout: "op_send": 721, 2024-03-07T07:43:53.268 INFO:teuthology.orchestra.run.smithi177.stdout: "op_send_bytes": 21, 2024-03-07T07:43:53.268 INFO:teuthology.orchestra.run.smithi177.stdout: "op_resend": 3, 2024-03-07T07:43:53.268 INFO:teuthology.orchestra.run.smithi177.stdout: "op_reply": 721, 2024-03-07T07:43:53.268 INFO:teuthology.orchestra.run.smithi177.stdout: "op_latency": { 2024-03-07T07:43:53.268 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 721, 2024-03-07T07:43:53.268 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.284992229, 2024-03-07T07:43:53.268 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000395273 2024-03-07T07:43:53.268 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.269 INFO:teuthology.orchestra.run.smithi177.stdout: "op_inflight": 0, 2024-03-07T07:43:53.269 INFO:teuthology.orchestra.run.smithi177.stdout: "oplen_avg": { 2024-03-07T07:43:53.269 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 721, 2024-03-07T07:43:53.269 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 722 2024-03-07T07:43:53.269 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.269 INFO:teuthology.orchestra.run.smithi177.stdout: "op": 721, 2024-03-07T07:43:53.269 INFO:teuthology.orchestra.run.smithi177.stdout: "op_r": 721, 2024-03-07T07:43:53.269 INFO:teuthology.orchestra.run.smithi177.stdout: "op_w": 0, 2024-03-07T07:43:53.269 INFO:teuthology.orchestra.run.smithi177.stdout: "op_rmw": 0, 2024-03-07T07:43:53.269 INFO:teuthology.orchestra.run.smithi177.stdout: "op_pg": 0, 2024-03-07T07:43:53.269 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_stat": 439, 2024-03-07T07:43:53.269 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_create": 0, 2024-03-07T07:43:53.270 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_read": 280, 2024-03-07T07:43:53.270 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_write": 0, 2024-03-07T07:43:53.270 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_writefull": 0, 2024-03-07T07:43:53.270 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_writesame": 0, 2024-03-07T07:43:53.270 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_append": 0, 2024-03-07T07:43:53.270 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_zero": 0, 2024-03-07T07:43:53.270 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_truncate": 0, 2024-03-07T07:43:53.270 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_delete": 0, 2024-03-07T07:43:53.270 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_mapext": 0, 2024-03-07T07:43:53.270 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_sparse_read": 0, 2024-03-07T07:43:53.270 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_clonerange": 0, 2024-03-07T07:43:53.270 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_getxattr": 1, 2024-03-07T07:43:53.270 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_setxattr": 0, 2024-03-07T07:43:53.271 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_cmpxattr": 0, 2024-03-07T07:43:53.271 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_rmxattr": 0, 2024-03-07T07:43:53.271 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_resetxattrs": 0, 2024-03-07T07:43:53.271 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_call": 0, 2024-03-07T07:43:53.271 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_watch": 0, 2024-03-07T07:43:53.271 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_notify": 0, 2024-03-07T07:43:53.271 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_src_cmpxattr": 0, 2024-03-07T07:43:53.271 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_pgls": 0, 2024-03-07T07:43:53.271 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_pgls_filter": 0, 2024-03-07T07:43:53.271 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_other": 0, 2024-03-07T07:43:53.271 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_active": 0, 2024-03-07T07:43:53.271 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_send": 0, 2024-03-07T07:43:53.271 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_resend": 0, 2024-03-07T07:43:53.271 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_ping": 0, 2024-03-07T07:43:53.271 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_active": 0, 2024-03-07T07:43:53.272 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_send": 0, 2024-03-07T07:43:53.272 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_resend": 0, 2024-03-07T07:43:53.272 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_active": 0, 2024-03-07T07:43:53.272 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_send": 0, 2024-03-07T07:43:53.272 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_resend": 0, 2024-03-07T07:43:53.272 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_active": 0, 2024-03-07T07:43:53.272 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_send": 0, 2024-03-07T07:43:53.272 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_resend": 0, 2024-03-07T07:43:53.272 INFO:teuthology.orchestra.run.smithi177.stdout: "command_active": 0, 2024-03-07T07:43:53.272 INFO:teuthology.orchestra.run.smithi177.stdout: "command_send": 0, 2024-03-07T07:43:53.272 INFO:teuthology.orchestra.run.smithi177.stdout: "command_resend": 0, 2024-03-07T07:43:53.272 INFO:teuthology.orchestra.run.smithi177.stdout: "map_epoch": 87, 2024-03-07T07:43:53.272 INFO:teuthology.orchestra.run.smithi177.stdout: "map_full": 0, 2024-03-07T07:43:53.273 INFO:teuthology.orchestra.run.smithi177.stdout: "map_inc": 2, 2024-03-07T07:43:53.273 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_sessions": 4, 2024-03-07T07:43:53.273 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_session_open": 4, 2024-03-07T07:43:53.273 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_session_close": 0, 2024-03-07T07:43:53.273 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_laggy": 0, 2024-03-07T07:43:53.273 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_wr": 0, 2024-03-07T07:43:53.273 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_rd": 2, 2024-03-07T07:43:53.273 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_del": 0 2024-03-07T07:43:53.273 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.273 INFO:teuthology.orchestra.run.smithi177.stdout: "oft": { 2024-03-07T07:43:53.273 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_objs": 0, 2024-03-07T07:43:53.273 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_kv_pairs": 0, 2024-03-07T07:43:53.273 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_updates": 0, 2024-03-07T07:43:53.274 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_removes": 0 2024-03-07T07:43:53.274 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.274 INFO:teuthology.orchestra.run.smithi177.stdout: "osd-slow-ops": { 2024-03-07T07:43:53.274 INFO:teuthology.orchestra.run.smithi177.stdout: "slow_ops_count": 0 2024-03-07T07:43:53.274 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.274 INFO:teuthology.orchestra.run.smithi177.stdout: "purge_queue": { 2024-03-07T07:43:53.274 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing_ops": 0, 2024-03-07T07:43:53.274 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing_ops_high_water": 0, 2024-03-07T07:43:53.274 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing": 0, 2024-03-07T07:43:53.274 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing_high_water": 0, 2024-03-07T07:43:53.274 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executed_ops": 0, 2024-03-07T07:43:53.274 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executed": 0, 2024-03-07T07:43:53.274 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_item_in_journal": 0 2024-03-07T07:43:53.274 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.275 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-07T07:43:53.275 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-03-07T07:43:53.275 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2024-03-07T07:43:53.275 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-03-07T07:43:53.275 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 819, 2024-03-07T07:43:53.275 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 858517, 2024-03-07T07:43:53.275 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-03-07T07:43:53.275 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 819, 2024-03-07T07:43:53.275 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2024-03-07T07:43:53.275 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2024-03-07T07:43:53.275 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 819, 2024-03-07T07:43:53.275 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 858517, 2024-03-07T07:43:53.275 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-03-07T07:43:53.275 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:53.275 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:53.276 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:53.276 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-07T07:43:53.276 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.276 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-objecter_bytes": { 2024-03-07T07:43:53.276 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-03-07T07:43:53.276 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2024-03-07T07:43:53.276 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-03-07T07:43:53.276 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2024-03-07T07:43:53.276 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2024-03-07T07:43:53.276 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-03-07T07:43:53.276 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2024-03-07T07:43:53.276 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 721, 2024-03-07T07:43:53.276 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 901, 2024-03-07T07:43:53.276 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 721, 2024-03-07T07:43:53.277 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 901, 2024-03-07T07:43:53.277 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-03-07T07:43:53.277 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:53.277 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:53.277 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:53.277 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-07T07:43:53.277 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.277 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-objecter_ops": { 2024-03-07T07:43:53.277 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-03-07T07:43:53.277 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 1024, 2024-03-07T07:43:53.277 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-03-07T07:43:53.277 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2024-03-07T07:43:53.277 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2024-03-07T07:43:53.277 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-03-07T07:43:53.277 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2024-03-07T07:43:53.278 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 721, 2024-03-07T07:43:53.278 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 721, 2024-03-07T07:43:53.278 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 721, 2024-03-07T07:43:53.278 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 721, 2024-03-07T07:43:53.278 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-03-07T07:43:53.278 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:53.278 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:53.278 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:53.278 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-07T07:43:53.278 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.278 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-write_buf_throttle": { 2024-03-07T07:43:53.278 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-03-07T07:43:53.278 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 3758096384, 2024-03-07T07:43:53.278 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-03-07T07:43:53.279 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2024-03-07T07:43:53.279 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2024-03-07T07:43:53.279 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-03-07T07:43:53.279 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2024-03-07T07:43:53.279 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2024-03-07T07:43:53.279 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2024-03-07T07:43:53.279 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 0, 2024-03-07T07:43:53.279 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 0, 2024-03-07T07:43:53.279 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-03-07T07:43:53.279 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:53.279 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:53.279 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:53.279 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-07T07:43:53.279 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:53.280 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-write_buf_throttle-0x55dbe5b44960": { 2024-03-07T07:43:53.280 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-03-07T07:43:53.280 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 3758096384, 2024-03-07T07:43:53.280 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-03-07T07:43:53.280 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2024-03-07T07:43:53.280 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2024-03-07T07:43:53.280 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-03-07T07:43:53.280 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2024-03-07T07:43:53.280 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2024-03-07T07:43:53.280 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2024-03-07T07:43:53.280 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 0, 2024-03-07T07:43:53.280 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 0, 2024-03-07T07:43:53.280 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-03-07T07:43:53.280 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:53.281 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:53.281 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:53.281 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-07T07:43:53.281 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-07T07:43:53.281 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-03-07T07:43:53.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:43:53 smithi177 ceph-mon[36578]: pgmap v429: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 10 MiB/s wr, 13 op/s 2024-03-07T07:43:53.679 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:43:53 smithi160 ceph-mon[31963]: pgmap v429: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 10 MiB/s wr, 13 op/s 2024-03-07T07:43:53.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:43:53 smithi178 ceph-mon[36650]: pgmap v429: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 10 MiB/s wr, 13 op/s 2024-03-07T07:43:54.074 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-03-07T07:43:54.074 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-03-07T07:43:54.075 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-03-07T07:43:54.075 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2024-03-07T07:43:54.075 DEBUG:tasks.check_counter:Getting stats from k 2024-03-07T07:43:54.075 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph daemon mds.k perf dump 2024-03-07T07:43:54.332 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.b/config 2024-03-07T07:43:55.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:43:55 smithi160 ceph-mon[31963]: pgmap v430: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4.7 MiB/s wr, 13 op/s 2024-03-07T07:43:55.762 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:43:55 smithi177 ceph-mon[36578]: pgmap v430: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4.7 MiB/s wr, 13 op/s 2024-03-07T07:43:55.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:43:55 smithi178 ceph-mon[36650]: pgmap v430: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4.7 MiB/s wr, 13 op/s 2024-03-07T07:43:56.143 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-03-07T07:43:56.144 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-0": { 2024-03-07T07:43:56.144 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 4, 2024-03-07T07:43:56.144 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 4, 2024-03-07T07:43:56.144 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 3072, 2024-03-07T07:43:56.144 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 448, 2024-03-07T07:43:56.144 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 4, 2024-03-07T07:43:56.144 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 0, 2024-03-07T07:43:56.144 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.001370731, 2024-03-07T07:43:56.144 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.000034804, 2024-03-07T07:43:56.144 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.000635815, 2024-03-07T07:43:56.144 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-07T07:43:56.144 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2024-03-07T07:43:56.145 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 3, 2024-03-07T07:43:56.145 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000728325, 2024-03-07T07:43:56.145 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000242775 2024-03-07T07:43:56.145 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:56.145 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2024-03-07T07:43:56.145 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:56.145 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:56.145 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:56.145 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:56.145 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-03-07T07:43:56.145 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 448 2024-03-07T07:43:56.145 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:56.145 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-1": { 2024-03-07T07:43:56.145 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 94, 2024-03-07T07:43:56.146 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 89, 2024-03-07T07:43:56.146 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 712272, 2024-03-07T07:43:56.146 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 52176, 2024-03-07T07:43:56.146 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 2, 2024-03-07T07:43:56.146 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 1, 2024-03-07T07:43:56.146 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.014867766, 2024-03-07T07:43:56.146 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.003451104, 2024-03-07T07:43:56.146 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.015054665, 2024-03-07T07:43:56.146 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.001487608, 2024-03-07T07:43:56.146 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2024-03-07T07:43:56.146 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 88, 2024-03-07T07:43:56.146 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.003550576, 2024-03-07T07:43:56.146 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000040347 2024-03-07T07:43:56.146 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:56.146 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2024-03-07T07:43:56.147 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:56.147 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:56.147 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:56.147 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:56.147 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 712272, 2024-03-07T07:43:56.147 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 52176 2024-03-07T07:43:56.147 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:56.147 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-2": { 2024-03-07T07:43:56.147 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 1, 2024-03-07T07:43:56.147 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 67, 2024-03-07T07:43:56.147 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 160, 2024-03-07T07:43:56.148 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 89328, 2024-03-07T07:43:56.148 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 3, 2024-03-07T07:43:56.148 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 2, 2024-03-07T07:43:56.148 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.003455317, 2024-03-07T07:43:56.148 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.001961573, 2024-03-07T07:43:56.148 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.000937548, 2024-03-07T07:43:56.148 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-07T07:43:56.148 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2024-03-07T07:43:56.148 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 65, 2024-03-07T07:43:56.148 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.002626924, 2024-03-07T07:43:56.148 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000040414 2024-03-07T07:43:56.148 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:56.148 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2024-03-07T07:43:56.148 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:56.149 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:56.149 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:56.149 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:56.149 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-07T07:43:56.149 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 89328 2024-03-07T07:43:56.149 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:56.149 INFO:teuthology.orchestra.run.smithi177.stdout: "cct": { 2024-03-07T07:43:56.149 INFO:teuthology.orchestra.run.smithi177.stdout: "total_workers": 0, 2024-03-07T07:43:56.149 INFO:teuthology.orchestra.run.smithi177.stdout: "unhealthy_workers": 0 2024-03-07T07:43:56.149 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:56.149 INFO:teuthology.orchestra.run.smithi177.stdout: "mempool": { 2024-03-07T07:43:56.149 INFO:teuthology.orchestra.run.smithi177.stdout: "bloom_filter_bytes": 0, 2024-03-07T07:43:56.149 INFO:teuthology.orchestra.run.smithi177.stdout: "bloom_filter_items": 0, 2024-03-07T07:43:56.149 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_alloc_bytes": 0, 2024-03-07T07:43:56.149 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_alloc_items": 0, 2024-03-07T07:43:56.150 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_data_bytes": 0, 2024-03-07T07:43:56.150 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_data_items": 0, 2024-03-07T07:43:56.150 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-07T07:43:56.150 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_onode_items": 0, 2024-03-07T07:43:56.150 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-07T07:43:56.150 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_meta_items": 0, 2024-03-07T07:43:56.150 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_other_bytes": 0, 2024-03-07T07:43:56.150 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_other_items": 0, 2024-03-07T07:43:56.150 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-07T07:43:56.150 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_buffer_items": 0, 2024-03-07T07:43:56.150 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_extent_bytes": 0, 2024-03-07T07:43:56.150 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_extent_items": 0, 2024-03-07T07:43:56.150 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_blob_bytes": 0, 2024-03-07T07:43:56.150 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_blob_items": 0, 2024-03-07T07:43:56.150 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-07T07:43:56.151 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_shared_blob_items": 0, 2024-03-07T07:43:56.151 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-07T07:43:56.151 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_inline_bl_items": 0, 2024-03-07T07:43:56.151 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_bytes": 0, 2024-03-07T07:43:56.151 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_items": 0, 2024-03-07T07:43:56.151 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_txc_bytes": 0, 2024-03-07T07:43:56.151 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_txc_items": 0, 2024-03-07T07:43:56.151 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-07T07:43:56.151 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_deferred_items": 0, 2024-03-07T07:43:56.151 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_bytes": 0, 2024-03-07T07:43:56.151 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_items": 0, 2024-03-07T07:43:56.151 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_bytes": 0, 2024-03-07T07:43:56.151 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_items": 0, 2024-03-07T07:43:56.151 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_reader_bytes": 0, 2024-03-07T07:43:56.151 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_reader_items": 0, 2024-03-07T07:43:56.152 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_writer_bytes": 0, 2024-03-07T07:43:56.152 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_writer_items": 0, 2024-03-07T07:43:56.152 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_anon_bytes": 148903, 2024-03-07T07:43:56.152 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_anon_items": 29, 2024-03-07T07:43:56.152 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_meta_bytes": 0, 2024-03-07T07:43:56.152 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_meta_items": 0, 2024-03-07T07:43:56.152 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_bytes": 0, 2024-03-07T07:43:56.152 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_items": 0, 2024-03-07T07:43:56.152 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mapbl_bytes": 0, 2024-03-07T07:43:56.152 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mapbl_items": 0, 2024-03-07T07:43:56.152 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pglog_bytes": 0, 2024-03-07T07:43:56.153 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pglog_items": 0, 2024-03-07T07:43:56.153 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_bytes": 0, 2024-03-07T07:43:56.153 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_items": 0, 2024-03-07T07:43:56.153 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_mapping_bytes": 0, 2024-03-07T07:43:56.153 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_mapping_items": 0, 2024-03-07T07:43:56.153 INFO:teuthology.orchestra.run.smithi177.stdout: "pgmap_bytes": 0, 2024-03-07T07:43:56.153 INFO:teuthology.orchestra.run.smithi177.stdout: "pgmap_items": 0, 2024-03-07T07:43:56.153 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_co_bytes": 928, 2024-03-07T07:43:56.153 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_co_items": 2, 2024-03-07T07:43:56.153 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_1_bytes": 0, 2024-03-07T07:43:56.153 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_1_items": 0, 2024-03-07T07:43:56.153 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_2_bytes": 0, 2024-03-07T07:43:56.154 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_2_items": 0 2024-03-07T07:43:56.154 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-07T07:43:56.154 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-07T07:43:56.154 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-03-07T07:43:56.154 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2024-03-07T07:43:56.154 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-03-07T07:43:56.154 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 95, 2024-03-07T07:43:56.154 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 699018, 2024-03-07T07:43:56.154 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-03-07T07:43:56.154 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 95, 2024-03-07T07:43:56.154 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2024-03-07T07:43:56.154 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2024-03-07T07:43:56.154 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 95, 2024-03-07T07:43:56.154 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 699018, 2024-03-07T07:43:56.154 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-03-07T07:43:56.155 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-07T07:43:56.155 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-07T07:43:56.155 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-07T07:43:56.155 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-07T07:43:56.155 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-07T07:43:56.155 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-03-07T07:43:56.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:43:56 smithi178 ceph-mon[36650]: pgmap v431: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 MiB/s wr, 13 op/s 2024-03-07T07:43:56.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:43:56 smithi177 ceph-mon[36578]: pgmap v431: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 MiB/s wr, 13 op/s 2024-03-07T07:43:56.886 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-03-07T07:43:56.886 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-03-07T07:43:56.886 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-03-07T07:43:56.886 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-03-07T07:43:56.886 DEBUG:tasks.check_counter:Getting stats from c 2024-03-07T07:43:56.887 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph daemon mds.c perf dump 2024-03-07T07:43:56.927 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:43:56 smithi160 ceph-mon[31963]: pgmap v431: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 MiB/s wr, 13 op/s 2024-03-07T07:43:57.140 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.c/config 2024-03-07T07:43:59.061 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-03-07T07:43:59.061 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-0": { 2024-03-07T07:43:59.061 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 40, 2024-03-07T07:43:59.061 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 42, 2024-03-07T07:43:59.062 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 10004, 2024-03-07T07:43:59.062 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 60704, 2024-03-07T07:43:59.062 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 14, 2024-03-07T07:43:59.062 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 5, 2024-03-07T07:43:59.062 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.005498290, 2024-03-07T07:43:59.062 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.000546675, 2024-03-07T07:43:59.062 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.003327848, 2024-03-07T07:43:59.062 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.000419591, 2024-03-07T07:43:59.062 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-03-07T07:43:59.062 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 40, 2024-03-07T07:43:59.062 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.011957604, 2024-03-07T07:43:59.063 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000298940 2024-03-07T07:43:59.063 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.063 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-03-07T07:43:59.063 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 26, 2024-03-07T07:43:59.063 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000002935, 2024-03-07T07:43:59.063 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000112 2024-03-07T07:43:59.063 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.063 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-03-07T07:43:59.063 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 4064 2024-03-07T07:43:59.063 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.063 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-1": { 2024-03-07T07:43:59.063 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 157, 2024-03-07T07:43:59.063 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 147, 2024-03-07T07:43:59.064 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 748134, 2024-03-07T07:43:59.064 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 72266, 2024-03-07T07:43:59.064 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 6, 2024-03-07T07:43:59.064 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 5, 2024-03-07T07:43:59.064 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.018704468, 2024-03-07T07:43:59.064 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.004377972, 2024-03-07T07:43:59.064 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.018584189, 2024-03-07T07:43:59.064 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.001817232, 2024-03-07T07:43:59.064 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-03-07T07:43:59.064 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 147, 2024-03-07T07:43:59.064 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.022053337, 2024-03-07T07:43:59.064 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000150022 2024-03-07T07:43:59.064 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.065 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-03-07T07:43:59.065 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 9, 2024-03-07T07:43:59.065 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000001408, 2024-03-07T07:43:59.065 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000156 2024-03-07T07:43:59.065 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.065 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 743760, 2024-03-07T07:43:59.065 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 59072 2024-03-07T07:43:59.065 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.065 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-2": { 2024-03-07T07:43:59.065 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 214, 2024-03-07T07:43:59.065 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 281, 2024-03-07T07:43:59.065 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 65845, 2024-03-07T07:43:59.065 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 285178, 2024-03-07T07:43:59.065 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 6, 2024-03-07T07:43:59.066 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 4, 2024-03-07T07:43:59.066 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.020002264, 2024-03-07T07:43:59.066 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.009053767, 2024-03-07T07:43:59.066 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.009979532, 2024-03-07T07:43:59.066 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.000976117, 2024-03-07T07:43:59.066 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-03-07T07:43:59.066 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 279, 2024-03-07T07:43:59.066 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.013393251, 2024-03-07T07:43:59.066 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000048004 2024-03-07T07:43:59.066 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.067 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-03-07T07:43:59.067 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 408, 2024-03-07T07:43:59.067 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000071029, 2024-03-07T07:43:59.067 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000174 2024-03-07T07:43:59.067 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.067 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-07T07:43:59.067 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 187504 2024-03-07T07:43:59.067 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.067 INFO:teuthology.orchestra.run.smithi178.stdout: "cct": { 2024-03-07T07:43:59.067 INFO:teuthology.orchestra.run.smithi178.stdout: "total_workers": 1, 2024-03-07T07:43:59.067 INFO:teuthology.orchestra.run.smithi178.stdout: "unhealthy_workers": 0 2024-03-07T07:43:59.067 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.067 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-MDSRank": { 2024-03-07T07:43:59.068 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-03-07T07:43:59.068 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-03-07T07:43:59.068 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 57, 2024-03-07T07:43:59.068 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.001289276, 2024-03-07T07:43:59.068 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000022618 2024-03-07T07:43:59.068 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-03-07T07:43:59.068 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.068 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-PurgeQueue": { 2024-03-07T07:43:59.068 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-03-07T07:43:59.068 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-03-07T07:43:59.068 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 2, 2024-03-07T07:43:59.068 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000005514, 2024-03-07T07:43:59.069 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000002757 2024-03-07T07:43:59.069 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-03-07T07:43:59.069 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.069 INFO:teuthology.orchestra.run.smithi178.stdout: "mds": { 2024-03-07T07:43:59.069 INFO:teuthology.orchestra.run.smithi178.stdout: "request": 6, 2024-03-07T07:43:59.069 INFO:teuthology.orchestra.run.smithi178.stdout: "reply": 5, 2024-03-07T07:43:59.069 INFO:teuthology.orchestra.run.smithi178.stdout: "reply_latency": { 2024-03-07T07:43:59.069 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 5, 2024-03-07T07:43:59.069 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.001978022, 2024-03-07T07:43:59.069 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000395604 2024-03-07T07:43:59.069 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.069 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_reply": 0, 2024-03-07T07:43:59.069 INFO:teuthology.orchestra.run.smithi178.stdout: "forward": 1, 2024-03-07T07:43:59.069 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_complete": 0, 2024-03-07T07:43:59.069 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_keys": 0, 2024-03-07T07:43:59.070 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_commit": 11, 2024-03-07T07:43:59.070 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_split": 0, 2024-03-07T07:43:59.070 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_merge": 0, 2024-03-07T07:43:59.070 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes": 16, 2024-03-07T07:43:59.070 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_top": 0, 2024-03-07T07:43:59.070 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_bottom": 16, 2024-03-07T07:43:59.070 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pin_tail": 0, 2024-03-07T07:43:59.070 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pinned": 13, 2024-03-07T07:43:59.070 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_expired": 4, 2024-03-07T07:43:59.070 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_with_caps": 2, 2024-03-07T07:43:59.070 INFO:teuthology.orchestra.run.smithi178.stdout: "caps": 2, 2024-03-07T07:43:59.070 INFO:teuthology.orchestra.run.smithi178.stdout: "subtrees": 3, 2024-03-07T07:43:59.070 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse": 12, 2024-03-07T07:43:59.070 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_hit": 9, 2024-03-07T07:43:59.070 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_forward": 0, 2024-03-07T07:43:59.071 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_discover": 3, 2024-03-07T07:43:59.071 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_dir_fetch": 0, 2024-03-07T07:43:59.071 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_remote_ino": 0, 2024-03-07T07:43:59.071 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_lock": 0, 2024-03-07T07:43:59.071 INFO:teuthology.orchestra.run.smithi178.stdout: "load_cent": 0, 2024-03-07T07:43:59.071 INFO:teuthology.orchestra.run.smithi178.stdout: "q": 0, 2024-03-07T07:43:59.071 INFO:teuthology.orchestra.run.smithi178.stdout: "exported": 0, 2024-03-07T07:43:59.071 INFO:teuthology.orchestra.run.smithi178.stdout: "exported_inodes": 0, 2024-03-07T07:43:59.071 INFO:teuthology.orchestra.run.smithi178.stdout: "imported": 0, 2024-03-07T07:43:59.071 INFO:teuthology.orchestra.run.smithi178.stdout: "imported_inodes": 0, 2024-03-07T07:43:59.071 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_dir_fetch": 0, 2024-03-07T07:43:59.071 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_backtrace_fetch": 0, 2024-03-07T07:43:59.072 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_peer_discover": 0, 2024-03-07T07:43:59.072 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rfiles": 4, 2024-03-07T07:43:59.072 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rbytes": 234, 2024-03-07T07:43:59.072 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rsnaps": 0, 2024-03-07T07:43:59.072 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_fetch": 0, 2024-03-07T07:43:59.072 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_set_tag": 0, 2024-03-07T07:43:59.072 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_repaired": 0, 2024-03-07T07:43:59.072 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_inotable_repaired": 0, 2024-03-07T07:43:59.072 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_inodes": 0, 2024-03-07T07:43:59.072 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_base_inodes": 0, 2024-03-07T07:43:59.072 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dirfrag_rstats": 0, 2024-03-07T07:43:59.072 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_file_inodes": 0, 2024-03-07T07:43:59.072 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_inode_file_caps": 0, 2024-03-07T07:43:59.072 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_revoke": 0, 2024-03-07T07:43:59.073 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_grant": 0, 2024-03-07T07:43:59.073 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_trunc": 0, 2024-03-07T07:43:59.073 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-07T07:43:59.073 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-07T07:43:59.073 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps": 0, 2024-03-07T07:43:59.073 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps_dirty": 0, 2024-03-07T07:43:59.073 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_cap_release": 0, 2024-03-07T07:43:59.073 INFO:teuthology.orchestra.run.smithi178.stdout: "process_request_cap_release": 0 2024-03-07T07:43:59.073 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.073 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_cache": { 2024-03-07T07:43:59.073 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update": 0, 2024-03-07T07:43:59.073 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update_receipt": 9, 2024-03-07T07:43:59.073 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_try_discover": 6, 2024-03-07T07:43:59.074 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_send_discover": 7, 2024-03-07T07:43:59.074 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_handle_discover": 0, 2024-03-07T07:43:59.074 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays": 0, 2024-03-07T07:43:59.074 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_delayed": 0, 2024-03-07T07:43:59.074 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_enqueuing": 0, 2024-03-07T07:43:59.074 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_created": 0, 2024-03-07T07:43:59.074 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_enqueued": 0, 2024-03-07T07:43:59.074 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_reintegrated": 0, 2024-03-07T07:43:59.074 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_migrated": 0, 2024-03-07T07:43:59.074 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_processing": 0, 2024-03-07T07:43:59.074 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_enqueued": 0, 2024-03-07T07:43:59.074 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_prioritized": 0, 2024-03-07T07:43:59.074 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_started": 0, 2024-03-07T07:43:59.074 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_completed": 0, 2024-03-07T07:43:59.075 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_enqueue_scrub": 0, 2024-03-07T07:43:59.075 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_exportdir": 0, 2024-03-07T07:43:59.075 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_flush": 0, 2024-03-07T07:43:59.075 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragmentdir": 0, 2024-03-07T07:43:59.075 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragstats": 0, 2024-03-07T07:43:59.075 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_inodestats": 0 2024-03-07T07:43:59.075 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.075 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_log": { 2024-03-07T07:43:59.075 INFO:teuthology.orchestra.run.smithi178.stdout: "evlrg": 0, 2024-03-07T07:43:59.075 INFO:teuthology.orchestra.run.smithi178.stdout: "evadd": 7, 2024-03-07T07:43:59.075 INFO:teuthology.orchestra.run.smithi178.stdout: "evex": 0, 2024-03-07T07:43:59.075 INFO:teuthology.orchestra.run.smithi178.stdout: "evtrm": 0, 2024-03-07T07:43:59.075 INFO:teuthology.orchestra.run.smithi178.stdout: "ev": 7, 2024-03-07T07:43:59.076 INFO:teuthology.orchestra.run.smithi178.stdout: "evexg": 0, 2024-03-07T07:43:59.076 INFO:teuthology.orchestra.run.smithi178.stdout: "evexd": 0, 2024-03-07T07:43:59.076 INFO:teuthology.orchestra.run.smithi178.stdout: "segadd": 2, 2024-03-07T07:43:59.076 INFO:teuthology.orchestra.run.smithi178.stdout: "segex": 0, 2024-03-07T07:43:59.076 INFO:teuthology.orchestra.run.smithi178.stdout: "segtrm": 0, 2024-03-07T07:43:59.076 INFO:teuthology.orchestra.run.smithi178.stdout: "seg": 2, 2024-03-07T07:43:59.076 INFO:teuthology.orchestra.run.smithi178.stdout: "segmjr": 2, 2024-03-07T07:43:59.076 INFO:teuthology.orchestra.run.smithi178.stdout: "segexg": 0, 2024-03-07T07:43:59.076 INFO:teuthology.orchestra.run.smithi178.stdout: "segexd": 0, 2024-03-07T07:43:59.076 INFO:teuthology.orchestra.run.smithi178.stdout: "expos": 4194304, 2024-03-07T07:43:59.076 INFO:teuthology.orchestra.run.smithi178.stdout: "wrpos": 4196264, 2024-03-07T07:43:59.076 INFO:teuthology.orchestra.run.smithi178.stdout: "rdpos": 0, 2024-03-07T07:43:59.076 INFO:teuthology.orchestra.run.smithi178.stdout: "jlat": { 2024-03-07T07:43:59.076 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 6, 2024-03-07T07:43:59.076 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.009212178, 2024-03-07T07:43:59.077 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.001535363 2024-03-07T07:43:59.077 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.077 INFO:teuthology.orchestra.run.smithi178.stdout: "replayed": 0 2024-03-07T07:43:59.077 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.077 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_mem": { 2024-03-07T07:43:59.077 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 20, 2024-03-07T07:43:59.077 INFO:teuthology.orchestra.run.smithi178.stdout: "ino+": 22, 2024-03-07T07:43:59.077 INFO:teuthology.orchestra.run.smithi178.stdout: "ino-": 2, 2024-03-07T07:43:59.077 INFO:teuthology.orchestra.run.smithi178.stdout: "dir": 19, 2024-03-07T07:43:59.077 INFO:teuthology.orchestra.run.smithi178.stdout: "dir+": 21, 2024-03-07T07:43:59.077 INFO:teuthology.orchestra.run.smithi178.stdout: "dir-": 2, 2024-03-07T07:43:59.077 INFO:teuthology.orchestra.run.smithi178.stdout: "dn": 16, 2024-03-07T07:43:59.077 INFO:teuthology.orchestra.run.smithi178.stdout: "dn+": 20, 2024-03-07T07:43:59.077 INFO:teuthology.orchestra.run.smithi178.stdout: "dn-": 4, 2024-03-07T07:43:59.078 INFO:teuthology.orchestra.run.smithi178.stdout: "cap": 2, 2024-03-07T07:43:59.078 INFO:teuthology.orchestra.run.smithi178.stdout: "cap+": 5, 2024-03-07T07:43:59.078 INFO:teuthology.orchestra.run.smithi178.stdout: "cap-": 3, 2024-03-07T07:43:59.078 INFO:teuthology.orchestra.run.smithi178.stdout: "rss": 47564, 2024-03-07T07:43:59.078 INFO:teuthology.orchestra.run.smithi178.stdout: "heap": 223612 2024-03-07T07:43:59.078 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.078 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_server": { 2024-03-07T07:43:59.078 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_client_request": 9, 2024-03-07T07:43:59.078 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_server_request": 0, 2024-03-07T07:43:59.078 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_request": 6, 2024-03-07T07:43:59.078 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_session": 11, 2024-03-07T07:43:59.078 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_peer_request": 1, 2024-03-07T07:43:59.078 INFO:teuthology.orchestra.run.smithi178.stdout: "req_create_latency": { 2024-03-07T07:43:59.078 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:43:59.079 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:43:59.079 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:43:59.079 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.079 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getattr_latency": { 2024-03-07T07:43:59.079 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 1, 2024-03-07T07:43:59.079 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.001118101, 2024-03-07T07:43:59.079 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.001118101 2024-03-07T07:43:59.079 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.079 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getfilelock_latency": { 2024-03-07T07:43:59.079 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:43:59.079 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:43:59.079 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:43:59.079 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.079 INFO:teuthology.orchestra.run.smithi178.stdout: "req_link_latency": { 2024-03-07T07:43:59.079 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:43:59.080 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:43:59.080 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:43:59.080 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.080 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookup_latency": { 2024-03-07T07:43:59.080 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 2, 2024-03-07T07:43:59.080 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000625907, 2024-03-07T07:43:59.080 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000312953 2024-03-07T07:43:59.080 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.080 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookuphash_latency": { 2024-03-07T07:43:59.080 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:43:59.080 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:43:59.080 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:43:59.080 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.080 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupino_latency": { 2024-03-07T07:43:59.080 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 2, 2024-03-07T07:43:59.081 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000234014, 2024-03-07T07:43:59.081 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000117007 2024-03-07T07:43:59.081 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.081 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupname_latency": { 2024-03-07T07:43:59.081 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:43:59.081 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:43:59.081 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:43:59.081 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.081 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupparent_latency": { 2024-03-07T07:43:59.081 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:43:59.081 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:43:59.081 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:43:59.081 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.082 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupsnap_latency": { 2024-03-07T07:43:59.082 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:43:59.082 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:43:59.082 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:43:59.082 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.082 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lssnap_latency": { 2024-03-07T07:43:59.082 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:43:59.082 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:43:59.082 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:43:59.082 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.082 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mkdir_latency": { 2024-03-07T07:43:59.082 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:43:59.082 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:43:59.082 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:43:59.082 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.083 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mknod_latency": { 2024-03-07T07:43:59.083 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:43:59.083 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:43:59.083 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:43:59.083 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.084 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mksnap_latency": { 2024-03-07T07:43:59.084 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:43:59.084 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:43:59.084 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:43:59.084 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.084 INFO:teuthology.orchestra.run.smithi178.stdout: "req_open_latency": { 2024-03-07T07:43:59.084 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:43:59.084 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:43:59.084 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:43:59.084 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.084 INFO:teuthology.orchestra.run.smithi178.stdout: "req_readdir_latency": { 2024-03-07T07:43:59.084 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:43:59.084 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:43:59.085 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:43:59.085 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.085 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rename_latency": { 2024-03-07T07:43:59.085 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:43:59.085 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:43:59.085 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:43:59.085 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.085 INFO:teuthology.orchestra.run.smithi178.stdout: "req_renamesnap_latency": { 2024-03-07T07:43:59.085 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:43:59.085 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:43:59.086 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:43:59.086 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.086 INFO:teuthology.orchestra.run.smithi178.stdout: "req_snapdiff_latency": { 2024-03-07T07:43:59.086 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:43:59.086 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:43:59.086 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:43:59.086 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.086 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmdir_latency": { 2024-03-07T07:43:59.086 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:43:59.086 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:43:59.086 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:43:59.086 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.086 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmsnap_latency": { 2024-03-07T07:43:59.087 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:43:59.087 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:43:59.087 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:43:59.087 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.087 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmxattr_latency": { 2024-03-07T07:43:59.087 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:43:59.087 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:43:59.087 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:43:59.087 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.087 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setattr_latency": { 2024-03-07T07:43:59.087 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:43:59.087 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:43:59.087 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:43:59.088 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.088 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setdirlayout_latency": { 2024-03-07T07:43:59.088 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:43:59.088 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:43:59.088 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:43:59.088 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.088 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setfilelock_latency": { 2024-03-07T07:43:59.088 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:43:59.088 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:43:59.088 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:43:59.088 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.088 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setlayout_latency": { 2024-03-07T07:43:59.088 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:43:59.088 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:43:59.089 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:43:59.089 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.089 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setxattr_latency": { 2024-03-07T07:43:59.089 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:43:59.089 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:43:59.089 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:43:59.089 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.089 INFO:teuthology.orchestra.run.smithi178.stdout: "req_symlink_latency": { 2024-03-07T07:43:59.089 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:43:59.089 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:43:59.089 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:43:59.089 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.089 INFO:teuthology.orchestra.run.smithi178.stdout: "req_unlink_latency": { 2024-03-07T07:43:59.090 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:43:59.090 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:43:59.090 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:43:59.090 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.090 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_revoke_eviction": 0, 2024-03-07T07:43:59.090 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_acquisition_throttle": 0, 2024-03-07T07:43:59.090 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getvxattr_latency": { 2024-03-07T07:43:59.090 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:43:59.090 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:43:59.090 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:43:59.090 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-03-07T07:43:59.090 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.090 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_sessions": { 2024-03-07T07:43:59.091 INFO:teuthology.orchestra.run.smithi178.stdout: "session_count": 1, 2024-03-07T07:43:59.091 INFO:teuthology.orchestra.run.smithi178.stdout: "session_add": 3, 2024-03-07T07:43:59.091 INFO:teuthology.orchestra.run.smithi178.stdout: "session_remove": 2, 2024-03-07T07:43:59.091 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_open": 1, 2024-03-07T07:43:59.091 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_stale": 0, 2024-03-07T07:43:59.091 INFO:teuthology.orchestra.run.smithi178.stdout: "total_load": 4, 2024-03-07T07:43:59.091 INFO:teuthology.orchestra.run.smithi178.stdout: "average_load": 2, 2024-03-07T07:43:59.091 INFO:teuthology.orchestra.run.smithi178.stdout: "avg_session_uptime": 166, 2024-03-07T07:43:59.091 INFO:teuthology.orchestra.run.smithi178.stdout: "mdthresh_evicted": 0 2024-03-07T07:43:59.091 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.091 INFO:teuthology.orchestra.run.smithi178.stdout: "mempool": { 2024-03-07T07:43:59.091 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_bytes": 0, 2024-03-07T07:43:59.091 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_items": 0, 2024-03-07T07:43:59.092 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_bytes": 0, 2024-03-07T07:43:59.092 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_items": 0, 2024-03-07T07:43:59.092 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_bytes": 0, 2024-03-07T07:43:59.092 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_items": 0, 2024-03-07T07:43:59.092 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-07T07:43:59.092 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_items": 0, 2024-03-07T07:43:59.092 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-07T07:43:59.092 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_items": 0, 2024-03-07T07:43:59.092 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_bytes": 0, 2024-03-07T07:43:59.092 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_items": 0, 2024-03-07T07:43:59.092 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-07T07:43:59.092 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_items": 0, 2024-03-07T07:43:59.092 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_bytes": 0, 2024-03-07T07:43:59.093 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_items": 0, 2024-03-07T07:43:59.093 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_bytes": 0, 2024-03-07T07:43:59.093 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_items": 0, 2024-03-07T07:43:59.093 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-07T07:43:59.093 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_items": 0, 2024-03-07T07:43:59.093 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-07T07:43:59.093 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_items": 0, 2024-03-07T07:43:59.093 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_bytes": 0, 2024-03-07T07:43:59.093 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_items": 0, 2024-03-07T07:43:59.093 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_bytes": 0, 2024-03-07T07:43:59.093 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_items": 0, 2024-03-07T07:43:59.093 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-07T07:43:59.093 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_items": 0, 2024-03-07T07:43:59.093 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_bytes": 0, 2024-03-07T07:43:59.094 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_items": 0, 2024-03-07T07:43:59.094 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_bytes": 0, 2024-03-07T07:43:59.094 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_items": 0, 2024-03-07T07:43:59.094 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_bytes": 0, 2024-03-07T07:43:59.094 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_items": 0, 2024-03-07T07:43:59.094 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_bytes": 0, 2024-03-07T07:43:59.094 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_items": 0, 2024-03-07T07:43:59.094 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_bytes": 247447, 2024-03-07T07:43:59.094 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_items": 87, 2024-03-07T07:43:59.094 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_bytes": 88, 2024-03-07T07:43:59.094 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_items": 1, 2024-03-07T07:43:59.094 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_bytes": 0, 2024-03-07T07:43:59.094 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_items": 0, 2024-03-07T07:43:59.095 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_bytes": 0, 2024-03-07T07:43:59.095 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_items": 0, 2024-03-07T07:43:59.095 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_bytes": 0, 2024-03-07T07:43:59.095 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_items": 0, 2024-03-07T07:43:59.095 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_bytes": 7624, 2024-03-07T07:43:59.095 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_items": 160, 2024-03-07T07:43:59.095 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_bytes": 0, 2024-03-07T07:43:59.095 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_items": 0, 2024-03-07T07:43:59.095 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_bytes": 0, 2024-03-07T07:43:59.095 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_items": 0, 2024-03-07T07:43:59.095 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_bytes": 80461, 2024-03-07T07:43:59.095 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_items": 336, 2024-03-07T07:43:59.095 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_bytes": 0, 2024-03-07T07:43:59.095 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_items": 0, 2024-03-07T07:43:59.096 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_bytes": 0, 2024-03-07T07:43:59.096 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_items": 0 2024-03-07T07:43:59.096 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.096 INFO:teuthology.orchestra.run.smithi178.stdout: "objecter": { 2024-03-07T07:43:59.096 INFO:teuthology.orchestra.run.smithi178.stdout: "op_active": 0, 2024-03-07T07:43:59.096 INFO:teuthology.orchestra.run.smithi178.stdout: "op_laggy": 0, 2024-03-07T07:43:59.096 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send": 45, 2024-03-07T07:43:59.096 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send_bytes": 12722, 2024-03-07T07:43:59.096 INFO:teuthology.orchestra.run.smithi178.stdout: "op_resend": 33, 2024-03-07T07:43:59.096 INFO:teuthology.orchestra.run.smithi178.stdout: "op_reply": 45, 2024-03-07T07:43:59.096 INFO:teuthology.orchestra.run.smithi178.stdout: "op_latency": { 2024-03-07T07:43:59.096 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 45, 2024-03-07T07:43:59.096 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.215996113, 2024-03-07T07:43:59.097 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.004799913 2024-03-07T07:43:59.097 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.097 INFO:teuthology.orchestra.run.smithi178.stdout: "op_inflight": 0, 2024-03-07T07:43:59.097 INFO:teuthology.orchestra.run.smithi178.stdout: "oplen_avg": { 2024-03-07T07:43:59.097 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 45, 2024-03-07T07:43:59.097 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 69 2024-03-07T07:43:59.097 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.097 INFO:teuthology.orchestra.run.smithi178.stdout: "op": 45, 2024-03-07T07:43:59.097 INFO:teuthology.orchestra.run.smithi178.stdout: "op_r": 0, 2024-03-07T07:43:59.097 INFO:teuthology.orchestra.run.smithi178.stdout: "op_w": 45, 2024-03-07T07:43:59.097 INFO:teuthology.orchestra.run.smithi178.stdout: "op_rmw": 0, 2024-03-07T07:43:59.097 INFO:teuthology.orchestra.run.smithi178.stdout: "op_pg": 0, 2024-03-07T07:43:59.097 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_stat": 0, 2024-03-07T07:43:59.098 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_create": 10, 2024-03-07T07:43:59.098 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_read": 0, 2024-03-07T07:43:59.098 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_write": 6, 2024-03-07T07:43:59.098 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writefull": 8, 2024-03-07T07:43:59.098 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writesame": 0, 2024-03-07T07:43:59.098 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_append": 0, 2024-03-07T07:43:59.098 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_zero": 0, 2024-03-07T07:43:59.098 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_truncate": 0, 2024-03-07T07:43:59.098 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_delete": 6, 2024-03-07T07:43:59.098 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_mapext": 0, 2024-03-07T07:43:59.098 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_sparse_read": 0, 2024-03-07T07:43:59.098 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_clonerange": 0, 2024-03-07T07:43:59.099 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_getxattr": 0, 2024-03-07T07:43:59.099 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_setxattr": 20, 2024-03-07T07:43:59.099 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_cmpxattr": 0, 2024-03-07T07:43:59.099 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_rmxattr": 0, 2024-03-07T07:43:59.099 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_resetxattrs": 0, 2024-03-07T07:43:59.099 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_call": 0, 2024-03-07T07:43:59.099 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_watch": 0, 2024-03-07T07:43:59.099 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_notify": 0, 2024-03-07T07:43:59.099 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_src_cmpxattr": 0, 2024-03-07T07:43:59.099 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls": 0, 2024-03-07T07:43:59.099 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls_filter": 0, 2024-03-07T07:43:59.099 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_other": 0, 2024-03-07T07:43:59.099 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_active": 0, 2024-03-07T07:43:59.100 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_send": 0, 2024-03-07T07:43:59.100 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_resend": 0, 2024-03-07T07:43:59.100 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_ping": 0, 2024-03-07T07:43:59.100 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_active": 0, 2024-03-07T07:43:59.100 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_send": 0, 2024-03-07T07:43:59.100 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_resend": 0, 2024-03-07T07:43:59.100 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_active": 0, 2024-03-07T07:43:59.100 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_send": 0, 2024-03-07T07:43:59.100 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_resend": 0, 2024-03-07T07:43:59.100 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_active": 0, 2024-03-07T07:43:59.100 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_send": 0, 2024-03-07T07:43:59.100 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_resend": 0, 2024-03-07T07:43:59.100 INFO:teuthology.orchestra.run.smithi178.stdout: "command_active": 0, 2024-03-07T07:43:59.101 INFO:teuthology.orchestra.run.smithi178.stdout: "command_send": 0, 2024-03-07T07:43:59.101 INFO:teuthology.orchestra.run.smithi178.stdout: "command_resend": 0, 2024-03-07T07:43:59.101 INFO:teuthology.orchestra.run.smithi178.stdout: "map_epoch": 87, 2024-03-07T07:43:59.101 INFO:teuthology.orchestra.run.smithi178.stdout: "map_full": 0, 2024-03-07T07:43:59.101 INFO:teuthology.orchestra.run.smithi178.stdout: "map_inc": 2, 2024-03-07T07:43:59.101 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_sessions": 8, 2024-03-07T07:43:59.101 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_open": 8, 2024-03-07T07:43:59.101 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_close": 0, 2024-03-07T07:43:59.101 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_laggy": 0, 2024-03-07T07:43:59.101 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_wr": 18, 2024-03-07T07:43:59.101 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_rd": 0, 2024-03-07T07:43:59.101 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_del": 1 2024-03-07T07:43:59.101 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.102 INFO:teuthology.orchestra.run.smithi178.stdout: "oft": { 2024-03-07T07:43:59.102 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_objs": 1, 2024-03-07T07:43:59.102 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_kv_pairs": 0, 2024-03-07T07:43:59.102 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_updates": 0, 2024-03-07T07:43:59.102 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_removes": 0 2024-03-07T07:43:59.102 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.102 INFO:teuthology.orchestra.run.smithi178.stdout: "osd-slow-ops": { 2024-03-07T07:43:59.102 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_ops_count": 0 2024-03-07T07:43:59.102 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.102 INFO:teuthology.orchestra.run.smithi178.stdout: "purge_queue": { 2024-03-07T07:43:59.102 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops": 0, 2024-03-07T07:43:59.102 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops_high_water": 0, 2024-03-07T07:43:59.102 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing": 0, 2024-03-07T07:43:59.102 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_high_water": 0, 2024-03-07T07:43:59.103 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed_ops": 0, 2024-03-07T07:43:59.103 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed": 0, 2024-03-07T07:43:59.103 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_item_in_journal": 0 2024-03-07T07:43:59.103 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.103 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-07T07:43:59.103 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-03-07T07:43:59.103 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-03-07T07:43:59.103 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-03-07T07:43:59.103 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 407, 2024-03-07T07:43:59.103 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 776961, 2024-03-07T07:43:59.103 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-03-07T07:43:59.104 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 407, 2024-03-07T07:43:59.104 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-03-07T07:43:59.104 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-03-07T07:43:59.104 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 407, 2024-03-07T07:43:59.104 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 776961, 2024-03-07T07:43:59.104 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-03-07T07:43:59.104 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:43:59.104 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:43:59.104 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:43:59.104 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-03-07T07:43:59.104 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.104 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_bytes": { 2024-03-07T07:43:59.104 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-03-07T07:43:59.105 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-03-07T07:43:59.105 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-03-07T07:43:59.105 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-03-07T07:43:59.105 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-03-07T07:43:59.105 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-03-07T07:43:59.105 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-03-07T07:43:59.105 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 45, 2024-03-07T07:43:59.105 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 12722, 2024-03-07T07:43:59.105 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 45, 2024-03-07T07:43:59.105 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 12722, 2024-03-07T07:43:59.105 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-03-07T07:43:59.105 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:43:59.105 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:43:59.106 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:43:59.106 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-03-07T07:43:59.106 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.106 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_ops": { 2024-03-07T07:43:59.106 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-03-07T07:43:59.106 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 1024, 2024-03-07T07:43:59.106 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-03-07T07:43:59.106 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-03-07T07:43:59.106 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-03-07T07:43:59.106 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-03-07T07:43:59.106 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-03-07T07:43:59.106 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 45, 2024-03-07T07:43:59.106 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 45, 2024-03-07T07:43:59.106 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 45, 2024-03-07T07:43:59.106 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 45, 2024-03-07T07:43:59.107 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-03-07T07:43:59.107 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:43:59.107 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:43:59.107 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:43:59.107 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-03-07T07:43:59.107 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.107 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle": { 2024-03-07T07:43:59.107 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-03-07T07:43:59.107 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-03-07T07:43:59.107 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-03-07T07:43:59.107 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-03-07T07:43:59.107 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-03-07T07:43:59.107 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-03-07T07:43:59.107 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-03-07T07:43:59.107 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-03-07T07:43:59.108 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-03-07T07:43:59.108 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 0, 2024-03-07T07:43:59.108 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 0, 2024-03-07T07:43:59.108 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-03-07T07:43:59.108 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:43:59.108 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:43:59.108 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:43:59.108 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-03-07T07:43:59.108 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:43:59.108 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle-0x555b212306e0": { 2024-03-07T07:43:59.108 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-03-07T07:43:59.108 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-03-07T07:43:59.109 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-03-07T07:43:59.109 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 7, 2024-03-07T07:43:59.109 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 1960, 2024-03-07T07:43:59.109 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-03-07T07:43:59.109 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 7, 2024-03-07T07:43:59.109 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-03-07T07:43:59.109 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-03-07T07:43:59.109 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 6, 2024-03-07T07:43:59.109 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 1960, 2024-03-07T07:43:59.109 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-03-07T07:43:59.109 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:43:59.109 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:43:59.110 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:43:59.110 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-03-07T07:43:59.110 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-03-07T07:43:59.110 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-03-07T07:43:59.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:43:59 smithi160 ceph-mon[31963]: pgmap v432: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.0 MiB/s wr, 10 op/s 2024-03-07T07:43:59.770 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:43:59 smithi177 ceph-mon[36578]: pgmap v432: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.0 MiB/s wr, 10 op/s 2024-03-07T07:43:59.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:43:59 smithi178 ceph-mon[36650]: pgmap v432: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.0 MiB/s wr, 10 op/s 2024-03-07T07:43:59.861 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-03-07T07:43:59.862 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-03-07T07:43:59.862 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-03-07T07:43:59.862 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=9 2024-03-07T07:43:59.862 DEBUG:tasks.check_counter:Getting stats from f 2024-03-07T07:43:59.862 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph daemon mds.f perf dump 2024-03-07T07:44:00.184 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.c/config 2024-03-07T07:44:01.661 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:01 smithi178 ceph-mon[36650]: pgmap v433: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-07T07:44:01.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:01 smithi160 ceph-mon[31963]: pgmap v433: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-07T07:44:01.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:01 smithi177 ceph-mon[36578]: pgmap v433: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-07T07:44:02.142 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-03-07T07:44:02.240 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-0": { 2024-03-07T07:44:02.240 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 655, 2024-03-07T07:44:02.240 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 654, 2024-03-07T07:44:02.240 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 825092, 2024-03-07T07:44:02.240 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 207158, 2024-03-07T07:44:02.240 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 6, 2024-03-07T07:44:02.240 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 2, 2024-03-07T07:44:02.241 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.039409393, 2024-03-07T07:44:02.241 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.011097294, 2024-03-07T07:44:02.241 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.011160680, 2024-03-07T07:44:02.241 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.013208681, 2024-03-07T07:44:02.241 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-03-07T07:44:02.241 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 653, 2024-03-07T07:44:02.241 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.014692479, 2024-03-07T07:44:02.241 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000022499 2024-03-07T07:44:02.241 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.241 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-03-07T07:44:02.241 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:02.241 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:02.241 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:02.241 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.242 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-03-07T07:44:02.242 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 448 2024-03-07T07:44:02.242 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.242 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-1": { 2024-03-07T07:44:02.242 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 105, 2024-03-07T07:44:02.242 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 99, 2024-03-07T07:44:02.242 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 731528, 2024-03-07T07:44:02.242 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 53610, 2024-03-07T07:44:02.242 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 4, 2024-03-07T07:44:02.242 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 3, 2024-03-07T07:44:02.242 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.015123834, 2024-03-07T07:44:02.242 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.003277084, 2024-03-07T07:44:02.242 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.015572453, 2024-03-07T07:44:02.243 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.001468138, 2024-03-07T07:44:02.243 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-03-07T07:44:02.243 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 98, 2024-03-07T07:44:02.243 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.005274983, 2024-03-07T07:44:02.243 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000053826 2024-03-07T07:44:02.243 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.243 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-03-07T07:44:02.243 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:02.243 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:02.243 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:02.243 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.243 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 729872, 2024-03-07T07:44:02.243 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 52272 2024-03-07T07:44:02.244 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.244 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-2": { 2024-03-07T07:44:02.244 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 264, 2024-03-07T07:44:02.244 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 329, 2024-03-07T07:44:02.244 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 64929, 2024-03-07T07:44:02.244 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 273091, 2024-03-07T07:44:02.244 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 4, 2024-03-07T07:44:02.244 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 3, 2024-03-07T07:44:02.244 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.022268731, 2024-03-07T07:44:02.244 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.008270109, 2024-03-07T07:44:02.244 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.005396506, 2024-03-07T07:44:02.244 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.006667304, 2024-03-07T07:44:02.244 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-03-07T07:44:02.245 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 327, 2024-03-07T07:44:02.245 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.008707379, 2024-03-07T07:44:02.245 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000026628 2024-03-07T07:44:02.245 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.245 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-03-07T07:44:02.245 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:02.245 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:02.245 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:02.245 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.245 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-07T07:44:02.245 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 189456 2024-03-07T07:44:02.245 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.246 INFO:teuthology.orchestra.run.smithi178.stdout: "cct": { 2024-03-07T07:44:02.246 INFO:teuthology.orchestra.run.smithi178.stdout: "total_workers": 1, 2024-03-07T07:44:02.246 INFO:teuthology.orchestra.run.smithi178.stdout: "unhealthy_workers": 0 2024-03-07T07:44:02.246 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.246 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-MDSRank": { 2024-03-07T07:44:02.246 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-03-07T07:44:02.246 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-03-07T07:44:02.246 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 1548, 2024-03-07T07:44:02.246 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.030164170, 2024-03-07T07:44:02.246 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000019485 2024-03-07T07:44:02.246 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-03-07T07:44:02.246 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.247 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-PurgeQueue": { 2024-03-07T07:44:02.247 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-03-07T07:44:02.247 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-03-07T07:44:02.247 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:02.247 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:02.247 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:02.247 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-03-07T07:44:02.247 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.247 INFO:teuthology.orchestra.run.smithi178.stdout: "mds": { 2024-03-07T07:44:02.247 INFO:teuthology.orchestra.run.smithi178.stdout: "request": 0, 2024-03-07T07:44:02.248 INFO:teuthology.orchestra.run.smithi178.stdout: "reply": 0, 2024-03-07T07:44:02.248 INFO:teuthology.orchestra.run.smithi178.stdout: "reply_latency": { 2024-03-07T07:44:02.248 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:02.248 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:02.248 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:02.248 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.248 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_reply": 0, 2024-03-07T07:44:02.248 INFO:teuthology.orchestra.run.smithi178.stdout: "forward": 0, 2024-03-07T07:44:02.248 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_complete": 0, 2024-03-07T07:44:02.248 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_keys": 0, 2024-03-07T07:44:02.248 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_commit": 0, 2024-03-07T07:44:02.249 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_split": 0, 2024-03-07T07:44:02.249 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_merge": 0, 2024-03-07T07:44:02.249 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes": 20, 2024-03-07T07:44:02.249 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_top": 4, 2024-03-07T07:44:02.249 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_bottom": 16, 2024-03-07T07:44:02.249 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pin_tail": 0, 2024-03-07T07:44:02.249 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pinned": 13, 2024-03-07T07:44:02.249 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_expired": 0, 2024-03-07T07:44:02.249 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_with_caps": 0, 2024-03-07T07:44:02.249 INFO:teuthology.orchestra.run.smithi178.stdout: "caps": 0, 2024-03-07T07:44:02.249 INFO:teuthology.orchestra.run.smithi178.stdout: "subtrees": 2, 2024-03-07T07:44:02.249 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse": 0, 2024-03-07T07:44:02.249 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_hit": 0, 2024-03-07T07:44:02.250 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_forward": 0, 2024-03-07T07:44:02.250 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_discover": 0, 2024-03-07T07:44:02.250 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_dir_fetch": 0, 2024-03-07T07:44:02.250 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_remote_ino": 0, 2024-03-07T07:44:02.250 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_lock": 0, 2024-03-07T07:44:02.250 INFO:teuthology.orchestra.run.smithi178.stdout: "load_cent": 0, 2024-03-07T07:44:02.250 INFO:teuthology.orchestra.run.smithi178.stdout: "q": 0, 2024-03-07T07:44:02.250 INFO:teuthology.orchestra.run.smithi178.stdout: "exported": 0, 2024-03-07T07:44:02.250 INFO:teuthology.orchestra.run.smithi178.stdout: "exported_inodes": 0, 2024-03-07T07:44:02.250 INFO:teuthology.orchestra.run.smithi178.stdout: "imported": 0, 2024-03-07T07:44:02.250 INFO:teuthology.orchestra.run.smithi178.stdout: "imported_inodes": 0, 2024-03-07T07:44:02.251 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_dir_fetch": 0, 2024-03-07T07:44:02.251 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_backtrace_fetch": 0, 2024-03-07T07:44:02.251 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_peer_discover": 0, 2024-03-07T07:44:02.251 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rfiles": 4, 2024-03-07T07:44:02.251 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rbytes": 234, 2024-03-07T07:44:02.251 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rsnaps": 0, 2024-03-07T07:44:02.251 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_fetch": 0, 2024-03-07T07:44:02.251 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_set_tag": 0, 2024-03-07T07:44:02.251 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_repaired": 0, 2024-03-07T07:44:02.251 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_inotable_repaired": 0, 2024-03-07T07:44:02.251 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_inodes": 0, 2024-03-07T07:44:02.252 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_base_inodes": 0, 2024-03-07T07:44:02.252 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dirfrag_rstats": 0, 2024-03-07T07:44:02.252 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_file_inodes": 0, 2024-03-07T07:44:02.252 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_inode_file_caps": 0, 2024-03-07T07:44:02.252 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_revoke": 0, 2024-03-07T07:44:02.252 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_grant": 0, 2024-03-07T07:44:02.252 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_trunc": 0, 2024-03-07T07:44:02.252 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-07T07:44:02.252 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-07T07:44:02.252 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps": 0, 2024-03-07T07:44:02.252 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps_dirty": 0, 2024-03-07T07:44:02.252 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_cap_release": 0, 2024-03-07T07:44:02.252 INFO:teuthology.orchestra.run.smithi178.stdout: "process_request_cap_release": 0 2024-03-07T07:44:02.253 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.253 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_cache": { 2024-03-07T07:44:02.253 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update": 0, 2024-03-07T07:44:02.253 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update_receipt": 0, 2024-03-07T07:44:02.253 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_try_discover": 0, 2024-03-07T07:44:02.253 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_send_discover": 0, 2024-03-07T07:44:02.253 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_handle_discover": 0, 2024-03-07T07:44:02.253 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays": 0, 2024-03-07T07:44:02.253 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_delayed": 0, 2024-03-07T07:44:02.253 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_enqueuing": 0, 2024-03-07T07:44:02.253 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_created": 0, 2024-03-07T07:44:02.253 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_enqueued": 0, 2024-03-07T07:44:02.253 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_reintegrated": 0, 2024-03-07T07:44:02.253 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_migrated": 0, 2024-03-07T07:44:02.254 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_processing": 0, 2024-03-07T07:44:02.254 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_enqueued": 0, 2024-03-07T07:44:02.254 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_prioritized": 0, 2024-03-07T07:44:02.254 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_started": 0, 2024-03-07T07:44:02.254 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_completed": 0, 2024-03-07T07:44:02.254 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_enqueue_scrub": 0, 2024-03-07T07:44:02.254 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_exportdir": 0, 2024-03-07T07:44:02.254 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_flush": 0, 2024-03-07T07:44:02.254 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragmentdir": 0, 2024-03-07T07:44:02.254 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragstats": 0, 2024-03-07T07:44:02.254 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_inodestats": 0 2024-03-07T07:44:02.254 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.254 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_log": { 2024-03-07T07:44:02.254 INFO:teuthology.orchestra.run.smithi178.stdout: "evlrg": 0, 2024-03-07T07:44:02.255 INFO:teuthology.orchestra.run.smithi178.stdout: "evadd": 0, 2024-03-07T07:44:02.255 INFO:teuthology.orchestra.run.smithi178.stdout: "evex": 0, 2024-03-07T07:44:02.255 INFO:teuthology.orchestra.run.smithi178.stdout: "evtrm": 0, 2024-03-07T07:44:02.255 INFO:teuthology.orchestra.run.smithi178.stdout: "ev": 171, 2024-03-07T07:44:02.255 INFO:teuthology.orchestra.run.smithi178.stdout: "evexg": 0, 2024-03-07T07:44:02.255 INFO:teuthology.orchestra.run.smithi178.stdout: "evexd": 0, 2024-03-07T07:44:02.255 INFO:teuthology.orchestra.run.smithi178.stdout: "segadd": 0, 2024-03-07T07:44:02.255 INFO:teuthology.orchestra.run.smithi178.stdout: "segex": 0, 2024-03-07T07:44:02.255 INFO:teuthology.orchestra.run.smithi178.stdout: "segtrm": 0, 2024-03-07T07:44:02.255 INFO:teuthology.orchestra.run.smithi178.stdout: "seg": 2, 2024-03-07T07:44:02.255 INFO:teuthology.orchestra.run.smithi178.stdout: "segmjr": 12, 2024-03-07T07:44:02.255 INFO:teuthology.orchestra.run.smithi178.stdout: "segexg": 0, 2024-03-07T07:44:02.255 INFO:teuthology.orchestra.run.smithi178.stdout: "segexd": 0, 2024-03-07T07:44:02.256 INFO:teuthology.orchestra.run.smithi178.stdout: "expos": 4723806, 2024-03-07T07:44:02.256 INFO:teuthology.orchestra.run.smithi178.stdout: "wrpos": 4825395, 2024-03-07T07:44:02.256 INFO:teuthology.orchestra.run.smithi178.stdout: "rdpos": 4823743, 2024-03-07T07:44:02.256 INFO:teuthology.orchestra.run.smithi178.stdout: "jlat": { 2024-03-07T07:44:02.256 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:02.256 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:02.256 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:02.256 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.256 INFO:teuthology.orchestra.run.smithi178.stdout: "replayed": 171 2024-03-07T07:44:02.256 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.256 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_mem": { 2024-03-07T07:44:02.256 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 14, 2024-03-07T07:44:02.257 INFO:teuthology.orchestra.run.smithi178.stdout: "ino+": 20, 2024-03-07T07:44:02.257 INFO:teuthology.orchestra.run.smithi178.stdout: "ino-": 6, 2024-03-07T07:44:02.257 INFO:teuthology.orchestra.run.smithi178.stdout: "dir": 9, 2024-03-07T07:44:02.257 INFO:teuthology.orchestra.run.smithi178.stdout: "dir+": 11, 2024-03-07T07:44:02.257 INFO:teuthology.orchestra.run.smithi178.stdout: "dir-": 2, 2024-03-07T07:44:02.257 INFO:teuthology.orchestra.run.smithi178.stdout: "dn": 20, 2024-03-07T07:44:02.258 INFO:teuthology.orchestra.run.smithi178.stdout: "dn+": 25, 2024-03-07T07:44:02.258 INFO:teuthology.orchestra.run.smithi178.stdout: "dn-": 5, 2024-03-07T07:44:02.258 INFO:teuthology.orchestra.run.smithi178.stdout: "cap": 0, 2024-03-07T07:44:02.258 INFO:teuthology.orchestra.run.smithi178.stdout: "cap+": 0, 2024-03-07T07:44:02.258 INFO:teuthology.orchestra.run.smithi178.stdout: "cap-": 0, 2024-03-07T07:44:02.258 INFO:teuthology.orchestra.run.smithi178.stdout: "rss": 48184, 2024-03-07T07:44:02.258 INFO:teuthology.orchestra.run.smithi178.stdout: "heap": 215420 2024-03-07T07:44:02.258 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.258 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_server": { 2024-03-07T07:44:02.258 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_client_request": 0, 2024-03-07T07:44:02.258 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_server_request": 0, 2024-03-07T07:44:02.259 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_request": 0, 2024-03-07T07:44:02.259 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_session": 0, 2024-03-07T07:44:02.259 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_peer_request": 0, 2024-03-07T07:44:02.259 INFO:teuthology.orchestra.run.smithi178.stdout: "req_create_latency": { 2024-03-07T07:44:02.259 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:02.259 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:02.259 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:02.259 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.259 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getattr_latency": { 2024-03-07T07:44:02.259 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:02.259 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:02.260 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:02.260 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.260 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getfilelock_latency": { 2024-03-07T07:44:02.260 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:02.260 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:02.260 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:02.260 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.260 INFO:teuthology.orchestra.run.smithi178.stdout: "req_link_latency": { 2024-03-07T07:44:02.260 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:02.260 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:02.260 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:02.260 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.261 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookup_latency": { 2024-03-07T07:44:02.261 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:02.261 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:02.261 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:02.261 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.261 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookuphash_latency": { 2024-03-07T07:44:02.261 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:02.261 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:02.261 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:02.261 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.262 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupino_latency": { 2024-03-07T07:44:02.262 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:02.262 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:02.262 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:02.262 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.262 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupname_latency": { 2024-03-07T07:44:02.262 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:02.262 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:02.262 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:02.262 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.262 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupparent_latency": { 2024-03-07T07:44:02.263 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:02.263 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:02.263 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:02.263 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.263 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupsnap_latency": { 2024-03-07T07:44:02.263 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:02.263 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:02.263 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:02.263 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.263 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lssnap_latency": { 2024-03-07T07:44:02.263 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:02.263 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:02.264 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:02.264 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.264 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mkdir_latency": { 2024-03-07T07:44:02.264 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:02.264 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:02.264 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:02.265 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.265 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mknod_latency": { 2024-03-07T07:44:02.265 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:02.265 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:02.265 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:02.266 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.266 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mksnap_latency": { 2024-03-07T07:44:02.266 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:02.266 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:02.266 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:02.266 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.266 INFO:teuthology.orchestra.run.smithi178.stdout: "req_open_latency": { 2024-03-07T07:44:02.266 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:02.266 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:02.266 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:02.266 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.266 INFO:teuthology.orchestra.run.smithi178.stdout: "req_readdir_latency": { 2024-03-07T07:44:02.266 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:02.267 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:02.267 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:02.267 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.267 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rename_latency": { 2024-03-07T07:44:02.267 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:02.268 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:02.268 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:02.268 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.268 INFO:teuthology.orchestra.run.smithi178.stdout: "req_renamesnap_latency": { 2024-03-07T07:44:02.268 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:02.268 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:02.268 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:02.268 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.268 INFO:teuthology.orchestra.run.smithi178.stdout: "req_snapdiff_latency": { 2024-03-07T07:44:02.269 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:02.269 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:02.269 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:02.269 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.269 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmdir_latency": { 2024-03-07T07:44:02.269 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:02.269 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:02.269 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:02.269 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.269 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmsnap_latency": { 2024-03-07T07:44:02.269 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:02.269 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:02.270 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:02.270 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.270 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmxattr_latency": { 2024-03-07T07:44:02.270 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:02.270 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:02.270 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:02.270 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.270 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setattr_latency": { 2024-03-07T07:44:02.270 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:02.270 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:02.270 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:02.271 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.271 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setdirlayout_latency": { 2024-03-07T07:44:02.271 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:02.271 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:02.271 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:02.271 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.271 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setfilelock_latency": { 2024-03-07T07:44:02.271 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:02.271 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:02.271 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:02.271 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.271 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setlayout_latency": { 2024-03-07T07:44:02.271 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:02.272 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:02.272 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:02.272 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.272 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setxattr_latency": { 2024-03-07T07:44:02.272 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:02.272 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:02.272 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:02.272 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.272 INFO:teuthology.orchestra.run.smithi178.stdout: "req_symlink_latency": { 2024-03-07T07:44:02.272 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:02.272 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:02.273 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:02.273 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.273 INFO:teuthology.orchestra.run.smithi178.stdout: "req_unlink_latency": { 2024-03-07T07:44:02.273 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:02.273 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:02.273 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:02.273 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.273 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_revoke_eviction": 0, 2024-03-07T07:44:02.273 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_acquisition_throttle": 0, 2024-03-07T07:44:02.273 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getvxattr_latency": { 2024-03-07T07:44:02.273 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:02.274 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:02.274 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:02.274 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-03-07T07:44:02.274 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.274 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_sessions": { 2024-03-07T07:44:02.274 INFO:teuthology.orchestra.run.smithi178.stdout: "session_count": 1, 2024-03-07T07:44:02.274 INFO:teuthology.orchestra.run.smithi178.stdout: "session_add": 3, 2024-03-07T07:44:02.274 INFO:teuthology.orchestra.run.smithi178.stdout: "session_remove": 2, 2024-03-07T07:44:02.274 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_open": 2, 2024-03-07T07:44:02.274 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_stale": 0, 2024-03-07T07:44:02.274 INFO:teuthology.orchestra.run.smithi178.stdout: "total_load": 0, 2024-03-07T07:44:02.274 INFO:teuthology.orchestra.run.smithi178.stdout: "average_load": 0, 2024-03-07T07:44:02.274 INFO:teuthology.orchestra.run.smithi178.stdout: "avg_session_uptime": 2989, 2024-03-07T07:44:02.275 INFO:teuthology.orchestra.run.smithi178.stdout: "mdthresh_evicted": 0 2024-03-07T07:44:02.275 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.275 INFO:teuthology.orchestra.run.smithi178.stdout: "mempool": { 2024-03-07T07:44:02.275 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_bytes": 0, 2024-03-07T07:44:02.275 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_items": 0, 2024-03-07T07:44:02.275 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_bytes": 0, 2024-03-07T07:44:02.275 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_items": 0, 2024-03-07T07:44:02.275 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_bytes": 0, 2024-03-07T07:44:02.275 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_items": 0, 2024-03-07T07:44:02.275 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-07T07:44:02.276 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_items": 0, 2024-03-07T07:44:02.276 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-07T07:44:02.276 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_items": 0, 2024-03-07T07:44:02.276 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_bytes": 0, 2024-03-07T07:44:02.276 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_items": 0, 2024-03-07T07:44:02.276 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-07T07:44:02.276 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_items": 0, 2024-03-07T07:44:02.276 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_bytes": 0, 2024-03-07T07:44:02.277 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_items": 0, 2024-03-07T07:44:02.277 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_bytes": 0, 2024-03-07T07:44:02.277 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_items": 0, 2024-03-07T07:44:02.277 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-07T07:44:02.277 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_items": 0, 2024-03-07T07:44:02.277 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-07T07:44:02.277 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_items": 0, 2024-03-07T07:44:02.277 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_bytes": 0, 2024-03-07T07:44:02.277 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_items": 0, 2024-03-07T07:44:02.277 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_bytes": 0, 2024-03-07T07:44:02.277 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_items": 0, 2024-03-07T07:44:02.277 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-07T07:44:02.278 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_items": 0, 2024-03-07T07:44:02.278 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_bytes": 0, 2024-03-07T07:44:02.278 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_items": 0, 2024-03-07T07:44:02.278 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_bytes": 0, 2024-03-07T07:44:02.278 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_items": 0, 2024-03-07T07:44:02.278 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_bytes": 0, 2024-03-07T07:44:02.279 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_items": 0, 2024-03-07T07:44:02.279 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_bytes": 0, 2024-03-07T07:44:02.279 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_items": 0, 2024-03-07T07:44:02.279 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_bytes": 189319, 2024-03-07T07:44:02.279 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_items": 50, 2024-03-07T07:44:02.279 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_bytes": 0, 2024-03-07T07:44:02.279 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_items": 0, 2024-03-07T07:44:02.279 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_bytes": 0, 2024-03-07T07:44:02.279 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_items": 0, 2024-03-07T07:44:02.279 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_bytes": 0, 2024-03-07T07:44:02.279 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_items": 0, 2024-03-07T07:44:02.279 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_bytes": 0, 2024-03-07T07:44:02.280 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_items": 0, 2024-03-07T07:44:02.280 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_bytes": 7624, 2024-03-07T07:44:02.280 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_items": 160, 2024-03-07T07:44:02.280 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_bytes": 0, 2024-03-07T07:44:02.280 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_items": 0, 2024-03-07T07:44:02.280 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_bytes": 0, 2024-03-07T07:44:02.280 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_items": 0, 2024-03-07T07:44:02.280 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_bytes": 56034, 2024-03-07T07:44:02.280 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_items": 269, 2024-03-07T07:44:02.280 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_bytes": 0, 2024-03-07T07:44:02.280 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_items": 0, 2024-03-07T07:44:02.281 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_bytes": 0, 2024-03-07T07:44:02.281 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_items": 0 2024-03-07T07:44:02.281 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.281 INFO:teuthology.orchestra.run.smithi178.stdout: "objecter": { 2024-03-07T07:44:02.281 INFO:teuthology.orchestra.run.smithi178.stdout: "op_active": 0, 2024-03-07T07:44:02.281 INFO:teuthology.orchestra.run.smithi178.stdout: "op_laggy": 0, 2024-03-07T07:44:02.281 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send": 917, 2024-03-07T07:44:02.281 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send_bytes": 26, 2024-03-07T07:44:02.282 INFO:teuthology.orchestra.run.smithi178.stdout: "op_resend": 4, 2024-03-07T07:44:02.282 INFO:teuthology.orchestra.run.smithi178.stdout: "op_reply": 917, 2024-03-07T07:44:02.282 INFO:teuthology.orchestra.run.smithi178.stdout: "op_latency": { 2024-03-07T07:44:02.282 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 917, 2024-03-07T07:44:02.282 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.774986061, 2024-03-07T07:44:02.282 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000845132 2024-03-07T07:44:02.282 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.282 INFO:teuthology.orchestra.run.smithi178.stdout: "op_inflight": 0, 2024-03-07T07:44:02.282 INFO:teuthology.orchestra.run.smithi178.stdout: "oplen_avg": { 2024-03-07T07:44:02.282 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 917, 2024-03-07T07:44:02.283 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 918 2024-03-07T07:44:02.283 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.283 INFO:teuthology.orchestra.run.smithi178.stdout: "op": 917, 2024-03-07T07:44:02.283 INFO:teuthology.orchestra.run.smithi178.stdout: "op_r": 917, 2024-03-07T07:44:02.283 INFO:teuthology.orchestra.run.smithi178.stdout: "op_w": 0, 2024-03-07T07:44:02.283 INFO:teuthology.orchestra.run.smithi178.stdout: "op_rmw": 0, 2024-03-07T07:44:02.283 INFO:teuthology.orchestra.run.smithi178.stdout: "op_pg": 0, 2024-03-07T07:44:02.283 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_stat": 545, 2024-03-07T07:44:02.283 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_create": 0, 2024-03-07T07:44:02.283 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_read": 369, 2024-03-07T07:44:02.283 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_write": 0, 2024-03-07T07:44:02.283 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writefull": 0, 2024-03-07T07:44:02.284 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writesame": 0, 2024-03-07T07:44:02.284 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_append": 0, 2024-03-07T07:44:02.284 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_zero": 0, 2024-03-07T07:44:02.284 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_truncate": 0, 2024-03-07T07:44:02.284 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_delete": 0, 2024-03-07T07:44:02.284 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_mapext": 0, 2024-03-07T07:44:02.284 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_sparse_read": 0, 2024-03-07T07:44:02.284 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_clonerange": 0, 2024-03-07T07:44:02.284 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_getxattr": 2, 2024-03-07T07:44:02.284 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_setxattr": 0, 2024-03-07T07:44:02.284 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_cmpxattr": 0, 2024-03-07T07:44:02.285 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_rmxattr": 0, 2024-03-07T07:44:02.285 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_resetxattrs": 0, 2024-03-07T07:44:02.285 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_call": 0, 2024-03-07T07:44:02.285 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_watch": 0, 2024-03-07T07:44:02.285 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_notify": 0, 2024-03-07T07:44:02.285 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_src_cmpxattr": 0, 2024-03-07T07:44:02.285 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls": 0, 2024-03-07T07:44:02.285 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls_filter": 0, 2024-03-07T07:44:02.285 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_other": 0, 2024-03-07T07:44:02.285 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_active": 0, 2024-03-07T07:44:02.285 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_send": 0, 2024-03-07T07:44:02.290 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_resend": 0, 2024-03-07T07:44:02.290 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_ping": 0, 2024-03-07T07:44:02.290 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_active": 0, 2024-03-07T07:44:02.290 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_send": 0, 2024-03-07T07:44:02.290 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_resend": 0, 2024-03-07T07:44:02.290 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_active": 0, 2024-03-07T07:44:02.290 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_send": 0, 2024-03-07T07:44:02.290 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_resend": 0, 2024-03-07T07:44:02.291 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_active": 0, 2024-03-07T07:44:02.291 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_send": 0, 2024-03-07T07:44:02.291 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_resend": 0, 2024-03-07T07:44:02.291 INFO:teuthology.orchestra.run.smithi178.stdout: "command_active": 0, 2024-03-07T07:44:02.291 INFO:teuthology.orchestra.run.smithi178.stdout: "command_send": 0, 2024-03-07T07:44:02.291 INFO:teuthology.orchestra.run.smithi178.stdout: "command_resend": 0, 2024-03-07T07:44:02.291 INFO:teuthology.orchestra.run.smithi178.stdout: "map_epoch": 87, 2024-03-07T07:44:02.291 INFO:teuthology.orchestra.run.smithi178.stdout: "map_full": 0, 2024-03-07T07:44:02.291 INFO:teuthology.orchestra.run.smithi178.stdout: "map_inc": 2, 2024-03-07T07:44:02.291 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_sessions": 5, 2024-03-07T07:44:02.291 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_open": 5, 2024-03-07T07:44:02.291 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_close": 0, 2024-03-07T07:44:02.292 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_laggy": 0, 2024-03-07T07:44:02.292 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_wr": 0, 2024-03-07T07:44:02.292 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_rd": 2, 2024-03-07T07:44:02.292 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_del": 0 2024-03-07T07:44:02.292 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.292 INFO:teuthology.orchestra.run.smithi178.stdout: "oft": { 2024-03-07T07:44:02.292 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_objs": 0, 2024-03-07T07:44:02.292 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_kv_pairs": 0, 2024-03-07T07:44:02.292 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_updates": 0, 2024-03-07T07:44:02.292 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_removes": 0 2024-03-07T07:44:02.292 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.292 INFO:teuthology.orchestra.run.smithi178.stdout: "osd-slow-ops": { 2024-03-07T07:44:02.293 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_ops_count": 0 2024-03-07T07:44:02.293 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.293 INFO:teuthology.orchestra.run.smithi178.stdout: "purge_queue": { 2024-03-07T07:44:02.293 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops": 0, 2024-03-07T07:44:02.293 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops_high_water": 0, 2024-03-07T07:44:02.293 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing": 0, 2024-03-07T07:44:02.293 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_high_water": 0, 2024-03-07T07:44:02.293 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed_ops": 0, 2024-03-07T07:44:02.293 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed": 0, 2024-03-07T07:44:02.294 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_item_in_journal": 0 2024-03-07T07:44:02.294 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.294 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-07T07:44:02.294 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-03-07T07:44:02.294 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-03-07T07:44:02.294 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-03-07T07:44:02.294 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 1020, 2024-03-07T07:44:02.294 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 1521525, 2024-03-07T07:44:02.294 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-03-07T07:44:02.294 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 1020, 2024-03-07T07:44:02.294 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-03-07T07:44:02.294 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-03-07T07:44:02.294 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 1020, 2024-03-07T07:44:02.294 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 1521525, 2024-03-07T07:44:02.295 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-03-07T07:44:02.295 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:02.295 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:02.295 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:02.295 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-03-07T07:44:02.295 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.295 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_bytes": { 2024-03-07T07:44:02.295 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-03-07T07:44:02.295 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-03-07T07:44:02.295 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-03-07T07:44:02.295 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-03-07T07:44:02.295 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-03-07T07:44:02.295 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-03-07T07:44:02.295 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-03-07T07:44:02.296 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 917, 2024-03-07T07:44:02.296 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 631101, 2024-03-07T07:44:02.296 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 917, 2024-03-07T07:44:02.296 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 631101, 2024-03-07T07:44:02.296 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-03-07T07:44:02.296 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:02.296 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:02.296 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:02.296 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-03-07T07:44:02.296 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.296 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_ops": { 2024-03-07T07:44:02.297 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-03-07T07:44:02.297 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 1024, 2024-03-07T07:44:02.297 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-03-07T07:44:02.297 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-03-07T07:44:02.297 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-03-07T07:44:02.297 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-03-07T07:44:02.297 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-03-07T07:44:02.297 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 917, 2024-03-07T07:44:02.297 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 917, 2024-03-07T07:44:02.297 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 917, 2024-03-07T07:44:02.297 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 917, 2024-03-07T07:44:02.297 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-03-07T07:44:02.297 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:02.297 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:02.297 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:02.298 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-03-07T07:44:02.298 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.298 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle": { 2024-03-07T07:44:02.298 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-03-07T07:44:02.298 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-03-07T07:44:02.298 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-03-07T07:44:02.298 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-03-07T07:44:02.298 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-03-07T07:44:02.298 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-03-07T07:44:02.298 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-03-07T07:44:02.298 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-03-07T07:44:02.298 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-03-07T07:44:02.298 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 0, 2024-03-07T07:44:02.298 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 0, 2024-03-07T07:44:02.299 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-03-07T07:44:02.299 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:02.299 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:02.299 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:02.299 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-03-07T07:44:02.299 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:02.299 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle-0x56412d580960": { 2024-03-07T07:44:02.299 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-03-07T07:44:02.299 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-03-07T07:44:02.299 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-03-07T07:44:02.299 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-03-07T07:44:02.299 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-03-07T07:44:02.299 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-03-07T07:44:02.299 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-03-07T07:44:02.300 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-03-07T07:44:02.300 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-03-07T07:44:02.300 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 0, 2024-03-07T07:44:02.300 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 0, 2024-03-07T07:44:02.300 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-03-07T07:44:02.300 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:02.300 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:02.300 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:02.300 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-03-07T07:44:02.300 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-03-07T07:44:02.300 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-03-07T07:44:02.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:02 smithi178 ceph-mon[36650]: pgmap v434: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-07T07:44:02.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:02 smithi177 ceph-mon[36578]: pgmap v434: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-07T07:44:02.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:02 smithi160 ceph-mon[31963]: pgmap v434: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-07T07:44:02.984 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-03-07T07:44:02.984 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-03-07T07:44:02.984 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-03-07T07:44:02.984 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-03-07T07:44:02.985 DEBUG:tasks.check_counter:Getting stats from i 2024-03-07T07:44:02.985 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph daemon mds.i perf dump 2024-03-07T07:44:03.240 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.c/config 2024-03-07T07:44:04.971 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-03-07T07:44:04.971 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-0": { 2024-03-07T07:44:04.971 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 733, 2024-03-07T07:44:04.971 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 637, 2024-03-07T07:44:04.971 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 216638, 2024-03-07T07:44:04.971 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 943297, 2024-03-07T07:44:04.971 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 18, 2024-03-07T07:44:04.971 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 3, 2024-03-07T07:44:04.971 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.052178937, 2024-03-07T07:44:04.971 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.015867523, 2024-03-07T07:44:04.972 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.017652284, 2024-03-07T07:44:04.972 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.014362537, 2024-03-07T07:44:04.972 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-03-07T07:44:04.972 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 638, 2024-03-07T07:44:04.972 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.250749795, 2024-03-07T07:44:04.972 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000393024 2024-03-07T07:44:04.972 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:04.972 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-03-07T07:44:04.972 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 519, 2024-03-07T07:44:04.972 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000094402, 2024-03-07T07:44:04.972 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000181 2024-03-07T07:44:04.972 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:04.972 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 41168, 2024-03-07T07:44:04.972 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 1456 2024-03-07T07:44:04.973 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:04.973 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-1": { 2024-03-07T07:44:04.973 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 612, 2024-03-07T07:44:04.973 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 589, 2024-03-07T07:44:04.973 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 913333, 2024-03-07T07:44:04.973 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 353910, 2024-03-07T07:44:04.973 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 19, 2024-03-07T07:44:04.973 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 5, 2024-03-07T07:44:04.973 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.050597360, 2024-03-07T07:44:04.973 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.015627370, 2024-03-07T07:44:04.973 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.030779430, 2024-03-07T07:44:04.973 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.009207887, 2024-03-07T07:44:04.973 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-03-07T07:44:04.974 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 592, 2024-03-07T07:44:04.974 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.156806122, 2024-03-07T07:44:04.974 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000264875 2024-03-07T07:44:04.974 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:04.974 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-03-07T07:44:04.974 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 428, 2024-03-07T07:44:04.974 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000065989, 2024-03-07T07:44:04.974 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000154 2024-03-07T07:44:04.974 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:04.974 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 750160, 2024-03-07T07:44:04.974 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 56592 2024-03-07T07:44:04.974 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:04.974 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-2": { 2024-03-07T07:44:04.974 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 657, 2024-03-07T07:44:04.975 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 519, 2024-03-07T07:44:04.975 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 238358, 2024-03-07T07:44:04.975 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 441852, 2024-03-07T07:44:04.975 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 15, 2024-03-07T07:44:04.975 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 8, 2024-03-07T07:44:04.975 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.049227207, 2024-03-07T07:44:04.975 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.013206470, 2024-03-07T07:44:04.975 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.021745293, 2024-03-07T07:44:04.975 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.014390034, 2024-03-07T07:44:04.975 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-03-07T07:44:04.975 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 511, 2024-03-07T07:44:04.975 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.726935335, 2024-03-07T07:44:04.975 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.001422574 2024-03-07T07:44:04.975 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:04.975 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-03-07T07:44:04.976 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 610, 2024-03-07T07:44:04.976 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000078885, 2024-03-07T07:44:04.976 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000129 2024-03-07T07:44:04.976 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:04.976 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 174192, 2024-03-07T07:44:04.976 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 286176 2024-03-07T07:44:04.976 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:04.976 INFO:teuthology.orchestra.run.smithi178.stdout: "cct": { 2024-03-07T07:44:04.976 INFO:teuthology.orchestra.run.smithi178.stdout: "total_workers": 1, 2024-03-07T07:44:04.976 INFO:teuthology.orchestra.run.smithi178.stdout: "unhealthy_workers": 0 2024-03-07T07:44:04.976 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:04.976 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-MDSRank": { 2024-03-07T07:44:04.976 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-03-07T07:44:04.977 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-03-07T07:44:04.977 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 428, 2024-03-07T07:44:04.977 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.118501336, 2024-03-07T07:44:04.977 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000276872 2024-03-07T07:44:04.977 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-03-07T07:44:04.977 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:04.977 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-PurgeQueue": { 2024-03-07T07:44:04.977 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-03-07T07:44:04.977 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-03-07T07:44:04.978 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 83, 2024-03-07T07:44:04.978 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.002569192, 2024-03-07T07:44:04.978 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000030954 2024-03-07T07:44:04.978 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-03-07T07:44:04.978 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:04.978 INFO:teuthology.orchestra.run.smithi178.stdout: "mds": { 2024-03-07T07:44:04.978 INFO:teuthology.orchestra.run.smithi178.stdout: "request": 52, 2024-03-07T07:44:04.978 INFO:teuthology.orchestra.run.smithi178.stdout: "reply": 52, 2024-03-07T07:44:04.978 INFO:teuthology.orchestra.run.smithi178.stdout: "reply_latency": { 2024-03-07T07:44:04.978 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 52, 2024-03-07T07:44:04.978 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.068465620, 2024-03-07T07:44:04.978 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.001316646 2024-03-07T07:44:04.978 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:04.978 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_reply": 0, 2024-03-07T07:44:04.979 INFO:teuthology.orchestra.run.smithi178.stdout: "forward": 0, 2024-03-07T07:44:04.979 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_complete": 0, 2024-03-07T07:44:04.979 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_keys": 0, 2024-03-07T07:44:04.979 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_commit": 60, 2024-03-07T07:44:04.979 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_split": 0, 2024-03-07T07:44:04.979 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_merge": 0, 2024-03-07T07:44:04.979 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes": 23, 2024-03-07T07:44:04.979 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_top": 4, 2024-03-07T07:44:04.979 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_bottom": 19, 2024-03-07T07:44:04.979 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pin_tail": 0, 2024-03-07T07:44:04.979 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pinned": 16, 2024-03-07T07:44:04.980 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_expired": 0, 2024-03-07T07:44:04.980 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_with_caps": 1, 2024-03-07T07:44:04.980 INFO:teuthology.orchestra.run.smithi178.stdout: "caps": 1, 2024-03-07T07:44:04.980 INFO:teuthology.orchestra.run.smithi178.stdout: "subtrees": 2, 2024-03-07T07:44:04.980 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse": 69, 2024-03-07T07:44:04.980 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_hit": 62, 2024-03-07T07:44:04.980 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_forward": 0, 2024-03-07T07:44:04.980 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_discover": 0, 2024-03-07T07:44:04.980 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_dir_fetch": 0, 2024-03-07T07:44:04.980 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_remote_ino": 0, 2024-03-07T07:44:04.980 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_lock": 0, 2024-03-07T07:44:04.980 INFO:teuthology.orchestra.run.smithi178.stdout: "load_cent": 0, 2024-03-07T07:44:04.980 INFO:teuthology.orchestra.run.smithi178.stdout: "q": 0, 2024-03-07T07:44:04.980 INFO:teuthology.orchestra.run.smithi178.stdout: "exported": 0, 2024-03-07T07:44:04.981 INFO:teuthology.orchestra.run.smithi178.stdout: "exported_inodes": 0, 2024-03-07T07:44:04.981 INFO:teuthology.orchestra.run.smithi178.stdout: "imported": 0, 2024-03-07T07:44:04.981 INFO:teuthology.orchestra.run.smithi178.stdout: "imported_inodes": 0, 2024-03-07T07:44:04.981 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_dir_fetch": 0, 2024-03-07T07:44:04.981 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_backtrace_fetch": 0, 2024-03-07T07:44:04.981 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_peer_discover": 0, 2024-03-07T07:44:04.981 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rfiles": 4, 2024-03-07T07:44:04.981 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rbytes": 234, 2024-03-07T07:44:04.981 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rsnaps": 0, 2024-03-07T07:44:04.981 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_fetch": 0, 2024-03-07T07:44:04.982 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_set_tag": 0, 2024-03-07T07:44:04.982 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_repaired": 0, 2024-03-07T07:44:04.982 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_inotable_repaired": 0, 2024-03-07T07:44:04.982 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_inodes": 0, 2024-03-07T07:44:04.982 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_base_inodes": 0, 2024-03-07T07:44:04.982 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dirfrag_rstats": 0, 2024-03-07T07:44:04.982 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_file_inodes": 0, 2024-03-07T07:44:04.982 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_inode_file_caps": 0, 2024-03-07T07:44:04.982 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_revoke": 14, 2024-03-07T07:44:04.982 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_grant": 32, 2024-03-07T07:44:04.982 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_trunc": 0, 2024-03-07T07:44:04.982 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-07T07:44:04.982 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-07T07:44:04.982 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps": 98, 2024-03-07T07:44:04.982 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps_dirty": 79, 2024-03-07T07:44:04.983 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_cap_release": 3, 2024-03-07T07:44:04.983 INFO:teuthology.orchestra.run.smithi178.stdout: "process_request_cap_release": 26 2024-03-07T07:44:04.983 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:04.983 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_cache": { 2024-03-07T07:44:04.983 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update": 6, 2024-03-07T07:44:04.983 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update_receipt": 0, 2024-03-07T07:44:04.983 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_try_discover": 0, 2024-03-07T07:44:04.983 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_send_discover": 0, 2024-03-07T07:44:04.983 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_handle_discover": 14, 2024-03-07T07:44:04.983 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays": 0, 2024-03-07T07:44:04.983 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_delayed": 0, 2024-03-07T07:44:04.983 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_enqueuing": 0, 2024-03-07T07:44:04.983 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_created": 6, 2024-03-07T07:44:04.983 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_enqueued": 6, 2024-03-07T07:44:04.984 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_reintegrated": 0, 2024-03-07T07:44:04.984 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_migrated": 0, 2024-03-07T07:44:04.984 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_processing": 0, 2024-03-07T07:44:04.984 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_enqueued": 0, 2024-03-07T07:44:04.984 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_prioritized": 0, 2024-03-07T07:44:04.984 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_started": 0, 2024-03-07T07:44:04.984 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_completed": 0, 2024-03-07T07:44:04.984 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_enqueue_scrub": 0, 2024-03-07T07:44:04.984 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_exportdir": 0, 2024-03-07T07:44:04.984 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_flush": 0, 2024-03-07T07:44:04.984 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragmentdir": 0, 2024-03-07T07:44:04.984 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragstats": 0, 2024-03-07T07:44:04.984 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_inodestats": 0 2024-03-07T07:44:04.985 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:04.985 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_log": { 2024-03-07T07:44:04.985 INFO:teuthology.orchestra.run.smithi178.stdout: "evlrg": 0, 2024-03-07T07:44:04.985 INFO:teuthology.orchestra.run.smithi178.stdout: "evadd": 171, 2024-03-07T07:44:04.985 INFO:teuthology.orchestra.run.smithi178.stdout: "evex": 138, 2024-03-07T07:44:04.985 INFO:teuthology.orchestra.run.smithi178.stdout: "evtrm": 138, 2024-03-07T07:44:04.985 INFO:teuthology.orchestra.run.smithi178.stdout: "ev": 33, 2024-03-07T07:44:04.985 INFO:teuthology.orchestra.run.smithi178.stdout: "evexg": 0, 2024-03-07T07:44:04.985 INFO:teuthology.orchestra.run.smithi178.stdout: "evexd": 18, 2024-03-07T07:44:04.985 INFO:teuthology.orchestra.run.smithi178.stdout: "segadd": 12, 2024-03-07T07:44:04.985 INFO:teuthology.orchestra.run.smithi178.stdout: "segex": 11, 2024-03-07T07:44:04.985 INFO:teuthology.orchestra.run.smithi178.stdout: "segtrm": 11, 2024-03-07T07:44:04.985 INFO:teuthology.orchestra.run.smithi178.stdout: "seg": 1, 2024-03-07T07:44:04.985 INFO:teuthology.orchestra.run.smithi178.stdout: "segmjr": 1, 2024-03-07T07:44:04.985 INFO:teuthology.orchestra.run.smithi178.stdout: "segexg": 0, 2024-03-07T07:44:04.986 INFO:teuthology.orchestra.run.smithi178.stdout: "segexd": 1, 2024-03-07T07:44:04.986 INFO:teuthology.orchestra.run.smithi178.stdout: "expos": 4723806, 2024-03-07T07:44:04.986 INFO:teuthology.orchestra.run.smithi178.stdout: "wrpos": 4825395, 2024-03-07T07:44:04.986 INFO:teuthology.orchestra.run.smithi178.stdout: "rdpos": 0, 2024-03-07T07:44:04.986 INFO:teuthology.orchestra.run.smithi178.stdout: "jlat": { 2024-03-07T07:44:04.986 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 141, 2024-03-07T07:44:04.986 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.223568390, 2024-03-07T07:44:04.986 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.001585591 2024-03-07T07:44:04.986 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:04.986 INFO:teuthology.orchestra.run.smithi178.stdout: "replayed": 0 2024-03-07T07:44:04.986 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:04.986 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_mem": { 2024-03-07T07:44:04.986 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 23, 2024-03-07T07:44:04.986 INFO:teuthology.orchestra.run.smithi178.stdout: "ino+": 29, 2024-03-07T07:44:04.987 INFO:teuthology.orchestra.run.smithi178.stdout: "ino-": 6, 2024-03-07T07:44:04.987 INFO:teuthology.orchestra.run.smithi178.stdout: "dir": 18, 2024-03-07T07:44:04.987 INFO:teuthology.orchestra.run.smithi178.stdout: "dir+": 20, 2024-03-07T07:44:04.987 INFO:teuthology.orchestra.run.smithi178.stdout: "dir-": 2, 2024-03-07T07:44:04.987 INFO:teuthology.orchestra.run.smithi178.stdout: "dn": 23, 2024-03-07T07:44:04.987 INFO:teuthology.orchestra.run.smithi178.stdout: "dn+": 34, 2024-03-07T07:44:04.987 INFO:teuthology.orchestra.run.smithi178.stdout: "dn-": 11, 2024-03-07T07:44:04.987 INFO:teuthology.orchestra.run.smithi178.stdout: "cap": 1, 2024-03-07T07:44:04.987 INFO:teuthology.orchestra.run.smithi178.stdout: "cap+": 19, 2024-03-07T07:44:04.987 INFO:teuthology.orchestra.run.smithi178.stdout: "cap-": 18, 2024-03-07T07:44:04.987 INFO:teuthology.orchestra.run.smithi178.stdout: "rss": 51696, 2024-03-07T07:44:04.987 INFO:teuthology.orchestra.run.smithi178.stdout: "heap": 231804 2024-03-07T07:44:04.987 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:04.987 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_server": { 2024-03-07T07:44:04.988 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_client_request": 77, 2024-03-07T07:44:04.988 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_server_request": 1, 2024-03-07T07:44:04.988 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_request": 52, 2024-03-07T07:44:04.988 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_session": 14, 2024-03-07T07:44:04.988 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_peer_request": 2, 2024-03-07T07:44:04.988 INFO:teuthology.orchestra.run.smithi178.stdout: "req_create_latency": { 2024-03-07T07:44:04.988 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 8, 2024-03-07T07:44:04.988 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.006924221, 2024-03-07T07:44:04.988 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000865527 2024-03-07T07:44:04.988 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:04.988 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getattr_latency": { 2024-03-07T07:44:04.988 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 17, 2024-03-07T07:44:04.988 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.003891004, 2024-03-07T07:44:04.988 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000228882 2024-03-07T07:44:04.989 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:04.989 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getfilelock_latency": { 2024-03-07T07:44:04.989 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:04.989 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:04.989 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:04.989 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:04.989 INFO:teuthology.orchestra.run.smithi178.stdout: "req_link_latency": { 2024-03-07T07:44:04.989 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:04.989 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:04.989 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:04.989 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:04.989 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookup_latency": { 2024-03-07T07:44:04.989 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:04.989 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:04.990 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:04.990 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:04.990 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookuphash_latency": { 2024-03-07T07:44:04.990 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:04.990 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:04.990 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:04.990 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:04.990 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupino_latency": { 2024-03-07T07:44:04.990 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:04.990 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:04.990 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:04.990 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:04.990 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupname_latency": { 2024-03-07T07:44:04.990 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:04.991 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:04.991 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:04.991 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:04.991 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupparent_latency": { 2024-03-07T07:44:04.991 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:04.991 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:04.991 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:04.991 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:04.991 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupsnap_latency": { 2024-03-07T07:44:04.991 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:04.991 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:04.991 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:04.991 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:04.991 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lssnap_latency": { 2024-03-07T07:44:04.991 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:04.992 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:04.992 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:04.992 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:04.992 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mkdir_latency": { 2024-03-07T07:44:04.992 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 8, 2024-03-07T07:44:04.992 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.010256524, 2024-03-07T07:44:04.993 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.001282065 2024-03-07T07:44:04.993 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:04.993 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mknod_latency": { 2024-03-07T07:44:04.993 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:04.993 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:04.993 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:04.993 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:04.993 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mksnap_latency": { 2024-03-07T07:44:04.993 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:04.993 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:04.993 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:04.994 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:04.994 INFO:teuthology.orchestra.run.smithi178.stdout: "req_open_latency": { 2024-03-07T07:44:04.994 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:04.994 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:04.994 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:04.994 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:04.994 INFO:teuthology.orchestra.run.smithi178.stdout: "req_readdir_latency": { 2024-03-07T07:44:04.994 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 2, 2024-03-07T07:44:04.994 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.003722776, 2024-03-07T07:44:04.994 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.001861388 2024-03-07T07:44:04.994 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:04.994 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rename_latency": { 2024-03-07T07:44:04.994 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 2, 2024-03-07T07:44:04.994 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.002760286, 2024-03-07T07:44:04.995 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.001380143 2024-03-07T07:44:04.995 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:04.995 INFO:teuthology.orchestra.run.smithi178.stdout: "req_renamesnap_latency": { 2024-03-07T07:44:04.995 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:04.995 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:04.995 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:04.995 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:04.995 INFO:teuthology.orchestra.run.smithi178.stdout: "req_snapdiff_latency": { 2024-03-07T07:44:04.995 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:04.995 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:04.996 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:04.996 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:04.996 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmdir_latency": { 2024-03-07T07:44:04.996 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 2, 2024-03-07T07:44:04.996 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.012804416, 2024-03-07T07:44:04.996 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.006402208 2024-03-07T07:44:04.996 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:04.996 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmsnap_latency": { 2024-03-07T07:44:04.996 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:04.996 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:04.996 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:04.996 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:04.996 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmxattr_latency": { 2024-03-07T07:44:04.997 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:04.997 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:04.997 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:04.997 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:04.997 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setattr_latency": { 2024-03-07T07:44:04.997 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 1, 2024-03-07T07:44:04.997 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.001364096, 2024-03-07T07:44:04.997 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.001364096 2024-03-07T07:44:04.997 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:04.997 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setdirlayout_latency": { 2024-03-07T07:44:04.997 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:04.997 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:04.997 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:04.998 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:04.998 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setfilelock_latency": { 2024-03-07T07:44:04.998 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:04.998 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:04.998 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:04.998 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:04.998 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setlayout_latency": { 2024-03-07T07:44:04.998 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:04.998 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:04.999 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:04.999 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:04.999 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setxattr_latency": { 2024-03-07T07:44:04.999 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 8, 2024-03-07T07:44:04.999 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.018576460, 2024-03-07T07:44:04.999 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.002322057 2024-03-07T07:44:04.999 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:04.999 INFO:teuthology.orchestra.run.smithi178.stdout: "req_symlink_latency": { 2024-03-07T07:44:04.999 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:04.999 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:05.000 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:05.000 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:05.000 INFO:teuthology.orchestra.run.smithi178.stdout: "req_unlink_latency": { 2024-03-07T07:44:05.000 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 4, 2024-03-07T07:44:05.000 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.008165837, 2024-03-07T07:44:05.000 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.002041459 2024-03-07T07:44:05.000 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:05.000 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_revoke_eviction": 0, 2024-03-07T07:44:05.000 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_acquisition_throttle": 0, 2024-03-07T07:44:05.000 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getvxattr_latency": { 2024-03-07T07:44:05.000 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:05.000 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:05.000 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:05.001 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-03-07T07:44:05.001 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:05.001 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_sessions": { 2024-03-07T07:44:05.001 INFO:teuthology.orchestra.run.smithi178.stdout: "session_count": 1, 2024-03-07T07:44:05.001 INFO:teuthology.orchestra.run.smithi178.stdout: "session_add": 3, 2024-03-07T07:44:05.001 INFO:teuthology.orchestra.run.smithi178.stdout: "session_remove": 2, 2024-03-07T07:44:05.001 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_open": 1, 2024-03-07T07:44:05.001 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_stale": 0, 2024-03-07T07:44:05.001 INFO:teuthology.orchestra.run.smithi178.stdout: "total_load": 13, 2024-03-07T07:44:05.001 INFO:teuthology.orchestra.run.smithi178.stdout: "average_load": 13, 2024-03-07T07:44:05.001 INFO:teuthology.orchestra.run.smithi178.stdout: "avg_session_uptime": 174, 2024-03-07T07:44:05.001 INFO:teuthology.orchestra.run.smithi178.stdout: "mdthresh_evicted": 0 2024-03-07T07:44:05.001 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:05.001 INFO:teuthology.orchestra.run.smithi178.stdout: "mempool": { 2024-03-07T07:44:05.002 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_bytes": 0, 2024-03-07T07:44:05.002 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_items": 0, 2024-03-07T07:44:05.002 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_bytes": 0, 2024-03-07T07:44:05.002 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_items": 0, 2024-03-07T07:44:05.002 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_bytes": 0, 2024-03-07T07:44:05.002 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_items": 0, 2024-03-07T07:44:05.002 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-07T07:44:05.002 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_items": 0, 2024-03-07T07:44:05.002 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-07T07:44:05.002 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_items": 0, 2024-03-07T07:44:05.002 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_bytes": 0, 2024-03-07T07:44:05.002 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_items": 0, 2024-03-07T07:44:05.002 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-07T07:44:05.003 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_items": 0, 2024-03-07T07:44:05.003 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_bytes": 0, 2024-03-07T07:44:05.003 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_items": 0, 2024-03-07T07:44:05.003 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_bytes": 0, 2024-03-07T07:44:05.003 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_items": 0, 2024-03-07T07:44:05.003 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-07T07:44:05.003 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_items": 0, 2024-03-07T07:44:05.003 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-07T07:44:05.003 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_items": 0, 2024-03-07T07:44:05.003 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_bytes": 0, 2024-03-07T07:44:05.003 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_items": 0, 2024-03-07T07:44:05.003 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_bytes": 0, 2024-03-07T07:44:05.003 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_items": 0, 2024-03-07T07:44:05.003 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-07T07:44:05.004 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_items": 0, 2024-03-07T07:44:05.004 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_bytes": 0, 2024-03-07T07:44:05.004 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_items": 0, 2024-03-07T07:44:05.004 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_bytes": 0, 2024-03-07T07:44:05.004 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_items": 0, 2024-03-07T07:44:05.004 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_bytes": 0, 2024-03-07T07:44:05.004 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_items": 0, 2024-03-07T07:44:05.004 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_bytes": 0, 2024-03-07T07:44:05.004 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_items": 0, 2024-03-07T07:44:05.004 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_bytes": 291351, 2024-03-07T07:44:05.004 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_items": 120, 2024-03-07T07:44:05.004 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_bytes": 616, 2024-03-07T07:44:05.004 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_items": 7, 2024-03-07T07:44:05.005 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_bytes": 0, 2024-03-07T07:44:05.005 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_items": 0, 2024-03-07T07:44:05.005 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_bytes": 0, 2024-03-07T07:44:05.005 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_items": 0, 2024-03-07T07:44:05.005 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_bytes": 0, 2024-03-07T07:44:05.005 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_items": 0, 2024-03-07T07:44:05.005 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_bytes": 7624, 2024-03-07T07:44:05.005 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_items": 160, 2024-03-07T07:44:05.005 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_bytes": 0, 2024-03-07T07:44:05.005 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_items": 0, 2024-03-07T07:44:05.005 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_bytes": 0, 2024-03-07T07:44:05.005 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_items": 0, 2024-03-07T07:44:05.005 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_bytes": 92898, 2024-03-07T07:44:05.005 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_items": 634, 2024-03-07T07:44:05.005 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_bytes": 0, 2024-03-07T07:44:05.006 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_items": 0, 2024-03-07T07:44:05.006 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_bytes": 0, 2024-03-07T07:44:05.006 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_items": 0 2024-03-07T07:44:05.006 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:05.006 INFO:teuthology.orchestra.run.smithi178.stdout: "objecter": { 2024-03-07T07:44:05.006 INFO:teuthology.orchestra.run.smithi178.stdout: "op_active": 0, 2024-03-07T07:44:05.006 INFO:teuthology.orchestra.run.smithi178.stdout: "op_laggy": 0, 2024-03-07T07:44:05.006 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send": 899, 2024-03-07T07:44:05.006 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send_bytes": 705579, 2024-03-07T07:44:05.006 INFO:teuthology.orchestra.run.smithi178.stdout: "op_resend": 31, 2024-03-07T07:44:05.006 INFO:teuthology.orchestra.run.smithi178.stdout: "op_reply": 899, 2024-03-07T07:44:05.006 INFO:teuthology.orchestra.run.smithi178.stdout: "op_latency": { 2024-03-07T07:44:05.006 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 899, 2024-03-07T07:44:05.006 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 18.676663968, 2024-03-07T07:44:05.007 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.020774932 2024-03-07T07:44:05.007 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:05.007 INFO:teuthology.orchestra.run.smithi178.stdout: "op_inflight": 0, 2024-03-07T07:44:05.007 INFO:teuthology.orchestra.run.smithi178.stdout: "oplen_avg": { 2024-03-07T07:44:05.007 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 899, 2024-03-07T07:44:05.007 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 1069 2024-03-07T07:44:05.007 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:05.007 INFO:teuthology.orchestra.run.smithi178.stdout: "op": 899, 2024-03-07T07:44:05.007 INFO:teuthology.orchestra.run.smithi178.stdout: "op_r": 3, 2024-03-07T07:44:05.007 INFO:teuthology.orchestra.run.smithi178.stdout: "op_w": 896, 2024-03-07T07:44:05.007 INFO:teuthology.orchestra.run.smithi178.stdout: "op_rmw": 0, 2024-03-07T07:44:05.007 INFO:teuthology.orchestra.run.smithi178.stdout: "op_pg": 0, 2024-03-07T07:44:05.008 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_stat": 40, 2024-03-07T07:44:05.008 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_create": 28, 2024-03-07T07:44:05.008 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_read": 3, 2024-03-07T07:44:05.008 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_write": 144, 2024-03-07T07:44:05.008 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writefull": 46, 2024-03-07T07:44:05.008 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writesame": 0, 2024-03-07T07:44:05.008 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_append": 0, 2024-03-07T07:44:05.008 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_zero": 0, 2024-03-07T07:44:05.008 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_truncate": 0, 2024-03-07T07:44:05.008 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_delete": 581, 2024-03-07T07:44:05.008 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_mapext": 0, 2024-03-07T07:44:05.008 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_sparse_read": 0, 2024-03-07T07:44:05.009 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_clonerange": 0, 2024-03-07T07:44:05.009 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_getxattr": 0, 2024-03-07T07:44:05.009 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_setxattr": 56, 2024-03-07T07:44:05.009 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_cmpxattr": 0, 2024-03-07T07:44:05.009 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_rmxattr": 0, 2024-03-07T07:44:05.009 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_resetxattrs": 0, 2024-03-07T07:44:05.009 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_call": 0, 2024-03-07T07:44:05.009 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_watch": 0, 2024-03-07T07:44:05.009 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_notify": 0, 2024-03-07T07:44:05.009 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_src_cmpxattr": 0, 2024-03-07T07:44:05.009 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls": 0, 2024-03-07T07:44:05.009 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls_filter": 0, 2024-03-07T07:44:05.009 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_other": 0, 2024-03-07T07:44:05.009 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_active": 0, 2024-03-07T07:44:05.010 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_send": 0, 2024-03-07T07:44:05.010 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_resend": 0, 2024-03-07T07:44:05.010 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_ping": 0, 2024-03-07T07:44:05.010 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_active": 0, 2024-03-07T07:44:05.010 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_send": 0, 2024-03-07T07:44:05.010 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_resend": 0, 2024-03-07T07:44:05.010 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_active": 0, 2024-03-07T07:44:05.010 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_send": 0, 2024-03-07T07:44:05.010 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_resend": 0, 2024-03-07T07:44:05.010 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_active": 0, 2024-03-07T07:44:05.010 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_send": 0, 2024-03-07T07:44:05.010 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_resend": 0, 2024-03-07T07:44:05.010 INFO:teuthology.orchestra.run.smithi178.stdout: "command_active": 0, 2024-03-07T07:44:05.010 INFO:teuthology.orchestra.run.smithi178.stdout: "command_send": 0, 2024-03-07T07:44:05.011 INFO:teuthology.orchestra.run.smithi178.stdout: "command_resend": 0, 2024-03-07T07:44:05.011 INFO:teuthology.orchestra.run.smithi178.stdout: "map_epoch": 87, 2024-03-07T07:44:05.011 INFO:teuthology.orchestra.run.smithi178.stdout: "map_full": 0, 2024-03-07T07:44:05.011 INFO:teuthology.orchestra.run.smithi178.stdout: "map_inc": 3, 2024-03-07T07:44:05.011 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_sessions": 12, 2024-03-07T07:44:05.011 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_open": 12, 2024-03-07T07:44:05.011 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_close": 0, 2024-03-07T07:44:05.011 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_laggy": 0, 2024-03-07T07:44:05.011 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_wr": 158, 2024-03-07T07:44:05.011 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_rd": 0, 2024-03-07T07:44:05.011 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_del": 13 2024-03-07T07:44:05.011 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:05.011 INFO:teuthology.orchestra.run.smithi178.stdout: "oft": { 2024-03-07T07:44:05.011 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_objs": 1, 2024-03-07T07:44:05.012 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_kv_pairs": 0, 2024-03-07T07:44:05.012 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_updates": 0, 2024-03-07T07:44:05.012 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_removes": 0 2024-03-07T07:44:05.012 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:05.012 INFO:teuthology.orchestra.run.smithi178.stdout: "osd-slow-ops": { 2024-03-07T07:44:05.012 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_ops_count": 0 2024-03-07T07:44:05.012 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:05.012 INFO:teuthology.orchestra.run.smithi178.stdout: "purge_queue": { 2024-03-07T07:44:05.012 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops": 0, 2024-03-07T07:44:05.012 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops_high_water": 64, 2024-03-07T07:44:05.012 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing": 0, 2024-03-07T07:44:05.012 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_high_water": 4, 2024-03-07T07:44:05.012 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed_ops": 69, 2024-03-07T07:44:05.012 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed": 6, 2024-03-07T07:44:05.013 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_item_in_journal": 0 2024-03-07T07:44:05.013 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:05.013 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-07T07:44:05.013 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-03-07T07:44:05.013 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-03-07T07:44:05.013 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-03-07T07:44:05.013 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 1999, 2024-03-07T07:44:05.013 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 1155871, 2024-03-07T07:44:05.013 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-03-07T07:44:05.013 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 1999, 2024-03-07T07:44:05.013 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-03-07T07:44:05.013 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-03-07T07:44:05.013 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 1999, 2024-03-07T07:44:05.013 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 1155871, 2024-03-07T07:44:05.013 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-03-07T07:44:05.014 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:05.014 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:05.014 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:05.014 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-03-07T07:44:05.014 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:05.014 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_bytes": { 2024-03-07T07:44:05.014 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-03-07T07:44:05.014 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-03-07T07:44:05.014 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-03-07T07:44:05.014 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-03-07T07:44:05.014 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-03-07T07:44:05.014 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-03-07T07:44:05.014 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-03-07T07:44:05.015 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 899, 2024-03-07T07:44:05.015 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 706185, 2024-03-07T07:44:05.015 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 899, 2024-03-07T07:44:05.015 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 706185, 2024-03-07T07:44:05.015 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-03-07T07:44:05.015 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:05.015 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:05.015 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:05.015 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-03-07T07:44:05.015 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:05.015 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_ops": { 2024-03-07T07:44:05.015 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-03-07T07:44:05.015 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 1024, 2024-03-07T07:44:05.015 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-03-07T07:44:05.015 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-03-07T07:44:05.016 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-03-07T07:44:05.016 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-03-07T07:44:05.016 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-03-07T07:44:05.016 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 899, 2024-03-07T07:44:05.016 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 899, 2024-03-07T07:44:05.016 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 899, 2024-03-07T07:44:05.016 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 899, 2024-03-07T07:44:05.016 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-03-07T07:44:05.016 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:05.016 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:05.016 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:05.017 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-03-07T07:44:05.017 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:05.017 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle": { 2024-03-07T07:44:05.017 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-03-07T07:44:05.017 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-03-07T07:44:05.017 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-03-07T07:44:05.017 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 6, 2024-03-07T07:44:05.017 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 606, 2024-03-07T07:44:05.017 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-03-07T07:44:05.017 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 6, 2024-03-07T07:44:05.017 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-03-07T07:44:05.017 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-03-07T07:44:05.017 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 3, 2024-03-07T07:44:05.018 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 606, 2024-03-07T07:44:05.018 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-03-07T07:44:05.018 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:05.018 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:05.018 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:05.018 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-03-07T07:44:05.018 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:05.018 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle-0x5647943a46e0": { 2024-03-07T07:44:05.018 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-03-07T07:44:05.018 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-03-07T07:44:05.018 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-03-07T07:44:05.018 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 171, 2024-03-07T07:44:05.018 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 631091, 2024-03-07T07:44:05.018 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-03-07T07:44:05.019 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 171, 2024-03-07T07:44:05.019 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-03-07T07:44:05.019 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-03-07T07:44:05.019 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 141, 2024-03-07T07:44:05.019 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 631091, 2024-03-07T07:44:05.019 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-03-07T07:44:05.019 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:05.019 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:05.019 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:05.019 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-03-07T07:44:05.019 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-03-07T07:44:05.019 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-03-07T07:44:05.678 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:05 smithi160 ceph-mon[31963]: pgmap v435: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-07T07:44:05.767 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:05 smithi178 ceph-mon[36650]: pgmap v435: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-07T07:44:05.794 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-03-07T07:44:05.794 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-03-07T07:44:05.794 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=6 2024-03-07T07:44:05.794 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=0 2024-03-07T07:44:05.794 DEBUG:tasks.check_counter:Getting stats from l 2024-03-07T07:44:05.795 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0 shell --fsid 826f6582-dc54-11ee-95c6-87774f69a715 -- ceph daemon mds.l perf dump 2024-03-07T07:44:05.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:05 smithi177 ceph-mon[36578]: pgmap v435: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-07T07:44:06.066 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/mon.c/config 2024-03-07T07:44:07.538 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:07 smithi178 ceph-mon[36650]: pgmap v436: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-07T07:44:07.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:07 smithi160 ceph-mon[31963]: pgmap v436: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-07T07:44:07.828 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-03-07T07:44:07.828 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-0": { 2024-03-07T07:44:07.828 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 301, 2024-03-07T07:44:07.828 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 301, 2024-03-07T07:44:07.828 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 83405, 2024-03-07T07:44:07.828 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 94958, 2024-03-07T07:44:07.828 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 6, 2024-03-07T07:44:07.828 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 2, 2024-03-07T07:44:07.829 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.018812234, 2024-03-07T07:44:07.829 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.005310745, 2024-03-07T07:44:07.829 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.006043105, 2024-03-07T07:44:07.829 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.005237390, 2024-03-07T07:44:07.829 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-03-07T07:44:07.829 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 300, 2024-03-07T07:44:07.829 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.008666936, 2024-03-07T07:44:07.829 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000028889 2024-03-07T07:44:07.829 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.829 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-03-07T07:44:07.829 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:07.829 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:07.829 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:07.829 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.830 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-03-07T07:44:07.830 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 448 2024-03-07T07:44:07.830 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.830 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-1": { 2024-03-07T07:44:07.830 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 316, 2024-03-07T07:44:07.830 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 310, 2024-03-07T07:44:07.830 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 783000, 2024-03-07T07:44:07.830 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 122335, 2024-03-07T07:44:07.830 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 4, 2024-03-07T07:44:07.830 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 3, 2024-03-07T07:44:07.830 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.030165766, 2024-03-07T07:44:07.830 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.007558178, 2024-03-07T07:44:07.830 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.020004408, 2024-03-07T07:44:07.830 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.006837799, 2024-03-07T07:44:07.831 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-03-07T07:44:07.831 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 309, 2024-03-07T07:44:07.831 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.008545930, 2024-03-07T07:44:07.831 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000027656 2024-03-07T07:44:07.831 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.831 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-03-07T07:44:07.831 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:07.831 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:07.831 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:07.831 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.831 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 731072, 2024-03-07T07:44:07.831 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 55232 2024-03-07T07:44:07.831 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.831 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-2": { 2024-03-07T07:44:07.832 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 290, 2024-03-07T07:44:07.832 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 359, 2024-03-07T07:44:07.832 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 97264, 2024-03-07T07:44:07.832 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 283662, 2024-03-07T07:44:07.832 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 4, 2024-03-07T07:44:07.832 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 3, 2024-03-07T07:44:07.832 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.023319464, 2024-03-07T07:44:07.832 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.008275116, 2024-03-07T07:44:07.832 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.006074468, 2024-03-07T07:44:07.832 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.006639033, 2024-03-07T07:44:07.832 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-03-07T07:44:07.832 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 357, 2024-03-07T07:44:07.832 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.010942316, 2024-03-07T07:44:07.832 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000030650 2024-03-07T07:44:07.833 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.833 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-03-07T07:44:07.833 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:07.833 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:07.833 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:07.833 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.833 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-07T07:44:07.833 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 191760 2024-03-07T07:44:07.833 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.833 INFO:teuthology.orchestra.run.smithi178.stdout: "cct": { 2024-03-07T07:44:07.833 INFO:teuthology.orchestra.run.smithi178.stdout: "total_workers": 1, 2024-03-07T07:44:07.833 INFO:teuthology.orchestra.run.smithi178.stdout: "unhealthy_workers": 0 2024-03-07T07:44:07.833 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.833 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-MDSRank": { 2024-03-07T07:44:07.834 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-03-07T07:44:07.834 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-03-07T07:44:07.834 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 1350, 2024-03-07T07:44:07.834 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.026008682, 2024-03-07T07:44:07.834 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000019265 2024-03-07T07:44:07.834 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-03-07T07:44:07.834 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.834 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-PurgeQueue": { 2024-03-07T07:44:07.834 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-03-07T07:44:07.834 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-03-07T07:44:07.834 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:07.834 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:07.834 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:07.834 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-03-07T07:44:07.835 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.835 INFO:teuthology.orchestra.run.smithi178.stdout: "mds": { 2024-03-07T07:44:07.835 INFO:teuthology.orchestra.run.smithi178.stdout: "request": 0, 2024-03-07T07:44:07.835 INFO:teuthology.orchestra.run.smithi178.stdout: "reply": 0, 2024-03-07T07:44:07.835 INFO:teuthology.orchestra.run.smithi178.stdout: "reply_latency": { 2024-03-07T07:44:07.835 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:07.835 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:07.835 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:07.835 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.835 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_reply": 0, 2024-03-07T07:44:07.835 INFO:teuthology.orchestra.run.smithi178.stdout: "forward": 0, 2024-03-07T07:44:07.835 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_complete": 0, 2024-03-07T07:44:07.835 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_keys": 0, 2024-03-07T07:44:07.836 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_commit": 0, 2024-03-07T07:44:07.836 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_split": 0, 2024-03-07T07:44:07.836 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_merge": 0, 2024-03-07T07:44:07.836 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes": 0, 2024-03-07T07:44:07.836 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_top": 0, 2024-03-07T07:44:07.836 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_bottom": 0, 2024-03-07T07:44:07.836 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pin_tail": 0, 2024-03-07T07:44:07.836 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pinned": 0, 2024-03-07T07:44:07.836 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_expired": 0, 2024-03-07T07:44:07.836 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_with_caps": 0, 2024-03-07T07:44:07.836 INFO:teuthology.orchestra.run.smithi178.stdout: "caps": 0, 2024-03-07T07:44:07.836 INFO:teuthology.orchestra.run.smithi178.stdout: "subtrees": 1, 2024-03-07T07:44:07.836 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse": 0, 2024-03-07T07:44:07.836 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_hit": 0, 2024-03-07T07:44:07.837 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_forward": 0, 2024-03-07T07:44:07.837 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_discover": 0, 2024-03-07T07:44:07.837 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_dir_fetch": 0, 2024-03-07T07:44:07.837 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_remote_ino": 0, 2024-03-07T07:44:07.837 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_lock": 0, 2024-03-07T07:44:07.837 INFO:teuthology.orchestra.run.smithi178.stdout: "load_cent": 0, 2024-03-07T07:44:07.837 INFO:teuthology.orchestra.run.smithi178.stdout: "q": 0, 2024-03-07T07:44:07.837 INFO:teuthology.orchestra.run.smithi178.stdout: "exported": 0, 2024-03-07T07:44:07.837 INFO:teuthology.orchestra.run.smithi178.stdout: "exported_inodes": 0, 2024-03-07T07:44:07.837 INFO:teuthology.orchestra.run.smithi178.stdout: "imported": 0, 2024-03-07T07:44:07.837 INFO:teuthology.orchestra.run.smithi178.stdout: "imported_inodes": 0, 2024-03-07T07:44:07.837 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_dir_fetch": 0, 2024-03-07T07:44:07.837 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_backtrace_fetch": 0, 2024-03-07T07:44:07.837 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_peer_discover": 0, 2024-03-07T07:44:07.838 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rfiles": 0, 2024-03-07T07:44:07.838 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rbytes": 0, 2024-03-07T07:44:07.838 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rsnaps": 0, 2024-03-07T07:44:07.838 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_fetch": 0, 2024-03-07T07:44:07.838 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_set_tag": 0, 2024-03-07T07:44:07.838 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_repaired": 0, 2024-03-07T07:44:07.838 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_inotable_repaired": 0, 2024-03-07T07:44:07.838 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_inodes": 0, 2024-03-07T07:44:07.838 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_base_inodes": 0, 2024-03-07T07:44:07.838 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dirfrag_rstats": 0, 2024-03-07T07:44:07.838 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_file_inodes": 0, 2024-03-07T07:44:07.838 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_inode_file_caps": 0, 2024-03-07T07:44:07.838 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_revoke": 0, 2024-03-07T07:44:07.838 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_grant": 0, 2024-03-07T07:44:07.839 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_trunc": 0, 2024-03-07T07:44:07.839 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-07T07:44:07.839 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-07T07:44:07.839 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps": 0, 2024-03-07T07:44:07.839 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps_dirty": 0, 2024-03-07T07:44:07.839 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_cap_release": 0, 2024-03-07T07:44:07.839 INFO:teuthology.orchestra.run.smithi178.stdout: "process_request_cap_release": 0 2024-03-07T07:44:07.839 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.839 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_cache": { 2024-03-07T07:44:07.839 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update": 0, 2024-03-07T07:44:07.839 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update_receipt": 0, 2024-03-07T07:44:07.839 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_try_discover": 0, 2024-03-07T07:44:07.839 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_send_discover": 0, 2024-03-07T07:44:07.839 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_handle_discover": 0, 2024-03-07T07:44:07.839 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays": 0, 2024-03-07T07:44:07.840 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_delayed": 0, 2024-03-07T07:44:07.840 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_enqueuing": 0, 2024-03-07T07:44:07.840 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_created": 0, 2024-03-07T07:44:07.840 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_enqueued": 0, 2024-03-07T07:44:07.840 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_reintegrated": 0, 2024-03-07T07:44:07.840 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_migrated": 0, 2024-03-07T07:44:07.840 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_processing": 0, 2024-03-07T07:44:07.840 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_enqueued": 0, 2024-03-07T07:44:07.840 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_prioritized": 0, 2024-03-07T07:44:07.840 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_started": 0, 2024-03-07T07:44:07.840 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_completed": 0, 2024-03-07T07:44:07.840 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_enqueue_scrub": 0, 2024-03-07T07:44:07.840 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_exportdir": 0, 2024-03-07T07:44:07.841 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_flush": 0, 2024-03-07T07:44:07.841 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragmentdir": 0, 2024-03-07T07:44:07.841 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragstats": 0, 2024-03-07T07:44:07.841 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_inodestats": 0 2024-03-07T07:44:07.841 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.841 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_log": { 2024-03-07T07:44:07.841 INFO:teuthology.orchestra.run.smithi178.stdout: "evlrg": 0, 2024-03-07T07:44:07.841 INFO:teuthology.orchestra.run.smithi178.stdout: "evadd": 0, 2024-03-07T07:44:07.841 INFO:teuthology.orchestra.run.smithi178.stdout: "evex": 0, 2024-03-07T07:44:07.841 INFO:teuthology.orchestra.run.smithi178.stdout: "evtrm": 0, 2024-03-07T07:44:07.841 INFO:teuthology.orchestra.run.smithi178.stdout: "ev": 7, 2024-03-07T07:44:07.841 INFO:teuthology.orchestra.run.smithi178.stdout: "evexg": 0, 2024-03-07T07:44:07.841 INFO:teuthology.orchestra.run.smithi178.stdout: "evexd": 0, 2024-03-07T07:44:07.841 INFO:teuthology.orchestra.run.smithi178.stdout: "segadd": 0, 2024-03-07T07:44:07.842 INFO:teuthology.orchestra.run.smithi178.stdout: "segex": 0, 2024-03-07T07:44:07.842 INFO:teuthology.orchestra.run.smithi178.stdout: "segtrm": 0, 2024-03-07T07:44:07.842 INFO:teuthology.orchestra.run.smithi178.stdout: "seg": 2, 2024-03-07T07:44:07.842 INFO:teuthology.orchestra.run.smithi178.stdout: "segmjr": 2, 2024-03-07T07:44:07.842 INFO:teuthology.orchestra.run.smithi178.stdout: "segexg": 0, 2024-03-07T07:44:07.842 INFO:teuthology.orchestra.run.smithi178.stdout: "segexd": 0, 2024-03-07T07:44:07.842 INFO:teuthology.orchestra.run.smithi178.stdout: "expos": 4194304, 2024-03-07T07:44:07.842 INFO:teuthology.orchestra.run.smithi178.stdout: "wrpos": 4196264, 2024-03-07T07:44:07.842 INFO:teuthology.orchestra.run.smithi178.stdout: "rdpos": 4196123, 2024-03-07T07:44:07.842 INFO:teuthology.orchestra.run.smithi178.stdout: "jlat": { 2024-03-07T07:44:07.842 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:07.842 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:07.842 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:07.842 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.842 INFO:teuthology.orchestra.run.smithi178.stdout: "replayed": 7 2024-03-07T07:44:07.843 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.843 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_mem": { 2024-03-07T07:44:07.843 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 3, 2024-03-07T07:44:07.843 INFO:teuthology.orchestra.run.smithi178.stdout: "ino+": 3, 2024-03-07T07:44:07.843 INFO:teuthology.orchestra.run.smithi178.stdout: "ino-": 0, 2024-03-07T07:44:07.843 INFO:teuthology.orchestra.run.smithi178.stdout: "dir": 1, 2024-03-07T07:44:07.843 INFO:teuthology.orchestra.run.smithi178.stdout: "dir+": 1, 2024-03-07T07:44:07.843 INFO:teuthology.orchestra.run.smithi178.stdout: "dir-": 0, 2024-03-07T07:44:07.843 INFO:teuthology.orchestra.run.smithi178.stdout: "dn": 0, 2024-03-07T07:44:07.843 INFO:teuthology.orchestra.run.smithi178.stdout: "dn+": 0, 2024-03-07T07:44:07.843 INFO:teuthology.orchestra.run.smithi178.stdout: "dn-": 0, 2024-03-07T07:44:07.843 INFO:teuthology.orchestra.run.smithi178.stdout: "cap": 0, 2024-03-07T07:44:07.843 INFO:teuthology.orchestra.run.smithi178.stdout: "cap+": 0, 2024-03-07T07:44:07.843 INFO:teuthology.orchestra.run.smithi178.stdout: "cap-": 0, 2024-03-07T07:44:07.843 INFO:teuthology.orchestra.run.smithi178.stdout: "rss": 46064, 2024-03-07T07:44:07.844 INFO:teuthology.orchestra.run.smithi178.stdout: "heap": 215420 2024-03-07T07:44:07.844 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.844 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_server": { 2024-03-07T07:44:07.844 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_client_request": 0, 2024-03-07T07:44:07.844 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_server_request": 0, 2024-03-07T07:44:07.844 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_request": 0, 2024-03-07T07:44:07.844 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_session": 0, 2024-03-07T07:44:07.844 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_peer_request": 0, 2024-03-07T07:44:07.844 INFO:teuthology.orchestra.run.smithi178.stdout: "req_create_latency": { 2024-03-07T07:44:07.844 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:07.844 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:07.844 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:07.844 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.844 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getattr_latency": { 2024-03-07T07:44:07.845 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:07.845 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:07.845 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:07.845 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.845 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getfilelock_latency": { 2024-03-07T07:44:07.845 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:07.845 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:07.845 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:07.845 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.845 INFO:teuthology.orchestra.run.smithi178.stdout: "req_link_latency": { 2024-03-07T07:44:07.845 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:07.845 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:07.845 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:07.845 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.846 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookup_latency": { 2024-03-07T07:44:07.846 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:07.846 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:07.846 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:07.846 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.846 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookuphash_latency": { 2024-03-07T07:44:07.846 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:07.846 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:07.846 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:07.846 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.846 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupino_latency": { 2024-03-07T07:44:07.846 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:07.846 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:07.846 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:07.846 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.847 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupname_latency": { 2024-03-07T07:44:07.847 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:07.847 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:07.847 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:07.847 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.847 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupparent_latency": { 2024-03-07T07:44:07.847 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:07.847 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:07.847 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:07.847 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.847 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupsnap_latency": { 2024-03-07T07:44:07.847 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:07.847 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:07.848 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:07.848 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.848 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lssnap_latency": { 2024-03-07T07:44:07.848 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:07.848 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:07.848 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:07.848 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.848 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mkdir_latency": { 2024-03-07T07:44:07.848 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:07.848 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:07.848 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:07.848 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.849 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mknod_latency": { 2024-03-07T07:44:07.849 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:07.849 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:07.849 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:07.849 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.849 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mksnap_latency": { 2024-03-07T07:44:07.849 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:07.850 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:07.850 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:07.850 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.850 INFO:teuthology.orchestra.run.smithi178.stdout: "req_open_latency": { 2024-03-07T07:44:07.850 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:07.850 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:07.850 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:07.850 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.850 INFO:teuthology.orchestra.run.smithi178.stdout: "req_readdir_latency": { 2024-03-07T07:44:07.850 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:07.850 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:07.850 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:07.850 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.850 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rename_latency": { 2024-03-07T07:44:07.851 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:07.851 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:07.851 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:07.851 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.851 INFO:teuthology.orchestra.run.smithi178.stdout: "req_renamesnap_latency": { 2024-03-07T07:44:07.851 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:07.851 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:07.851 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:07.851 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.851 INFO:teuthology.orchestra.run.smithi178.stdout: "req_snapdiff_latency": { 2024-03-07T07:44:07.851 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:07.851 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:07.851 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:07.851 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.852 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmdir_latency": { 2024-03-07T07:44:07.852 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:07.852 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:07.852 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:07.852 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.852 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmsnap_latency": { 2024-03-07T07:44:07.852 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:07.852 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:07.852 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:07.852 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.852 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmxattr_latency": { 2024-03-07T07:44:07.852 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:07.852 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:07.852 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:07.853 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.853 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setattr_latency": { 2024-03-07T07:44:07.853 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:07.853 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:07.853 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:07.853 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.853 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setdirlayout_latency": { 2024-03-07T07:44:07.853 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:07.853 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:07.853 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:07.853 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.853 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setfilelock_latency": { 2024-03-07T07:44:07.853 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:07.853 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:07.854 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:07.854 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.854 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setlayout_latency": { 2024-03-07T07:44:07.854 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:07.854 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:07.854 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:07.854 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.854 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setxattr_latency": { 2024-03-07T07:44:07.854 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:07.854 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:07.854 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:07.854 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.854 INFO:teuthology.orchestra.run.smithi178.stdout: "req_symlink_latency": { 2024-03-07T07:44:07.854 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:07.855 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:07.855 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:07.855 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.855 INFO:teuthology.orchestra.run.smithi178.stdout: "req_unlink_latency": { 2024-03-07T07:44:07.855 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:07.855 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:07.855 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:07.855 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.855 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_revoke_eviction": 0, 2024-03-07T07:44:07.855 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_acquisition_throttle": 0, 2024-03-07T07:44:07.855 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getvxattr_latency": { 2024-03-07T07:44:07.855 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:07.855 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:07.855 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:07.855 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-03-07T07:44:07.856 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.856 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_sessions": { 2024-03-07T07:44:07.856 INFO:teuthology.orchestra.run.smithi178.stdout: "session_count": 1, 2024-03-07T07:44:07.856 INFO:teuthology.orchestra.run.smithi178.stdout: "session_add": 3, 2024-03-07T07:44:07.856 INFO:teuthology.orchestra.run.smithi178.stdout: "session_remove": 2, 2024-03-07T07:44:07.856 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_open": 2, 2024-03-07T07:44:07.856 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_stale": 0, 2024-03-07T07:44:07.856 INFO:teuthology.orchestra.run.smithi178.stdout: "total_load": 0, 2024-03-07T07:44:07.856 INFO:teuthology.orchestra.run.smithi178.stdout: "average_load": 0, 2024-03-07T07:44:07.856 INFO:teuthology.orchestra.run.smithi178.stdout: "avg_session_uptime": 2995, 2024-03-07T07:44:07.856 INFO:teuthology.orchestra.run.smithi178.stdout: "mdthresh_evicted": 0 2024-03-07T07:44:07.856 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.856 INFO:teuthology.orchestra.run.smithi178.stdout: "mempool": { 2024-03-07T07:44:07.857 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_bytes": 0, 2024-03-07T07:44:07.857 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_items": 0, 2024-03-07T07:44:07.857 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_bytes": 0, 2024-03-07T07:44:07.857 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_items": 0, 2024-03-07T07:44:07.857 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_bytes": 0, 2024-03-07T07:44:07.857 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_items": 0, 2024-03-07T07:44:07.857 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-07T07:44:07.857 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_items": 0, 2024-03-07T07:44:07.857 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-07T07:44:07.857 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_items": 0, 2024-03-07T07:44:07.857 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_bytes": 0, 2024-03-07T07:44:07.857 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_items": 0, 2024-03-07T07:44:07.857 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-07T07:44:07.857 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_items": 0, 2024-03-07T07:44:07.858 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_bytes": 0, 2024-03-07T07:44:07.858 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_items": 0, 2024-03-07T07:44:07.858 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_bytes": 0, 2024-03-07T07:44:07.858 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_items": 0, 2024-03-07T07:44:07.858 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-07T07:44:07.858 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_items": 0, 2024-03-07T07:44:07.858 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-07T07:44:07.858 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_items": 0, 2024-03-07T07:44:07.858 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_bytes": 0, 2024-03-07T07:44:07.858 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_items": 0, 2024-03-07T07:44:07.858 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_bytes": 0, 2024-03-07T07:44:07.858 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_items": 0, 2024-03-07T07:44:07.858 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-07T07:44:07.858 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_items": 0, 2024-03-07T07:44:07.858 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_bytes": 0, 2024-03-07T07:44:07.859 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_items": 0, 2024-03-07T07:44:07.859 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_bytes": 0, 2024-03-07T07:44:07.859 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_items": 0, 2024-03-07T07:44:07.859 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_bytes": 0, 2024-03-07T07:44:07.859 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_items": 0, 2024-03-07T07:44:07.859 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_bytes": 0, 2024-03-07T07:44:07.859 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_items": 0, 2024-03-07T07:44:07.859 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_bytes": 189319, 2024-03-07T07:44:07.859 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_items": 50, 2024-03-07T07:44:07.859 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_bytes": 0, 2024-03-07T07:44:07.859 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_items": 0, 2024-03-07T07:44:07.859 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_bytes": 0, 2024-03-07T07:44:07.859 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_items": 0, 2024-03-07T07:44:07.859 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_bytes": 0, 2024-03-07T07:44:07.860 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_items": 0, 2024-03-07T07:44:07.860 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_bytes": 0, 2024-03-07T07:44:07.860 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_items": 0, 2024-03-07T07:44:07.860 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_bytes": 7624, 2024-03-07T07:44:07.860 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_items": 160, 2024-03-07T07:44:07.860 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_bytes": 0, 2024-03-07T07:44:07.860 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_items": 0, 2024-03-07T07:44:07.860 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_bytes": 0, 2024-03-07T07:44:07.860 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_items": 0, 2024-03-07T07:44:07.860 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_bytes": 8320, 2024-03-07T07:44:07.860 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_items": 13, 2024-03-07T07:44:07.860 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_bytes": 0, 2024-03-07T07:44:07.860 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_items": 0, 2024-03-07T07:44:07.860 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_bytes": 0, 2024-03-07T07:44:07.861 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_items": 0 2024-03-07T07:44:07.861 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.861 INFO:teuthology.orchestra.run.smithi178.stdout: "objecter": { 2024-03-07T07:44:07.861 INFO:teuthology.orchestra.run.smithi178.stdout: "op_active": 0, 2024-03-07T07:44:07.861 INFO:teuthology.orchestra.run.smithi178.stdout: "op_laggy": 0, 2024-03-07T07:44:07.861 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send": 797, 2024-03-07T07:44:07.861 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send_bytes": 21, 2024-03-07T07:44:07.861 INFO:teuthology.orchestra.run.smithi178.stdout: "op_resend": 3, 2024-03-07T07:44:07.861 INFO:teuthology.orchestra.run.smithi178.stdout: "op_reply": 797, 2024-03-07T07:44:07.861 INFO:teuthology.orchestra.run.smithi178.stdout: "op_latency": { 2024-03-07T07:44:07.861 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 797, 2024-03-07T07:44:07.861 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.370993331, 2024-03-07T07:44:07.861 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000465487 2024-03-07T07:44:07.861 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.861 INFO:teuthology.orchestra.run.smithi178.stdout: "op_inflight": 0, 2024-03-07T07:44:07.862 INFO:teuthology.orchestra.run.smithi178.stdout: "oplen_avg": { 2024-03-07T07:44:07.862 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 797, 2024-03-07T07:44:07.862 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 798 2024-03-07T07:44:07.862 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.862 INFO:teuthology.orchestra.run.smithi178.stdout: "op": 797, 2024-03-07T07:44:07.862 INFO:teuthology.orchestra.run.smithi178.stdout: "op_r": 797, 2024-03-07T07:44:07.862 INFO:teuthology.orchestra.run.smithi178.stdout: "op_w": 0, 2024-03-07T07:44:07.862 INFO:teuthology.orchestra.run.smithi178.stdout: "op_rmw": 0, 2024-03-07T07:44:07.862 INFO:teuthology.orchestra.run.smithi178.stdout: "op_pg": 0, 2024-03-07T07:44:07.862 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_stat": 498, 2024-03-07T07:44:07.862 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_create": 0, 2024-03-07T07:44:07.862 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_read": 297, 2024-03-07T07:44:07.862 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_write": 0, 2024-03-07T07:44:07.862 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writefull": 0, 2024-03-07T07:44:07.863 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writesame": 0, 2024-03-07T07:44:07.863 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_append": 0, 2024-03-07T07:44:07.863 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_zero": 0, 2024-03-07T07:44:07.863 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_truncate": 0, 2024-03-07T07:44:07.863 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_delete": 0, 2024-03-07T07:44:07.863 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_mapext": 0, 2024-03-07T07:44:07.863 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_sparse_read": 0, 2024-03-07T07:44:07.863 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_clonerange": 0, 2024-03-07T07:44:07.863 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_getxattr": 1, 2024-03-07T07:44:07.863 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_setxattr": 0, 2024-03-07T07:44:07.863 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_cmpxattr": 0, 2024-03-07T07:44:07.863 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_rmxattr": 0, 2024-03-07T07:44:07.863 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_resetxattrs": 0, 2024-03-07T07:44:07.863 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_call": 0, 2024-03-07T07:44:07.863 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_watch": 0, 2024-03-07T07:44:07.864 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_notify": 0, 2024-03-07T07:44:07.864 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_src_cmpxattr": 0, 2024-03-07T07:44:07.864 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls": 0, 2024-03-07T07:44:07.864 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls_filter": 0, 2024-03-07T07:44:07.864 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_other": 0, 2024-03-07T07:44:07.864 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_active": 0, 2024-03-07T07:44:07.864 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_send": 0, 2024-03-07T07:44:07.864 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_resend": 0, 2024-03-07T07:44:07.864 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_ping": 0, 2024-03-07T07:44:07.864 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_active": 0, 2024-03-07T07:44:07.864 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_send": 0, 2024-03-07T07:44:07.864 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_resend": 0, 2024-03-07T07:44:07.864 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_active": 0, 2024-03-07T07:44:07.864 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_send": 0, 2024-03-07T07:44:07.865 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_resend": 0, 2024-03-07T07:44:07.865 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_active": 0, 2024-03-07T07:44:07.865 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_send": 0, 2024-03-07T07:44:07.865 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_resend": 0, 2024-03-07T07:44:07.865 INFO:teuthology.orchestra.run.smithi178.stdout: "command_active": 0, 2024-03-07T07:44:07.865 INFO:teuthology.orchestra.run.smithi178.stdout: "command_send": 0, 2024-03-07T07:44:07.865 INFO:teuthology.orchestra.run.smithi178.stdout: "command_resend": 0, 2024-03-07T07:44:07.865 INFO:teuthology.orchestra.run.smithi178.stdout: "map_epoch": 87, 2024-03-07T07:44:07.865 INFO:teuthology.orchestra.run.smithi178.stdout: "map_full": 0, 2024-03-07T07:44:07.865 INFO:teuthology.orchestra.run.smithi178.stdout: "map_inc": 2, 2024-03-07T07:44:07.865 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_sessions": 5, 2024-03-07T07:44:07.865 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_open": 5, 2024-03-07T07:44:07.865 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_close": 0, 2024-03-07T07:44:07.865 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_laggy": 0, 2024-03-07T07:44:07.866 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_wr": 0, 2024-03-07T07:44:07.866 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_rd": 2, 2024-03-07T07:44:07.866 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_del": 0 2024-03-07T07:44:07.866 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.866 INFO:teuthology.orchestra.run.smithi178.stdout: "oft": { 2024-03-07T07:44:07.866 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_objs": 0, 2024-03-07T07:44:07.866 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_kv_pairs": 0, 2024-03-07T07:44:07.866 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_updates": 0, 2024-03-07T07:44:07.866 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_removes": 0 2024-03-07T07:44:07.866 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.866 INFO:teuthology.orchestra.run.smithi178.stdout: "osd-slow-ops": { 2024-03-07T07:44:07.866 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_ops_count": 0 2024-03-07T07:44:07.866 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.866 INFO:teuthology.orchestra.run.smithi178.stdout: "purge_queue": { 2024-03-07T07:44:07.866 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops": 0, 2024-03-07T07:44:07.867 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops_high_water": 0, 2024-03-07T07:44:07.867 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing": 0, 2024-03-07T07:44:07.867 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_high_water": 0, 2024-03-07T07:44:07.867 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed_ops": 0, 2024-03-07T07:44:07.867 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed": 0, 2024-03-07T07:44:07.867 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_item_in_journal": 0 2024-03-07T07:44:07.867 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.867 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-07T07:44:07.867 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-03-07T07:44:07.867 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-03-07T07:44:07.867 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-03-07T07:44:07.867 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 903, 2024-03-07T07:44:07.867 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 873961, 2024-03-07T07:44:07.867 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-03-07T07:44:07.868 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 903, 2024-03-07T07:44:07.868 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-03-07T07:44:07.868 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-03-07T07:44:07.868 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 903, 2024-03-07T07:44:07.868 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 873961, 2024-03-07T07:44:07.868 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-03-07T07:44:07.868 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:07.868 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:07.868 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:07.868 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-03-07T07:44:07.868 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.868 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_bytes": { 2024-03-07T07:44:07.868 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-03-07T07:44:07.868 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-03-07T07:44:07.869 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-03-07T07:44:07.869 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-03-07T07:44:07.869 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-03-07T07:44:07.869 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-03-07T07:44:07.869 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-03-07T07:44:07.869 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 797, 2024-03-07T07:44:07.869 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 1965, 2024-03-07T07:44:07.869 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 797, 2024-03-07T07:44:07.869 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 1965, 2024-03-07T07:44:07.869 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-03-07T07:44:07.869 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:07.869 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:07.869 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:07.869 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-03-07T07:44:07.869 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.870 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_ops": { 2024-03-07T07:44:07.870 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-03-07T07:44:07.870 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 1024, 2024-03-07T07:44:07.870 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-03-07T07:44:07.870 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-03-07T07:44:07.870 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-03-07T07:44:07.870 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-03-07T07:44:07.870 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-03-07T07:44:07.870 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 797, 2024-03-07T07:44:07.870 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 797, 2024-03-07T07:44:07.870 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 797, 2024-03-07T07:44:07.870 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 797, 2024-03-07T07:44:07.870 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-03-07T07:44:07.870 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:07.871 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:07.871 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:07.871 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-03-07T07:44:07.871 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.871 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle": { 2024-03-07T07:44:07.871 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-03-07T07:44:07.871 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-03-07T07:44:07.871 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-03-07T07:44:07.871 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-03-07T07:44:07.871 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-03-07T07:44:07.871 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-03-07T07:44:07.871 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-03-07T07:44:07.871 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-03-07T07:44:07.871 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-03-07T07:44:07.871 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 0, 2024-03-07T07:44:07.872 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 0, 2024-03-07T07:44:07.872 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-03-07T07:44:07.872 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:07.872 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:07.872 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:07.872 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-03-07T07:44:07.872 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-03-07T07:44:07.872 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle-0x55a073da2960": { 2024-03-07T07:44:07.872 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-03-07T07:44:07.872 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-03-07T07:44:07.872 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-03-07T07:44:07.872 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-03-07T07:44:07.872 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-03-07T07:44:07.873 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-03-07T07:44:07.873 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-03-07T07:44:07.873 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-03-07T07:44:07.873 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-03-07T07:44:07.873 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 0, 2024-03-07T07:44:07.873 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 0, 2024-03-07T07:44:07.873 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-03-07T07:44:07.873 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-03-07T07:44:07.873 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-03-07T07:44:07.873 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-03-07T07:44:07.873 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-03-07T07:44:07.873 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-03-07T07:44:07.873 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-03-07T07:44:07.874 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:07 smithi177 ceph-mon[36578]: pgmap v436: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-07T07:44:08.629 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-03-07T07:44:08.629 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-03-07T07:44:08.629 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-03-07T07:44:08.630 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-03-07T07:44:08.630 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-07T07:44:08.640 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-07T07:44:08.641 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-03-07T07:44:08.641 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T07:44:08.660 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-03-07T07:44:08.660 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T07:44:08.717 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T07:44:08.783 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:08 smithi160 ceph-mon[31963]: pgmap v437: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-07T07:44:08.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:08 smithi178 ceph-mon[36650]: pgmap v437: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-07T07:44:08.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:08 smithi177 ceph-mon[36578]: pgmap v437: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-07T07:44:08.865 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:08.862+0000 7f1da420d640 1 -- 172.21.15.160:0/2146239876 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d9c102150 msgr2=0x7f1d9c1089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:08.865 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:08.862+0000 7f1da420d640 1 --2- 172.21.15.160:0/2146239876 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d9c102150 0x7f1d9c1089e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1d94009ca0 tx=0x7f1d9402c100 comp rx=0 tx=0).stop 2024-03-07T07:44:08.865 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:08.862+0000 7f1da420d640 1 -- 172.21.15.160:0/2146239876 shutdown_connections 2024-03-07T07:44:08.865 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:08.862+0000 7f1da420d640 1 --2- 172.21.15.160:0/2146239876 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d9c102150 0x7f1d9c1089e0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:08.865 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:08.862+0000 7f1da420d640 1 --2- 172.21.15.160:0/2146239876 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d9c101790 0x7f1d9c101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:08.865 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:08.862+0000 7f1da420d640 1 --2- 172.21.15.160:0/2146239876 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d9c1005a0 0x7f1d9c1009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:08.865 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:08.862+0000 7f1da420d640 1 -- 172.21.15.160:0/2146239876 >> 172.21.15.160:0/2146239876 conn(0x7f1d9c0fbb10 msgr2=0x7f1d9c0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:44:08.865 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:08.862+0000 7f1da420d640 1 -- 172.21.15.160:0/2146239876 shutdown_connections 2024-03-07T07:44:08.865 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:08.862+0000 7f1da420d640 1 -- 172.21.15.160:0/2146239876 wait complete. 2024-03-07T07:44:08.865 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:08.863+0000 7f1da420d640 1 Processor -- start 2024-03-07T07:44:08.865 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:08.863+0000 7f1da420d640 1 -- start start 2024-03-07T07:44:08.866 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:08.863+0000 7f1da420d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d9c1005a0 0x7f1d9c196860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:44:08.866 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:08.863+0000 7f1da420d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d9c101790 0x7f1d9c196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:44:08.866 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:08.863+0000 7f1da420d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d9c102150 0x7f1d9c19de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:44:08.866 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:08.863+0000 7f1da420d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c10c230 con 0x7f1d9c102150 2024-03-07T07:44:08.866 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:08.863+0000 7f1da420d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c10c0b0 con 0x7f1d9c1005a0 2024-03-07T07:44:08.866 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:08.863+0000 7f1da420d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c10c3b0 con 0x7f1d9c101790 2024-03-07T07:44:08.866 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:08.863+0000 7f1da27aa640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d9c102150 0x7f1d9c19de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:44:08.866 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:08.863+0000 7f1da17a8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d9c101790 0x7f1d9c196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:44:08.866 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:08.863+0000 7f1da1fa9640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d9c1005a0 0x7f1d9c196860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:44:08.866 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:08.863+0000 7f1da27aa640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d9c102150 0x7f1d9c19de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:46700/0 (socket says 172.21.15.160:46700) 2024-03-07T07:44:08.866 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:08.863+0000 7f1da17a8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d9c101790 0x7f1d9c196da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.160:35450/0 (socket says 172.21.15.160:35450) 2024-03-07T07:44:08.866 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:08.863+0000 7f1da27aa640 1 -- 172.21.15.160:0/1602006050 learned_addr learned my addr 172.21.15.160:0/1602006050 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:44:08.866 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:08.864+0000 7f1da27aa640 1 -- 172.21.15.160:0/1602006050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d9c101790 msgr2=0x7f1d9c196da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:08.866 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:08.864+0000 7f1da27aa640 1 --2- 172.21.15.160:0/1602006050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d9c101790 0x7f1d9c196da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:08.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:08.864+0000 7f1da27aa640 1 -- 172.21.15.160:0/1602006050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d9c1005a0 msgr2=0x7f1d9c196860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:08.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:08.864+0000 7f1da27aa640 1 --2- 172.21.15.160:0/1602006050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d9c1005a0 0x7f1d9c196860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:08.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:08.864+0000 7f1da27aa640 1 -- 172.21.15.160:0/1602006050 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d9c19e480 con 0x7f1d9c102150 2024-03-07T07:44:08.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:08.864+0000 7f1da1fa9640 1 --2- 172.21.15.160:0/1602006050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d9c1005a0 0x7f1d9c196860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:44:08.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:08.864+0000 7f1da17a8640 1 --2- 172.21.15.160:0/1602006050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d9c101790 0x7f1d9c196da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:44:08.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:08.864+0000 7f1da27aa640 1 --2- 172.21.15.160:0/1602006050 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d9c102150 0x7f1d9c19de10 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f1d94037040 tx=0x7f1d94035d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:44:08.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:08.864+0000 7f1d92ffd640 1 -- 172.21.15.160:0/1602006050 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d94039040 con 0x7f1d9c102150 2024-03-07T07:44:08.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:08.864+0000 7f1da420d640 1 -- 172.21.15.160:0/1602006050 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d9c19e710 con 0x7f1d9c102150 2024-03-07T07:44:08.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:08.864+0000 7f1d92ffd640 1 -- 172.21.15.160:0/1602006050 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1d94044070 con 0x7f1d9c102150 2024-03-07T07:44:08.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:08.864+0000 7f1d92ffd640 1 -- 172.21.15.160:0/1602006050 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d94004460 con 0x7f1d9c102150 2024-03-07T07:44:08.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:08.864+0000 7f1da420d640 1 -- 172.21.15.160:0/1602006050 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d9c19ec00 con 0x7f1d9c102150 2024-03-07T07:44:08.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:08.865+0000 7f1da420d640 1 -- 172.21.15.160:0/1602006050 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d9c107860 con 0x7f1d9c102150 2024-03-07T07:44:08.868 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:08.866+0000 7f1d92ffd640 1 -- 172.21.15.160:0/1602006050 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 20) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f1d94004600 con 0x7f1d9c102150 2024-03-07T07:44:08.868 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:08.866+0000 7f1d92ffd640 1 --2- 172.21.15.160:0/1602006050 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f1d880a1d80 0x7f1d880a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:44:08.868 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:08.867+0000 7f1da1fa9640 1 --2- 172.21.15.160:0/1602006050 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f1d880a1d80 0x7f1d880a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:44:08.868 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:08.867+0000 7f1d92ffd640 1 -- 172.21.15.160:0/1602006050 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1d940f03e0 con 0x7f1d9c102150 2024-03-07T07:44:08.868 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:08.867+0000 7f1da1fa9640 1 --2- 172.21.15.160:0/1602006050 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f1d880a1d80 0x7f1d880a4240 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c1015f0 tx=0x7f1d98005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:44:08.871 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:08.871+0000 7f1d92ffd640 1 -- 172.21.15.160:0/1602006050 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1d940bd810 con 0x7f1d9c102150 2024-03-07T07:44:09.047 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.047+0000 7f1da420d640 1 -- 172.21.15.160:0/1602006050 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d9c062260 con 0x7f1d9c102150 2024-03-07T07:44:09.049 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.049+0000 7f1d92ffd640 1 -- 172.21.15.160:0/1602006050 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22721 (secure 0 0 0) 0x7f1d940bd110 con 0x7f1d9c102150 2024-03-07T07:44:09.049 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:44:09.049 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":87,"fsid":"826f6582-dc54-11ee-95c6-87774f69a715","created":"2024-03-07T07:31:14.229288+0000","modified":"2024-03-07T07:40:09.354265+0000","last_up_change":"2024-03-07T07:38:20.776225+0000","last_in_change":"2024-03-07T07:38:00.508917+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T07:34:34.529424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T07:39:12.075432+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T07:39:13.367820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bee66c67-fb6e-4717-9685-ca62587d5f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6803","nonce":1368482926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6805","nonce":1368482926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6809","nonce":1368482926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6807","nonce":1368482926}]},"public_addr":"172.21.15.160:6803/1368482926","cluster_addr":"172.21.15.160:6805/1368482926","heartbeat_back_addr":"172.21.15.160:6809/1368482926","heartbeat_front_addr":"172.21.15.160:6807/1368482926","state":["exists","up"]},{"osd":1,"uuid":"49236251-f750-424e-992b-bddf55762556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6801","nonce":2922391884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6803","nonce":2922391884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6807","nonce":2922391884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6805","nonce":2922391884}]},"public_addr":"172.21.15.177:6801/2922391884","cluster_addr":"172.21.15.177:6803/2922391884","heartbeat_back_addr":"172.21.15.177:6807/2922391884","heartbeat_front_addr":"172.21.15.177:6805/2922391884","state":["exists","up"]},{"osd":2,"uuid":"7a432b28-f7ee-43fe-9196-7aaeccfecf77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6801","nonce":1217901551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6803","nonce":1217901551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6807","nonce":1217901551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6805","nonce":1217901551}]},"public_addr":"172.21.15.178:6801/1217901551","cluster_addr":"172.21.15.178:6803/1217901551","heartbeat_back_addr":"172.21.15.178:6807/1217901551","heartbeat_front_addr":"172.21.15.178:6805/1217901551","state":["exists","up"]},{"osd":3,"uuid":"4ad22a20-f1be-4002-a48d-ecd3953e63f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6811","nonce":42543989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6813","nonce":42543989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6817","nonce":42543989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6815","nonce":42543989}]},"public_addr":"172.21.15.160:6811/42543989","cluster_addr":"172.21.15.160:6813/42543989","heartbeat_back_addr":"172.21.15.160:6817/42543989","heartbeat_front_addr":"172.21.15.160:6815/42543989","state":["exists","up"]},{"osd":4,"uuid":"2b65b104-ac82-4ce3-b6c5-11a399103045","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6809","nonce":88178553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6811","nonce":88178553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6815","nonce":88178553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6813","nonce":88178553}]},"public_addr":"172.21.15.177:6809/88178553","cluster_addr":"172.21.15.177:6811/88178553","heartbeat_back_addr":"172.21.15.177:6815/88178553","heartbeat_front_addr":"172.21.15.177:6813/88178553","state":["exists","up"]},{"osd":5,"uuid":"30a1e714-08ff-44e8-9264-9ef164604b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6809","nonce":964732891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6811","nonce":964732891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6815","nonce":964732891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6813","nonce":964732891}]},"public_addr":"172.21.15.178:6809/964732891","cluster_addr":"172.21.15.178:6811/964732891","heartbeat_back_addr":"172.21.15.178:6815/964732891","heartbeat_front_addr":"172.21.15.178:6813/964732891","state":["exists","up"]},{"osd":6,"uuid":"b1e1646e-f8a9-42b8-b62d-9d7b5d6a1ff4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6819","nonce":3251327001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6821","nonce":3251327001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6825","nonce":3251327001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6823","nonce":3251327001}]},"public_addr":"172.21.15.160:6819/3251327001","cluster_addr":"172.21.15.160:6821/3251327001","heartbeat_back_addr":"172.21.15.160:6825/3251327001","heartbeat_front_addr":"172.21.15.160:6823/3251327001","state":["exists","up"]},{"osd":7,"uuid":"e66b400b-9747-4f6d-b05b-db22468e2750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6817","nonce":1175500850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6819","nonce":1175500850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6823","nonce":1175500850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6821","nonce":1175500850}]},"public_addr":"172.21.15.177:6817/1175500850","cluster_addr":"172.21.15.177:6819/1175500850","heartbeat_back_addr":"172.21.15.177:6823/1175500850","heartbeat_front_addr":"172.21.15.177:6821/1175500850","state":["exists","up"]},{"osd":8,"uuid":"3ee0f2c4-fca4-49fc-a98b-07aa0fcba0e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6817","nonce":688015531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6819","nonce":688015531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6823","nonce":688015531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6821","nonce":688015531}]},"public_addr":"172.21.15.178:6817/688015531","cluster_addr":"172.21.15.178:6819/688015531","heartbeat_back_addr":"172.21.15.178:6823/688015531","heartbeat_front_addr":"172.21.15.178:6821/688015531","state":["exists","up"]},{"osd":9,"uuid":"856039d7-f7ee-4dbd-b019-0073e705ce0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6827","nonce":3265797851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6829","nonce":3265797851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6833","nonce":3265797851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6831","nonce":3265797851}]},"public_addr":"172.21.15.160:6827/3265797851","cluster_addr":"172.21.15.160:6829/3265797851","heartbeat_back_addr":"172.21.15.160:6833/3265797851","heartbeat_front_addr":"172.21.15.160:6831/3265797851","state":["exists","up"]},{"osd":10,"uuid":"bdc9899d-d720-4429-ad53-4c388a4dbe75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6825","nonce":2867473412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6827","nonce":2867473412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6831","nonce":2867473412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6829","nonce":2867473412}]},"public_addr":"172.21.15.177:6825/2867473412","cluster_addr":"172.21.15.177:6827/2867473412","heartbeat_back_addr":"172.21.15.177:6831/2867473412","heartbeat_front_addr":"172.21.15.177:6829/2867473412","state":["exists","up"]},{"osd":11,"uuid":"cc9adb54-ef49-47c6-90e3-fc03bc0f5cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6825","nonce":4068724731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6827","nonce":4068724731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6831","nonce":4068724731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6829","nonce":4068724731}]},"public_addr":"172.21.15.178:6825/4068724731","cluster_addr":"172.21.15.178:6827/4068724731","heartbeat_back_addr":"172.21.15.178:6831/4068724731","heartbeat_front_addr":"172.21.15.178:6829/4068724731","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:33:44.141585+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:07.685585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:31.044386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:56.308925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:35:21.582097+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:35:45.983540+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:36:11.530242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:36:36.844681+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:02.303437+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:27.629867+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:53.289495+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:38:18.536169+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/3027450612":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/874709722":"2024-03-08T07:32:08.190837+0000","172.21.15.160:6800/1130675225":"2024-03-08T07:31:41.166858+0000","172.21.15.160:0/3857193980":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/3861968297":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6800/3859174315":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/3013534743":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6801/3859174315":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/1310937667":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6801/1130675225":"2024-03-08T07:31:41.166858+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T07:44:09.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.051+0000 7f1da420d640 1 -- 172.21.15.160:0/1602006050 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f1d880a1d80 msgr2=0x7f1d880a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:09.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.051+0000 7f1da420d640 1 --2- 172.21.15.160:0/1602006050 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f1d880a1d80 0x7f1d880a4240 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c1015f0 tx=0x7f1d98005f90 comp rx=0 tx=0).stop 2024-03-07T07:44:09.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.051+0000 7f1da420d640 1 -- 172.21.15.160:0/1602006050 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d9c102150 msgr2=0x7f1d9c19de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:09.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.051+0000 7f1da420d640 1 --2- 172.21.15.160:0/1602006050 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d9c102150 0x7f1d9c19de10 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f1d94037040 tx=0x7f1d94035d90 comp rx=0 tx=0).stop 2024-03-07T07:44:09.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.051+0000 7f1da420d640 1 -- 172.21.15.160:0/1602006050 shutdown_connections 2024-03-07T07:44:09.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.051+0000 7f1da420d640 1 --2- 172.21.15.160:0/1602006050 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f1d880a1d80 0x7f1d880a4240 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:09.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.051+0000 7f1da420d640 1 --2- 172.21.15.160:0/1602006050 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d9c102150 0x7f1d9c19de10 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:09.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.051+0000 7f1da420d640 1 --2- 172.21.15.160:0/1602006050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d9c101790 0x7f1d9c196da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:09.052 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.051+0000 7f1da420d640 1 --2- 172.21.15.160:0/1602006050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d9c1005a0 0x7f1d9c196860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:09.052 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.051+0000 7f1da420d640 1 -- 172.21.15.160:0/1602006050 >> 172.21.15.160:0/1602006050 conn(0x7f1d9c0fbb10 msgr2=0x7f1d9c104a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:44:09.052 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.051+0000 7f1da420d640 1 -- 172.21.15.160:0/1602006050 shutdown_connections 2024-03-07T07:44:09.052 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.051+0000 7f1da420d640 1 -- 172.21.15.160:0/1602006050 wait complete. 2024-03-07T07:44:09.063 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-07T07:34:34.529424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-07T07:39:12.075432+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-07T07:39:13.367820+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-07T07:44:09.063 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-07T07:44:09.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.208+0000 7fe786554640 1 -- 172.21.15.160:0/1716376509 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe780102160 msgr2=0x7fe780104550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:09.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.208+0000 7fe786554640 1 --2- 172.21.15.160:0/1716376509 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe780102160 0x7fe780104550 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe770009900 tx=0x7fe77002bc30 comp rx=0 tx=0).stop 2024-03-07T07:44:09.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.208+0000 7fe786554640 1 -- 172.21.15.160:0/1716376509 shutdown_connections 2024-03-07T07:44:09.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.208+0000 7fe786554640 1 --2- 172.21.15.160:0/1716376509 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7801073c0 0x7fe7801097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:09.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.208+0000 7fe786554640 1 --2- 172.21.15.160:0/1716376509 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe780104a90 0x7fe780106e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:09.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.208+0000 7fe786554640 1 --2- 172.21.15.160:0/1716376509 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe780102160 0x7fe780104550 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:09.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.208+0000 7fe786554640 1 -- 172.21.15.160:0/1716376509 >> 172.21.15.160:0/1716376509 conn(0x7fe7800fbaf0 msgr2=0x7fe7800fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:44:09.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.208+0000 7fe786554640 1 -- 172.21.15.160:0/1716376509 shutdown_connections 2024-03-07T07:44:09.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.208+0000 7fe786554640 1 -- 172.21.15.160:0/1716376509 wait complete. 2024-03-07T07:44:09.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.208+0000 7fe786554640 1 Processor -- start 2024-03-07T07:44:09.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.208+0000 7fe786554640 1 -- start start 2024-03-07T07:44:09.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.209+0000 7fe786554640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe780102160 0x7fe7801979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:44:09.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.209+0000 7fe786554640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe780104a90 0x7fe780197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:44:09.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.209+0000 7fe786554640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7801073c0 0x7fe78019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:44:09.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.209+0000 7fe786554640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe78010d2e0 con 0x7fe7801073c0 2024-03-07T07:44:09.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.209+0000 7fe786554640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe78010d160 con 0x7fe780104a90 2024-03-07T07:44:09.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.209+0000 7fe786554640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe78010d460 con 0x7fe780102160 2024-03-07T07:44:09.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.209+0000 7fe784af1640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7801073c0 0x7fe78019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:44:09.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.209+0000 7fe784af1640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7801073c0 0x7fe78019ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:46728/0 (socket says 172.21.15.160:46728) 2024-03-07T07:44:09.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.209+0000 7fe77f7fe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe780104a90 0x7fe780197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:44:09.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.209+0000 7fe784af1640 1 -- 172.21.15.160:0/1853701015 learned_addr learned my addr 172.21.15.160:0/1853701015 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:44:09.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.209+0000 7fe77ffff640 1 --2- 172.21.15.160:0/1853701015 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe780102160 0x7fe7801979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:44:09.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.209+0000 7fe784af1640 1 -- 172.21.15.160:0/1853701015 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe780102160 msgr2=0x7fe7801979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:09.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.209+0000 7fe784af1640 1 --2- 172.21.15.160:0/1853701015 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe780102160 0x7fe7801979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:09.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.209+0000 7fe784af1640 1 -- 172.21.15.160:0/1853701015 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe780104a90 msgr2=0x7fe780197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:09.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.209+0000 7fe784af1640 1 --2- 172.21.15.160:0/1853701015 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe780104a90 0x7fe780197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:09.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.209+0000 7fe784af1640 1 -- 172.21.15.160:0/1853701015 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe78019f600 con 0x7fe7801073c0 2024-03-07T07:44:09.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.209+0000 7fe77f7fe640 1 --2- 172.21.15.160:0/1853701015 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe780104a90 0x7fe780197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:44:09.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.209+0000 7fe784af1640 1 --2- 172.21.15.160:0/1853701015 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7801073c0 0x7fe78019ef90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe77800eb10 tx=0x7fe77800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:44:09.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.209+0000 7fe77d7fa640 1 -- 172.21.15.160:0/1853701015 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe778004b20 con 0x7fe7801073c0 2024-03-07T07:44:09.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.209+0000 7fe77d7fa640 1 -- 172.21.15.160:0/1853701015 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe778004cc0 con 0x7fe7801073c0 2024-03-07T07:44:09.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.209+0000 7fe77d7fa640 1 -- 172.21.15.160:0/1853701015 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe778010760 con 0x7fe7801073c0 2024-03-07T07:44:09.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.209+0000 7fe786554640 1 -- 172.21.15.160:0/1853701015 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe78019f8f0 con 0x7fe7801073c0 2024-03-07T07:44:09.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.209+0000 7fe786554640 1 -- 172.21.15.160:0/1853701015 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe78019fdf0 con 0x7fe7801073c0 2024-03-07T07:44:09.214 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.210+0000 7fe786554640 1 -- 172.21.15.160:0/1853701015 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7800fd5e0 con 0x7fe7801073c0 2024-03-07T07:44:09.214 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.211+0000 7fe77d7fa640 1 -- 172.21.15.160:0/1853701015 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 20) v1 ==== 136754+0+0 (secure 0 0 0) 0x7fe778005020 con 0x7fe7801073c0 2024-03-07T07:44:09.214 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.212+0000 7fe77d7fa640 1 --2- 172.21.15.160:0/1853701015 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fe7680a1d80 0x7fe7680a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:44:09.214 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.212+0000 7fe77ffff640 1 --2- 172.21.15.160:0/1853701015 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fe7680a1d80 0x7fe7680a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:44:09.214 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.212+0000 7fe77d7fa640 1 -- 172.21.15.160:0/1853701015 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe7780ce360 con 0x7fe7801073c0 2024-03-07T07:44:09.214 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.213+0000 7fe77ffff640 1 --2- 172.21.15.160:0/1853701015 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fe7680a1d80 0x7fe7680a4240 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe77002c7b0 tx=0x7fe770005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:44:09.216 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.217+0000 7fe77d7fa640 1 -- 172.21.15.160:0/1853701015 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fe77809b790 con 0x7fe7801073c0 2024-03-07T07:44:09.391 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.391+0000 7fe786554640 1 -- 172.21.15.160:0/1853701015 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe780062260 con 0x7fe7801073c0 2024-03-07T07:44:09.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.391+0000 7fe77d7fa640 1 -- 172.21.15.160:0/1853701015 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe77809b090 con 0x7fe7801073c0 2024-03-07T07:44:09.392 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 1 2024-03-07T07:44:09.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.393+0000 7fe786554640 1 -- 172.21.15.160:0/1853701015 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fe7680a1d80 msgr2=0x7fe7680a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:09.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.393+0000 7fe786554640 1 --2- 172.21.15.160:0/1853701015 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fe7680a1d80 0x7fe7680a4240 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe77002c7b0 tx=0x7fe770005fd0 comp rx=0 tx=0).stop 2024-03-07T07:44:09.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.393+0000 7fe786554640 1 -- 172.21.15.160:0/1853701015 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7801073c0 msgr2=0x7fe78019ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:09.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.393+0000 7fe786554640 1 --2- 172.21.15.160:0/1853701015 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7801073c0 0x7fe78019ef90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe77800eb10 tx=0x7fe77800c2d0 comp rx=0 tx=0).stop 2024-03-07T07:44:09.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.394+0000 7fe786554640 1 -- 172.21.15.160:0/1853701015 shutdown_connections 2024-03-07T07:44:09.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.394+0000 7fe786554640 1 --2- 172.21.15.160:0/1853701015 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fe7680a1d80 0x7fe7680a4240 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:09.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.394+0000 7fe786554640 1 --2- 172.21.15.160:0/1853701015 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7801073c0 0x7fe78019ef90 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:09.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.394+0000 7fe786554640 1 --2- 172.21.15.160:0/1853701015 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe780104a90 0x7fe780197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:09.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.394+0000 7fe786554640 1 --2- 172.21.15.160:0/1853701015 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe780102160 0x7fe7801979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:09.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.394+0000 7fe786554640 1 -- 172.21.15.160:0/1853701015 >> 172.21.15.160:0/1853701015 conn(0x7fe7800fbaf0 msgr2=0x7fe780102ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:44:09.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.394+0000 7fe786554640 1 -- 172.21.15.160:0/1853701015 shutdown_connections 2024-03-07T07:44:09.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.394+0000 7fe786554640 1 -- 172.21.15.160:0/1853701015 wait complete. 2024-03-07T07:44:09.405 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-07T07:44:09.513 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.510+0000 7f6802372640 1 -- 172.21.15.160:0/325747721 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f67fc102180 msgr2=0x7f67fc104570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:09.513 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.510+0000 7f6802372640 1 --2- 172.21.15.160:0/325747721 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f67fc102180 0x7f67fc104570 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f67ec009860 tx=0x7f67ec02bb70 comp rx=0 tx=0).stop 2024-03-07T07:44:09.513 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.510+0000 7f6802372640 1 -- 172.21.15.160:0/325747721 shutdown_connections 2024-03-07T07:44:09.513 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.510+0000 7f6802372640 1 --2- 172.21.15.160:0/325747721 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67fc1073e0 0x7f67fc1097d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:09.513 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.510+0000 7f6802372640 1 --2- 172.21.15.160:0/325747721 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67fc104ab0 0x7f67fc106ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:09.513 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.510+0000 7f6802372640 1 --2- 172.21.15.160:0/325747721 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f67fc102180 0x7f67fc104570 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:09.513 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.510+0000 7f6802372640 1 -- 172.21.15.160:0/325747721 >> 172.21.15.160:0/325747721 conn(0x7f67fc0fbaf0 msgr2=0x7f67fc0fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:44:09.513 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.510+0000 7f6802372640 1 -- 172.21.15.160:0/325747721 shutdown_connections 2024-03-07T07:44:09.513 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.510+0000 7f6802372640 1 -- 172.21.15.160:0/325747721 wait complete. 2024-03-07T07:44:09.514 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.511+0000 7f6802372640 1 Processor -- start 2024-03-07T07:44:09.514 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.511+0000 7f6802372640 1 -- start start 2024-03-07T07:44:09.514 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.511+0000 7f6802372640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67fc102180 0x7f67fc199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:44:09.514 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.511+0000 7f6802372640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67fc104ab0 0x7f67fc19a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:44:09.514 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.511+0000 7f6802372640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f67fc1073e0 0x7f67fc1a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:44:09.514 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.511+0000 7f6802372640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67fc10d2e0 con 0x7f67fc102180 2024-03-07T07:44:09.514 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.511+0000 7f6802372640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67fc10d160 con 0x7f67fc1073e0 2024-03-07T07:44:09.514 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.511+0000 7f6802372640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67fc10d460 con 0x7f67fc104ab0 2024-03-07T07:44:09.514 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.511+0000 7f67fbfff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67fc102180 0x7f67fc199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:44:09.514 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.511+0000 7f67fbfff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67fc102180 0x7f67fc199c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:36882/0 (socket says 172.21.15.160:36882) 2024-03-07T07:44:09.514 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.511+0000 7f67fbfff640 1 -- 172.21.15.160:0/1938052845 learned_addr learned my addr 172.21.15.160:0/1938052845 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:44:09.514 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.512+0000 7f67fb7fe640 1 --2- 172.21.15.160:0/1938052845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67fc104ab0 0x7f67fc19a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:44:09.514 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.512+0000 7f680090f640 1 --2- 172.21.15.160:0/1938052845 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f67fc1073e0 0x7f67fc1a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:44:09.515 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.512+0000 7f67fbfff640 1 -- 172.21.15.160:0/1938052845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67fc104ab0 msgr2=0x7f67fc19a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:09.515 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.512+0000 7f67fbfff640 1 --2- 172.21.15.160:0/1938052845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67fc104ab0 0x7f67fc19a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:09.515 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.512+0000 7f67fbfff640 1 -- 172.21.15.160:0/1938052845 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f67fc1073e0 msgr2=0x7f67fc1a11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:09.515 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.512+0000 7f67fbfff640 1 --2- 172.21.15.160:0/1938052845 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f67fc1073e0 0x7f67fc1a11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:09.515 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.512+0000 7f67fbfff640 1 -- 172.21.15.160:0/1938052845 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67fc1a1860 con 0x7f67fc102180 2024-03-07T07:44:09.515 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.512+0000 7f67fbfff640 1 --2- 172.21.15.160:0/1938052845 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67fc102180 0x7f67fc199c50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f67ec005940 tx=0x7f67ec032470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:44:09.515 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.512+0000 7f67f97fa640 1 -- 172.21.15.160:0/1938052845 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ec041020 con 0x7f67fc102180 2024-03-07T07:44:09.515 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.512+0000 7f6802372640 1 -- 172.21.15.160:0/1938052845 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67fc1a1af0 con 0x7f67fc102180 2024-03-07T07:44:09.515 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.512+0000 7f67f97fa640 1 -- 172.21.15.160:0/1938052845 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f67ec02c9f0 con 0x7f67fc102180 2024-03-07T07:44:09.515 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.512+0000 7f67f97fa640 1 -- 172.21.15.160:0/1938052845 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ec039040 con 0x7f67fc102180 2024-03-07T07:44:09.515 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.512+0000 7f6802372640 1 -- 172.21.15.160:0/1938052845 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67fc1a1fe0 con 0x7f67fc102180 2024-03-07T07:44:09.516 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.513+0000 7f6802372640 1 -- 172.21.15.160:0/1938052845 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67fc108ed0 con 0x7f67fc102180 2024-03-07T07:44:09.516 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.514+0000 7f67f97fa640 1 -- 172.21.15.160:0/1938052845 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 20) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f67ec044020 con 0x7f67fc102180 2024-03-07T07:44:09.516 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.515+0000 7f67f97fa640 1 --2- 172.21.15.160:0/1938052845 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f67e40a1d80 0x7f67e40a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:44:09.516 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.515+0000 7f67f97fa640 1 -- 172.21.15.160:0/1938052845 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f67ec0efb70 con 0x7f67fc102180 2024-03-07T07:44:09.517 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.515+0000 7f67fb7fe640 1 --2- 172.21.15.160:0/1938052845 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f67e40a1d80 0x7f67e40a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:44:09.517 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.515+0000 7f67fb7fe640 1 --2- 172.21.15.160:0/1938052845 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f67e40a1d80 0x7f67e40a4240 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f67fc069280 tx=0x7f67f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:44:09.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.518+0000 7f67f97fa640 1 -- 172.21.15.160:0/1938052845 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f67ec04b020 con 0x7f67fc102180 2024-03-07T07:44:09.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:09 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/1602006050' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:44:09.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:09 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/1853701015' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-07T07:44:09.694 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.694+0000 7f6802372640 1 -- 172.21.15.160:0/1938052845 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f67fc02d280 con 0x7f67fc102180 2024-03-07T07:44:09.694 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.694+0000 7f67f97fa640 1 -- 172.21.15.160:0/1938052845 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f67ec0bd0c0 con 0x7f67fc102180 2024-03-07T07:44:09.694 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 32 2024-03-07T07:44:09.696 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.696+0000 7f6802372640 1 -- 172.21.15.160:0/1938052845 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f67e40a1d80 msgr2=0x7f67e40a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:09.696 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.696+0000 7f6802372640 1 --2- 172.21.15.160:0/1938052845 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f67e40a1d80 0x7f67e40a4240 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f67fc069280 tx=0x7f67f0009260 comp rx=0 tx=0).stop 2024-03-07T07:44:09.697 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.696+0000 7f6802372640 1 -- 172.21.15.160:0/1938052845 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67fc102180 msgr2=0x7f67fc199c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:09.697 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.696+0000 7f6802372640 1 --2- 172.21.15.160:0/1938052845 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67fc102180 0x7f67fc199c50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f67ec005940 tx=0x7f67ec032470 comp rx=0 tx=0).stop 2024-03-07T07:44:09.697 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.696+0000 7f6802372640 1 -- 172.21.15.160:0/1938052845 shutdown_connections 2024-03-07T07:44:09.697 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.696+0000 7f6802372640 1 --2- 172.21.15.160:0/1938052845 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f67e40a1d80 0x7f67e40a4240 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:09.697 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.696+0000 7f6802372640 1 --2- 172.21.15.160:0/1938052845 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f67fc1073e0 0x7f67fc1a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:09.697 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.696+0000 7f6802372640 1 --2- 172.21.15.160:0/1938052845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67fc104ab0 0x7f67fc19a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:09.697 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.696+0000 7f6802372640 1 --2- 172.21.15.160:0/1938052845 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67fc102180 0x7f67fc199c50 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:09.697 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.696+0000 7f6802372640 1 -- 172.21.15.160:0/1938052845 >> 172.21.15.160:0/1938052845 conn(0x7f67fc0fbaf0 msgr2=0x7f67fc101910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:44:09.697 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.696+0000 7f6802372640 1 -- 172.21.15.160:0/1938052845 shutdown_connections 2024-03-07T07:44:09.697 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.696+0000 7f6802372640 1 -- 172.21.15.160:0/1938052845 wait complete. 2024-03-07T07:44:09.707 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-07T07:44:09.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:09 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/1602006050' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:44:09.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:09 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/1853701015' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-07T07:44:09.814 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.812+0000 7f1ce1a34640 1 -- 172.21.15.160:0/1755192340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cdc1020e0 msgr2=0x7f1cdc1044d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:09.814 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.812+0000 7f1ce1a34640 1 --2- 172.21.15.160:0/1755192340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cdc1020e0 0x7f1cdc1044d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc009900 tx=0x7f1ccc02bc30 comp rx=0 tx=0).stop 2024-03-07T07:44:09.814 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.812+0000 7f1ce1a34640 1 -- 172.21.15.160:0/1755192340 shutdown_connections 2024-03-07T07:44:09.814 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.812+0000 7f1ce1a34640 1 --2- 172.21.15.160:0/1755192340 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cdc107340 0x7f1cdc109790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:09.814 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.812+0000 7f1ce1a34640 1 --2- 172.21.15.160:0/1755192340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1cdc104a10 0x7f1cdc106e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:09.814 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.812+0000 7f1ce1a34640 1 --2- 172.21.15.160:0/1755192340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cdc1020e0 0x7f1cdc1044d0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:09.814 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.812+0000 7f1ce1a34640 1 -- 172.21.15.160:0/1755192340 >> 172.21.15.160:0/1755192340 conn(0x7f1cdc0fbaf0 msgr2=0x7f1cdc0fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:44:09.814 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.812+0000 7f1ce1a34640 1 -- 172.21.15.160:0/1755192340 shutdown_connections 2024-03-07T07:44:09.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.812+0000 7f1ce1a34640 1 -- 172.21.15.160:0/1755192340 wait complete. 2024-03-07T07:44:09.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.812+0000 7f1ce1a34640 1 Processor -- start 2024-03-07T07:44:09.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.812+0000 7f1ce1a34640 1 -- start start 2024-03-07T07:44:09.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.813+0000 7f1ce1a34640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1cdc1020e0 0x7f1cdc1979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:44:09.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.813+0000 7f1ce1a34640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cdc104a10 0x7f1cdc197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:44:09.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.813+0000 7f1ce1a34640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cdc107340 0x7f1cdc19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:44:09.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.813+0000 7f1ce1a34640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc10d2f0 con 0x7f1cdc107340 2024-03-07T07:44:09.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.813+0000 7f1ce1a34640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc10d170 con 0x7f1cdc104a10 2024-03-07T07:44:09.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.813+0000 7f1ce1a34640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc10d470 con 0x7f1cdc1020e0 2024-03-07T07:44:09.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.813+0000 7f1cdaffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1cdc1020e0 0x7f1cdc1979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:44:09.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.813+0000 7f1cdb7fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cdc107340 0x7f1cdc19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:44:09.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.813+0000 7f1cda7fc640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cdc104a10 0x7f1cdc197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:44:09.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.813+0000 7f1cdaffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1cdc1020e0 0x7f1cdc1979c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.160:47392/0 (socket says 172.21.15.160:47392) 2024-03-07T07:44:09.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.813+0000 7f1cdaffd640 1 -- 172.21.15.160:0/2771793775 learned_addr learned my addr 172.21.15.160:0/2771793775 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:44:09.816 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.813+0000 7f1cdaffd640 1 -- 172.21.15.160:0/2771793775 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cdc104a10 msgr2=0x7f1cdc197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:09.816 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.813+0000 7f1cdaffd640 1 --2- 172.21.15.160:0/2771793775 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cdc104a10 0x7f1cdc197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:09.816 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.813+0000 7f1cdaffd640 1 -- 172.21.15.160:0/2771793775 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cdc107340 msgr2=0x7f1cdc19ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:09.816 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.813+0000 7f1cdaffd640 1 --2- 172.21.15.160:0/2771793775 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cdc107340 0x7f1cdc19ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:09.816 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.813+0000 7f1cdaffd640 1 -- 172.21.15.160:0/2771793775 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cdc19f5e0 con 0x7f1cdc1020e0 2024-03-07T07:44:09.816 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.813+0000 7f1cdb7fe640 1 --2- 172.21.15.160:0/2771793775 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cdc107340 0x7f1cdc19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:44:09.816 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.813+0000 7f1cdaffd640 1 --2- 172.21.15.160:0/2771793775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1cdc1020e0 0x7f1cdc1979c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc009230 tx=0x7f1ccc02c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:44:09.816 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.813+0000 7f1ce0a32640 1 -- 172.21.15.160:0/2771793775 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc041070 con 0x7f1cdc1020e0 2024-03-07T07:44:09.816 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.813+0000 7f1ce0a32640 1 -- 172.21.15.160:0/2771793775 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1ccc004450 con 0x7f1cdc1020e0 2024-03-07T07:44:09.816 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.813+0000 7f1ce1a34640 1 -- 172.21.15.160:0/2771793775 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cdc19f870 con 0x7f1cdc1020e0 2024-03-07T07:44:09.816 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.813+0000 7f1ce0a32640 1 -- 172.21.15.160:0/2771793775 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc003e20 con 0x7f1cdc1020e0 2024-03-07T07:44:09.816 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.814+0000 7f1ce1a34640 1 -- 172.21.15.160:0/2771793775 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cdc19fd60 con 0x7f1cdc1020e0 2024-03-07T07:44:09.817 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.814+0000 7f1ce1a34640 1 -- 172.21.15.160:0/2771793775 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cdc0fd5e0 con 0x7f1cdc1020e0 2024-03-07T07:44:09.817 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.815+0000 7f1ce0a32640 1 -- 172.21.15.160:0/2771793775 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 20) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f1ccc044070 con 0x7f1cdc1020e0 2024-03-07T07:44:09.817 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.816+0000 7f1ce0a32640 1 --2- 172.21.15.160:0/2771793775 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f1cc40a1d80 0x7f1cc40a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:44:09.818 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.816+0000 7f1cda7fc640 1 --2- 172.21.15.160:0/2771793775 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f1cc40a1d80 0x7f1cc40a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:44:09.818 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.816+0000 7f1ce0a32640 1 -- 172.21.15.160:0/2771793775 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1ccc0ef8d0 con 0x7f1cdc1020e0 2024-03-07T07:44:09.818 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.816+0000 7f1cda7fc640 1 --2- 172.21.15.160:0/2771793775 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f1cc40a1d80 0x7f1cc40a4240 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc198ed0 tx=0x7f1cd0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:44:09.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.820+0000 7f1ce0a32640 1 -- 172.21.15.160:0/2771793775 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1ccc0bc6b0 con 0x7f1cdc1020e0 2024-03-07T07:44:09.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:09 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/1602006050' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:44:09.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:09 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/1853701015' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-07T07:44:09.994 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.994+0000 7f1ce1a34640 1 -- 172.21.15.160:0/2771793775 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1cdc02d1c0 con 0x7f1cdc1020e0 2024-03-07T07:44:09.995 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.995+0000 7f1ce0a32640 1 -- 172.21.15.160:0/2771793775 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1ccc0c0750 con 0x7f1cdc1020e0 2024-03-07T07:44:09.995 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 64 2024-03-07T07:44:09.997 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.997+0000 7f1ce1a34640 1 -- 172.21.15.160:0/2771793775 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f1cc40a1d80 msgr2=0x7f1cc40a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:09.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.997+0000 7f1ce1a34640 1 --2- 172.21.15.160:0/2771793775 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f1cc40a1d80 0x7f1cc40a4240 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc198ed0 tx=0x7f1cd0005f90 comp rx=0 tx=0).stop 2024-03-07T07:44:09.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.997+0000 7f1ce1a34640 1 -- 172.21.15.160:0/2771793775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1cdc1020e0 msgr2=0x7f1cdc1979c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:09.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.997+0000 7f1ce1a34640 1 --2- 172.21.15.160:0/2771793775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1cdc1020e0 0x7f1cdc1979c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc009230 tx=0x7f1ccc02c650 comp rx=0 tx=0).stop 2024-03-07T07:44:09.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.997+0000 7f1ce1a34640 1 -- 172.21.15.160:0/2771793775 shutdown_connections 2024-03-07T07:44:09.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.997+0000 7f1ce1a34640 1 --2- 172.21.15.160:0/2771793775 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f1cc40a1d80 0x7f1cc40a4240 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:09.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.997+0000 7f1ce1a34640 1 --2- 172.21.15.160:0/2771793775 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cdc107340 0x7f1cdc19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:09.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.997+0000 7f1ce1a34640 1 --2- 172.21.15.160:0/2771793775 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cdc104a10 0x7f1cdc197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:09.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.997+0000 7f1ce1a34640 1 --2- 172.21.15.160:0/2771793775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1cdc1020e0 0x7f1cdc1979c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:09.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.997+0000 7f1ce1a34640 1 -- 172.21.15.160:0/2771793775 >> 172.21.15.160:0/2771793775 conn(0x7f1cdc0fbaf0 msgr2=0x7f1cdc102b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:44:09.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.997+0000 7f1ce1a34640 1 -- 172.21.15.160:0/2771793775 shutdown_connections 2024-03-07T07:44:09.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:09.997+0000 7f1ce1a34640 1 -- 172.21.15.160:0/2771793775 wait complete. 2024-03-07T07:44:10.009 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-07T07:44:10.117 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.110+0000 7fc5814d0640 1 -- 172.21.15.160:0/1090683869 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc57c101670 msgr2=0x7fc57c101a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:10.117 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.110+0000 7fc5814d0640 1 --2- 172.21.15.160:0/1090683869 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc57c101670 0x7fc57c101a90 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc56c009900 tx=0x7fc56c02bc30 comp rx=0 tx=0).stop 2024-03-07T07:44:10.117 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.110+0000 7fc5814d0640 1 -- 172.21.15.160:0/1090683869 shutdown_connections 2024-03-07T07:44:10.117 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.110+0000 7fc5814d0640 1 --2- 172.21.15.160:0/1090683869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc57c103220 0x7fc57c109ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:10.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.110+0000 7fc5814d0640 1 --2- 172.21.15.160:0/1090683869 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc57c102860 0x7fc57c102ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:10.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.110+0000 7fc5814d0640 1 --2- 172.21.15.160:0/1090683869 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc57c101670 0x7fc57c101a90 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:10.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.110+0000 7fc5814d0640 1 -- 172.21.15.160:0/1090683869 >> 172.21.15.160:0/1090683869 conn(0x7fc57c0fcc00 msgr2=0x7fc57c0ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:44:10.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.110+0000 7fc5814d0640 1 -- 172.21.15.160:0/1090683869 shutdown_connections 2024-03-07T07:44:10.119 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.110+0000 7fc5814d0640 1 -- 172.21.15.160:0/1090683869 wait complete. 2024-03-07T07:44:10.119 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.111+0000 7fc5814d0640 1 Processor -- start 2024-03-07T07:44:10.119 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.111+0000 7fc5814d0640 1 -- start start 2024-03-07T07:44:10.119 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.111+0000 7fc5814d0640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc57c101670 0x7fc57c1979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:44:10.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.111+0000 7fc5814d0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc57c102860 0x7fc57c197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:44:10.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.111+0000 7fc5814d0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc57c103220 0x7fc57c19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:44:10.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.111+0000 7fc5814d0640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc57c10d2f0 con 0x7fc57c102860 2024-03-07T07:44:10.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.111+0000 7fc5814d0640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc57c10d170 con 0x7fc57c101670 2024-03-07T07:44:10.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.111+0000 7fc5814d0640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc57c10d470 con 0x7fc57c103220 2024-03-07T07:44:10.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.111+0000 7fc57a7fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc57c102860 0x7fc57c197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:44:10.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.111+0000 7fc57a7fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc57c102860 0x7fc57c197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:36924/0 (socket says 172.21.15.160:36924) 2024-03-07T07:44:10.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.111+0000 7fc57a7fc640 1 -- 172.21.15.160:0/3586718391 learned_addr learned my addr 172.21.15.160:0/3586718391 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:44:10.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.111+0000 7fc57affd640 1 --2- 172.21.15.160:0/3586718391 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc57c101670 0x7fc57c1979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:44:10.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.111+0000 7fc57b7fe640 1 --2- 172.21.15.160:0/3586718391 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc57c103220 0x7fc57c19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:44:10.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.112+0000 7fc57a7fc640 1 -- 172.21.15.160:0/3586718391 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc57c103220 msgr2=0x7fc57c19ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:10.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.112+0000 7fc57a7fc640 1 --2- 172.21.15.160:0/3586718391 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc57c103220 0x7fc57c19ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:10.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.112+0000 7fc57a7fc640 1 -- 172.21.15.160:0/3586718391 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc57c101670 msgr2=0x7fc57c1979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:10.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.112+0000 7fc57a7fc640 1 --2- 172.21.15.160:0/3586718391 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc57c101670 0x7fc57c1979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:10.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.112+0000 7fc57a7fc640 1 -- 172.21.15.160:0/3586718391 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc57c19f5e0 con 0x7fc57c102860 2024-03-07T07:44:10.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.112+0000 7fc57affd640 1 --2- 172.21.15.160:0/3586718391 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc57c101670 0x7fc57c1979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:44:10.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.112+0000 7fc57a7fc640 1 --2- 172.21.15.160:0/3586718391 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc57c102860 0x7fc57c197f00 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc57000e9f0 tx=0x7fc57000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:44:10.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.112+0000 7fc563fff640 1 -- 172.21.15.160:0/3586718391 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57000cd80 con 0x7fc57c102860 2024-03-07T07:44:10.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.112+0000 7fc5814d0640 1 -- 172.21.15.160:0/3586718391 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc57c19f8d0 con 0x7fc57c102860 2024-03-07T07:44:10.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.112+0000 7fc563fff640 1 -- 172.21.15.160:0/3586718391 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc570010910 con 0x7fc57c102860 2024-03-07T07:44:10.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.112+0000 7fc563fff640 1 -- 172.21.15.160:0/3586718391 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5700188f0 con 0x7fc57c102860 2024-03-07T07:44:10.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.112+0000 7fc5814d0640 1 -- 172.21.15.160:0/3586718391 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc57c19fe70 con 0x7fc57c102860 2024-03-07T07:44:10.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.113+0000 7fc5814d0640 1 -- 172.21.15.160:0/3586718391 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc57c108a30 con 0x7fc57c102860 2024-03-07T07:44:10.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.114+0000 7fc563fff640 1 -- 172.21.15.160:0/3586718391 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 20) v1 ==== 136754+0+0 (secure 0 0 0) 0x7fc570010ac0 con 0x7fc57c102860 2024-03-07T07:44:10.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.115+0000 7fc563fff640 1 --2- 172.21.15.160:0/3586718391 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fc5640a1d80 0x7fc5640a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:44:10.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.115+0000 7fc57affd640 1 --2- 172.21.15.160:0/3586718391 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fc5640a1d80 0x7fc5640a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:44:10.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.115+0000 7fc563fff640 1 -- 172.21.15.160:0/3586718391 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc5700ce510 con 0x7fc57c102860 2024-03-07T07:44:10.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.115+0000 7fc57affd640 1 --2- 172.21.15.160:0/3586718391 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fc5640a1d80 0x7fc5640a4240 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc56c000c00 tx=0x7fc56c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:44:10.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.118+0000 7fc563fff640 1 -- 172.21.15.160:0/3586718391 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc57009b2f0 con 0x7fc57c102860 2024-03-07T07:44:10.306 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.305+0000 7fc5814d0640 1 -- 172.21.15.160:0/3586718391 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc57c02d030 con 0x7fc57c102860 2024-03-07T07:44:10.306 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.306+0000 7fc563fff640 1 -- 172.21.15.160:0/3586718391 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) v1 ==== 81+0+601 (secure 0 0 0) 0x7fc57009f390 con 0x7fc57c102860 2024-03-07T07:44:10.306 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:0/3027450612 2024-03-08T07:32:08.190837+0000 2024-03-07T07:44:10.306 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:0/874709722 2024-03-08T07:32:08.190837+0000 2024-03-07T07:44:10.306 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:6800/1130675225 2024-03-08T07:31:41.166858+0000 2024-03-07T07:44:10.307 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:0/3857193980 2024-03-08T07:32:08.190837+0000 2024-03-07T07:44:10.307 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:0/3861968297 2024-03-08T07:31:41.166858+0000 2024-03-07T07:44:10.307 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:6800/3859174315 2024-03-08T07:32:08.190837+0000 2024-03-07T07:44:10.307 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:0/3013534743 2024-03-08T07:31:41.166858+0000 2024-03-07T07:44:10.307 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:6801/3859174315 2024-03-08T07:32:08.190837+0000 2024-03-07T07:44:10.307 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:0/1310937667 2024-03-08T07:31:41.166858+0000 2024-03-07T07:44:10.307 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160:6801/1130675225 2024-03-08T07:31:41.166858+0000 2024-03-07T07:44:10.307 INFO:teuthology.orchestra.run.smithi160.stderr:listed 10 entries 2024-03-07T07:44:10.308 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.308+0000 7fc5814d0640 1 -- 172.21.15.160:0/3586718391 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fc5640a1d80 msgr2=0x7fc5640a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:10.308 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.308+0000 7fc5814d0640 1 --2- 172.21.15.160:0/3586718391 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fc5640a1d80 0x7fc5640a4240 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc56c000c00 tx=0x7fc56c005fd0 comp rx=0 tx=0).stop 2024-03-07T07:44:10.308 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.308+0000 7fc5814d0640 1 -- 172.21.15.160:0/3586718391 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc57c102860 msgr2=0x7fc57c197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:10.308 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.308+0000 7fc5814d0640 1 --2- 172.21.15.160:0/3586718391 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc57c102860 0x7fc57c197f00 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc57000e9f0 tx=0x7fc57000eeb0 comp rx=0 tx=0).stop 2024-03-07T07:44:10.308 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.308+0000 7fc5814d0640 1 -- 172.21.15.160:0/3586718391 shutdown_connections 2024-03-07T07:44:10.309 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.308+0000 7fc5814d0640 1 --2- 172.21.15.160:0/3586718391 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fc5640a1d80 0x7fc5640a4240 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:10.309 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.308+0000 7fc5814d0640 1 --2- 172.21.15.160:0/3586718391 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc57c103220 0x7fc57c19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:10.309 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.308+0000 7fc5814d0640 1 --2- 172.21.15.160:0/3586718391 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc57c102860 0x7fc57c197f00 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:10.309 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.308+0000 7fc5814d0640 1 --2- 172.21.15.160:0/3586718391 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc57c101670 0x7fc57c1979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:10.309 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.308+0000 7fc5814d0640 1 -- 172.21.15.160:0/3586718391 >> 172.21.15.160:0/3586718391 conn(0x7fc57c0fcc00 msgr2=0x7fc57c105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:44:10.309 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.308+0000 7fc5814d0640 1 -- 172.21.15.160:0/3586718391 shutdown_connections 2024-03-07T07:44:10.309 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.308+0000 7fc5814d0640 1 -- 172.21.15.160:0/3586718391 wait complete. 2024-03-07T07:44:10.319 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-07T07:44:10.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:44:10.319 DEBUG:teuthology.orchestra.run.smithi160:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-07T07:44:10.360 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi160.front.sepia.ceph.com 2024-03-07T07:44:10.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:44:10.361 DEBUG:teuthology.orchestra.run.smithi160:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-07T07:44:10.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:44:10.432 DEBUG:teuthology.orchestra.run.smithi160:> ip netns list 2024-03-07T07:44:10.487 INFO:teuthology.orchestra.run.smithi160.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-07T07:44:10.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:44:10.488 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-07T07:44:10.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:44:10.556 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip link delete ceph-brx 2024-03-07T07:44:10.621 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:10 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/1938052845' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-07T07:44:10.621 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:10 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/2771793775' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-07T07:44:10.621 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:10 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/3586718391' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-07T07:44:10.621 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:10 smithi160 ceph-mon[31963]: pgmap v438: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-07T07:44:10.667 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-07T07:44:10.678 INFO:tasks.cephadm:Teardown begin 2024-03-07T07:44:10.678 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-07T07:44:10.678 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T07:44:10.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:10 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/1938052845' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-07T07:44:10.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:10 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/2771793775' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-07T07:44:10.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:10 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/3586718391' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-07T07:44:10.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:10 smithi178 ceph-mon[36650]: pgmap v438: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-07T07:44:10.811 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.809+0000 7f10a63c9640 1 -- 172.21.15.160:0/415826257 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10a0102100 msgr2=0x7f10a01044f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:10.811 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.809+0000 7f10a63c9640 1 --2- 172.21.15.160:0/415826257 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10a0102100 0x7f10a01044f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1090009900 tx=0x7f109002bc30 comp rx=0 tx=0).stop 2024-03-07T07:44:10.811 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.809+0000 7f10a63c9640 1 -- 172.21.15.160:0/415826257 shutdown_connections 2024-03-07T07:44:10.811 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.809+0000 7f10a63c9640 1 --2- 172.21.15.160:0/415826257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10a0107360 0x7f10a01097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:10.811 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.809+0000 7f10a63c9640 1 --2- 172.21.15.160:0/415826257 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10a0104a30 0x7f10a0106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:10.811 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.809+0000 7f10a63c9640 1 --2- 172.21.15.160:0/415826257 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10a0102100 0x7f10a01044f0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:10.811 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.809+0000 7f10a63c9640 1 -- 172.21.15.160:0/415826257 >> 172.21.15.160:0/415826257 conn(0x7f10a00fbaf0 msgr2=0x7f10a00fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:44:10.811 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.809+0000 7f10a63c9640 1 -- 172.21.15.160:0/415826257 shutdown_connections 2024-03-07T07:44:10.812 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.809+0000 7f10a63c9640 1 -- 172.21.15.160:0/415826257 wait complete. 2024-03-07T07:44:10.812 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.809+0000 7f10a63c9640 1 Processor -- start 2024-03-07T07:44:10.812 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.809+0000 7f10a63c9640 1 -- start start 2024-03-07T07:44:10.812 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.810+0000 7f10a63c9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10a0102100 0x7f10a0197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:44:10.812 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.810+0000 7f10a63c9640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10a0104a30 0x7f10a0197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:44:10.812 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.810+0000 7f10a63c9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10a0107360 0x7f10a019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:44:10.812 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.810+0000 7f10a63c9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a010d320 con 0x7f10a0107360 2024-03-07T07:44:10.812 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.810+0000 7f10a63c9640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a010d1a0 con 0x7f10a0104a30 2024-03-07T07:44:10.812 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.810+0000 7f10a63c9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a010d4a0 con 0x7f10a0102100 2024-03-07T07:44:10.812 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.810+0000 7f10a4966640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10a0107360 0x7f10a019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:44:10.812 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.810+0000 7f109ffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10a0102100 0x7f10a0197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:44:10.812 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.810+0000 7f10a4966640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10a0107360 0x7f10a019ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:36934/0 (socket says 172.21.15.160:36934) 2024-03-07T07:44:10.812 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.810+0000 7f109f7fe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10a0104a30 0x7f10a0197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:44:10.812 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.810+0000 7f109ffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10a0102100 0x7f10a0197990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.160:47418/0 (socket says 172.21.15.160:47418) 2024-03-07T07:44:10.813 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.810+0000 7f10a4966640 1 -- 172.21.15.160:0/2977333709 learned_addr learned my addr 172.21.15.160:0/2977333709 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:44:10.813 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.810+0000 7f109ffff640 1 -- 172.21.15.160:0/2977333709 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10a0104a30 msgr2=0x7f10a0197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:10.813 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.810+0000 7f109ffff640 1 --2- 172.21.15.160:0/2977333709 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10a0104a30 0x7f10a0197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:10.813 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.810+0000 7f109ffff640 1 -- 172.21.15.160:0/2977333709 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10a0107360 msgr2=0x7f10a019ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:10.813 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.810+0000 7f109ffff640 1 --2- 172.21.15.160:0/2977333709 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10a0107360 0x7f10a019ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:10.813 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.810+0000 7f109ffff640 1 -- 172.21.15.160:0/2977333709 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a019f5b0 con 0x7f10a0102100 2024-03-07T07:44:10.813 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.810+0000 7f10a4966640 1 --2- 172.21.15.160:0/2977333709 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10a0107360 0x7f10a019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:44:10.813 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.810+0000 7f109f7fe640 1 --2- 172.21.15.160:0/2977333709 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10a0104a30 0x7f10a0197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:44:10.813 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.810+0000 7f109ffff640 1 --2- 172.21.15.160:0/2977333709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10a0102100 0x7f10a0197990 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1090009230 tx=0x7f109002c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:44:10.813 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.811+0000 7f109d7fa640 1 -- 172.21.15.160:0/2977333709 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1090041070 con 0x7f10a0102100 2024-03-07T07:44:10.813 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.811+0000 7f109d7fa640 1 -- 172.21.15.160:0/2977333709 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1090004450 con 0x7f10a0102100 2024-03-07T07:44:10.813 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.811+0000 7f10a63c9640 1 -- 172.21.15.160:0/2977333709 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10a019f840 con 0x7f10a0102100 2024-03-07T07:44:10.813 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.811+0000 7f109d7fa640 1 -- 172.21.15.160:0/2977333709 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1090003df0 con 0x7f10a0102100 2024-03-07T07:44:10.813 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.811+0000 7f10a63c9640 1 -- 172.21.15.160:0/2977333709 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10a019fd30 con 0x7f10a0102100 2024-03-07T07:44:10.814 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.811+0000 7f10a63c9640 1 -- 172.21.15.160:0/2977333709 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10a00fd5e0 con 0x7f10a0102100 2024-03-07T07:44:10.814 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.813+0000 7f109d7fa640 1 -- 172.21.15.160:0/2977333709 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 20) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f1090044070 con 0x7f10a0102100 2024-03-07T07:44:10.814 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.813+0000 7f109d7fa640 1 --2- 172.21.15.160:0/2977333709 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f10880a1d80 0x7f10880a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:44:10.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.813+0000 7f109f7fe640 1 --2- 172.21.15.160:0/2977333709 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f10880a1d80 0x7f10880a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:44:10.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.813+0000 7f109d7fa640 1 -- 172.21.15.160:0/2977333709 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f10900ef750 con 0x7f10a0102100 2024-03-07T07:44:10.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.814+0000 7f109f7fe640 1 --2- 172.21.15.160:0/2977333709 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f10880a1d80 0x7f10880a4240 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f10a0198ea0 tx=0x7f1094005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:44:10.816 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:10.816+0000 7f109d7fa640 1 -- 172.21.15.160:0/2977333709 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f10900bc530 con 0x7f10a0102100 2024-03-07T07:44:10.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:10 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/1938052845' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-07T07:44:10.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:10 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/2771793775' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-07T07:44:10.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:10 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/3586718391' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-07T07:44:10.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:10 smithi177 ceph-mon[36578]: pgmap v438: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-07T07:44:11.027 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.027+0000 7f10a63c9640 1 -- 172.21.15.160:0/2977333709 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f10a002d030 con 0x7f10a0102100 2024-03-07T07:44:11.028 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.028+0000 7f109d7fa640 1 -- 172.21.15.160:0/2977333709 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11411 (secure 0 0 0) 0x7f10900c05d0 con 0x7f10a0102100 2024-03-07T07:44:11.028 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:44:11.028 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/2615961083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2615961083},{"type":"v1","addr":"172.21.15.160:6835","nonce":2615961083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1759006033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1759006033},{"type":"v1","addr":"172.21.15.160:6837","nonce":1759006033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6839/3763944721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":3763944721},{"type":"v1","addr":"172.21.15.160:6839","nonce":3763944721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6841/3611426637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6840","nonce":3611426637},{"type":"v1","addr":"172.21.15.160:6841","nonce":3611426637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1457364761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1457364761},{"type":"v1","addr":"172.21.15.177:6833","nonce":1457364761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/170428292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":170428292},{"type":"v1","addr":"172.21.15.177:6835","nonce":170428292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-07T07:39:14.164243+0000","modified":"2024-03-07T07:41:10.589691+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24511,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/645365846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":645365846},{"type":"v1","addr":"172.21.15.178:6833","nonce":645365846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6837/4088036516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":4088036516},{"type":"v1","addr":"172.21.15.177:6837","nonce":4088036516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/647840547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":647840547},{"type":"v1","addr":"172.21.15.178:6835","nonce":647840547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.178:6837/1247829154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1247829154},{"type":"v1","addr":"172.21.15.178:6837","nonce":1247829154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24497":{"gid":24497,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/3329773838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3329773838},{"type":"v1","addr":"172.21.15.177:6839","nonce":3329773838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24511":{"gid":24511,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.178:6839/2947354024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":2947354024},{"type":"v1","addr":"172.21.15.178:6839","nonce":2947354024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24511,"qdb_cluster":[24487,24497,24511]},"id":1}]} 2024-03-07T07:44:11.028 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 22 2024-03-07T07:44:11.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.030+0000 7f10a63c9640 1 -- 172.21.15.160:0/2977333709 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f10880a1d80 msgr2=0x7f10880a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:11.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.030+0000 7f10a63c9640 1 --2- 172.21.15.160:0/2977333709 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f10880a1d80 0x7f10880a4240 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f10a0198ea0 tx=0x7f1094005f90 comp rx=0 tx=0).stop 2024-03-07T07:44:11.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.030+0000 7f10a63c9640 1 -- 172.21.15.160:0/2977333709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10a0102100 msgr2=0x7f10a0197990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:11.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.030+0000 7f10a63c9640 1 --2- 172.21.15.160:0/2977333709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10a0102100 0x7f10a0197990 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1090009230 tx=0x7f109002c650 comp rx=0 tx=0).stop 2024-03-07T07:44:11.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.030+0000 7f10a63c9640 1 -- 172.21.15.160:0/2977333709 shutdown_connections 2024-03-07T07:44:11.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.030+0000 7f10a63c9640 1 --2- 172.21.15.160:0/2977333709 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f10880a1d80 0x7f10880a4240 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:11.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.030+0000 7f10a63c9640 1 --2- 172.21.15.160:0/2977333709 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10a0107360 0x7f10a019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:11.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.030+0000 7f10a63c9640 1 --2- 172.21.15.160:0/2977333709 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10a0104a30 0x7f10a0197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:11.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.030+0000 7f10a63c9640 1 --2- 172.21.15.160:0/2977333709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10a0102100 0x7f10a0197990 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:11.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.030+0000 7f10a63c9640 1 -- 172.21.15.160:0/2977333709 >> 172.21.15.160:0/2977333709 conn(0x7f10a00fbaf0 msgr2=0x7f10a01029a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:44:11.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.030+0000 7f10a63c9640 1 -- 172.21.15.160:0/2977333709 shutdown_connections 2024-03-07T07:44:11.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.030+0000 7f10a63c9640 1 -- 172.21.15.160:0/2977333709 wait complete. 2024-03-07T07:44:11.041 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T07:44:11.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.143+0000 7f579161a640 1 -- 172.21.15.160:0/3484281776 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f578c1017f0 msgr2=0x7f578c101c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:11.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.143+0000 7f579161a640 1 --2- 172.21.15.160:0/3484281776 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f578c1017f0 0x7f578c101c70 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5780009900 tx=0x7f578002bc30 comp rx=0 tx=0).stop 2024-03-07T07:44:11.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.143+0000 7f579161a640 1 -- 172.21.15.160:0/3484281776 shutdown_connections 2024-03-07T07:44:11.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.143+0000 7f579161a640 1 --2- 172.21.15.160:0/3484281776 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f578c1021b0 0x7f578c108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:11.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.143+0000 7f579161a640 1 --2- 172.21.15.160:0/3484281776 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f578c1017f0 0x7f578c101c70 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:11.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.143+0000 7f579161a640 1 --2- 172.21.15.160:0/3484281776 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f578c100600 0x7f578c100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:11.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.143+0000 7f579161a640 1 -- 172.21.15.160:0/3484281776 >> 172.21.15.160:0/3484281776 conn(0x7f578c0fbb90 msgr2=0x7f578c0fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:44:11.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.143+0000 7f579161a640 1 -- 172.21.15.160:0/3484281776 shutdown_connections 2024-03-07T07:44:11.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.143+0000 7f579161a640 1 -- 172.21.15.160:0/3484281776 wait complete. 2024-03-07T07:44:11.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.143+0000 7f579161a640 1 Processor -- start 2024-03-07T07:44:11.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.144+0000 7f579161a640 1 -- start start 2024-03-07T07:44:11.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.144+0000 7f579161a640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f578c100600 0x7f578c06a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:44:11.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.144+0000 7f579161a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f578c1017f0 0x7f578c19c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:44:11.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.144+0000 7f579161a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f578c1021b0 0x7f578c19ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:44:11.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.144+0000 7f579161a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578c10c220 con 0x7f578c1017f0 2024-03-07T07:44:11.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.144+0000 7f579161a640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578c10c0a0 con 0x7f578c100600 2024-03-07T07:44:11.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.144+0000 7f579161a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578c10c3a0 con 0x7f578c1021b0 2024-03-07T07:44:11.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.144+0000 7f578a7fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f578c1017f0 0x7f578c19c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:44:11.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.144+0000 7f578a7fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f578c1017f0 0x7f578c19c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:36950/0 (socket says 172.21.15.160:36950) 2024-03-07T07:44:11.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.144+0000 7f578a7fc640 1 -- 172.21.15.160:0/3519044429 learned_addr learned my addr 172.21.15.160:0/3519044429 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:44:11.149 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.144+0000 7f578affd640 1 --2- 172.21.15.160:0/3519044429 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f578c100600 0x7f578c06a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:44:11.149 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.144+0000 7f578b7fe640 1 --2- 172.21.15.160:0/3519044429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f578c1021b0 0x7f578c19ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:44:11.149 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.144+0000 7f578a7fc640 1 -- 172.21.15.160:0/3519044429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f578c1021b0 msgr2=0x7f578c19ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:11.149 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.144+0000 7f578a7fc640 1 --2- 172.21.15.160:0/3519044429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f578c1021b0 0x7f578c19ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:11.149 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.144+0000 7f578a7fc640 1 -- 172.21.15.160:0/3519044429 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f578c100600 msgr2=0x7f578c06a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:11.149 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.144+0000 7f578a7fc640 1 --2- 172.21.15.160:0/3519044429 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f578c100600 0x7f578c06a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:11.149 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.144+0000 7f578a7fc640 1 -- 172.21.15.160:0/3519044429 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f578c19f5c0 con 0x7f578c1017f0 2024-03-07T07:44:11.149 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.144+0000 7f578a7fc640 1 --2- 172.21.15.160:0/3519044429 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f578c1017f0 0x7f578c19c6c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f5780009230 tx=0x7f578002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:44:11.150 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.145+0000 7f5773fff640 1 -- 172.21.15.160:0/3519044429 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5780041070 con 0x7f578c1017f0 2024-03-07T07:44:11.150 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.145+0000 7f5773fff640 1 -- 172.21.15.160:0/3519044429 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5780004450 con 0x7f578c1017f0 2024-03-07T07:44:11.150 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.145+0000 7f5773fff640 1 -- 172.21.15.160:0/3519044429 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5780003db0 con 0x7f578c1017f0 2024-03-07T07:44:11.150 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.145+0000 7f579161a640 1 -- 172.21.15.160:0/3519044429 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f578c19f850 con 0x7f578c1017f0 2024-03-07T07:44:11.150 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.145+0000 7f579161a640 1 -- 172.21.15.160:0/3519044429 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f578c19fd40 con 0x7f578c1017f0 2024-03-07T07:44:11.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.146+0000 7f579161a640 1 -- 172.21.15.160:0/3519044429 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f578c1079c0 con 0x7f578c1017f0 2024-03-07T07:44:11.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.147+0000 7f5773fff640 1 -- 172.21.15.160:0/3519044429 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 20) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f5780044070 con 0x7f578c1017f0 2024-03-07T07:44:11.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.147+0000 7f5773fff640 1 --2- 172.21.15.160:0/3519044429 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f57740a1e00 0x7f57740a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:44:11.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.147+0000 7f578affd640 1 --2- 172.21.15.160:0/3519044429 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f57740a1e00 0x7f57740a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:44:11.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.147+0000 7f5773fff640 1 -- 172.21.15.160:0/3519044429 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f57800efc50 con 0x7f578c1017f0 2024-03-07T07:44:11.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.148+0000 7f578affd640 1 --2- 172.21.15.160:0/3519044429 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f57740a1e00 0x7f57740a42c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f578c101650 tx=0x7f577c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:44:11.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.151+0000 7f5773fff640 1 -- 172.21.15.160:0/3519044429 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f57800bd080 con 0x7f578c1017f0 2024-03-07T07:44:11.363 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.363+0000 7f579161a640 1 -- 172.21.15.160:0/3519044429 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f578c02d030 con 0x7f578c1017f0 2024-03-07T07:44:11.363 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.364+0000 7f5773fff640 1 -- 172.21.15.160:0/3519044429 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11411 (secure 0 0 0) 0x7f57800bc980 con 0x7f578c1017f0 2024-03-07T07:44:11.364 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:44:11.364 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/2615961083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2615961083},{"type":"v1","addr":"172.21.15.160:6835","nonce":2615961083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1759006033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1759006033},{"type":"v1","addr":"172.21.15.160:6837","nonce":1759006033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6839/3763944721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":3763944721},{"type":"v1","addr":"172.21.15.160:6839","nonce":3763944721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6841/3611426637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6840","nonce":3611426637},{"type":"v1","addr":"172.21.15.160:6841","nonce":3611426637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1457364761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1457364761},{"type":"v1","addr":"172.21.15.177:6833","nonce":1457364761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/170428292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":170428292},{"type":"v1","addr":"172.21.15.177:6835","nonce":170428292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-07T07:39:14.164243+0000","modified":"2024-03-07T07:41:10.589691+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24511,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/645365846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":645365846},{"type":"v1","addr":"172.21.15.178:6833","nonce":645365846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6837/4088036516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":4088036516},{"type":"v1","addr":"172.21.15.177:6837","nonce":4088036516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/647840547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":647840547},{"type":"v1","addr":"172.21.15.178:6835","nonce":647840547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.178:6837/1247829154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1247829154},{"type":"v1","addr":"172.21.15.178:6837","nonce":1247829154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24497":{"gid":24497,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/3329773838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3329773838},{"type":"v1","addr":"172.21.15.177:6839","nonce":3329773838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24511":{"gid":24511,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.178:6839/2947354024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":2947354024},{"type":"v1","addr":"172.21.15.178:6839","nonce":2947354024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24511,"qdb_cluster":[24487,24497,24511]},"id":1}]} 2024-03-07T07:44:11.364 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 22 2024-03-07T07:44:11.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.366+0000 7f579161a640 1 -- 172.21.15.160:0/3519044429 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f57740a1e00 msgr2=0x7f57740a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:11.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.366+0000 7f579161a640 1 --2- 172.21.15.160:0/3519044429 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f57740a1e00 0x7f57740a42c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f578c101650 tx=0x7f577c005f90 comp rx=0 tx=0).stop 2024-03-07T07:44:11.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.366+0000 7f579161a640 1 -- 172.21.15.160:0/3519044429 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f578c1017f0 msgr2=0x7f578c19c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:11.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.366+0000 7f579161a640 1 --2- 172.21.15.160:0/3519044429 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f578c1017f0 0x7f578c19c6c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f5780009230 tx=0x7f578002c650 comp rx=0 tx=0).stop 2024-03-07T07:44:11.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.366+0000 7f579161a640 1 -- 172.21.15.160:0/3519044429 shutdown_connections 2024-03-07T07:44:11.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.366+0000 7f579161a640 1 --2- 172.21.15.160:0/3519044429 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f57740a1e00 0x7f57740a42c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:11.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.366+0000 7f579161a640 1 --2- 172.21.15.160:0/3519044429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f578c1021b0 0x7f578c19ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:11.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.366+0000 7f579161a640 1 --2- 172.21.15.160:0/3519044429 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f578c1017f0 0x7f578c19c6c0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:11.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.366+0000 7f579161a640 1 --2- 172.21.15.160:0/3519044429 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f578c100600 0x7f578c06a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:11.367 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.366+0000 7f579161a640 1 -- 172.21.15.160:0/3519044429 >> 172.21.15.160:0/3519044429 conn(0x7f578c0fbb90 msgr2=0x7f578c104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:44:11.367 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.366+0000 7f579161a640 1 -- 172.21.15.160:0/3519044429 shutdown_connections 2024-03-07T07:44:11.367 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.366+0000 7f579161a640 1 -- 172.21.15.160:0/3519044429 wait complete. 2024-03-07T07:44:11.377 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T07:44:11.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.480+0000 7fa7866dd640 1 -- 172.21.15.160:0/1031855176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7801011f0 msgr2=0x7fa780101670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:11.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.480+0000 7fa7866dd640 1 --2- 172.21.15.160:0/1031855176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7801011f0 0x7fa780101670 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa778009ca0 tx=0x7fa77802c0f0 comp rx=0 tx=0).stop 2024-03-07T07:44:11.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.480+0000 7fa7866dd640 1 -- 172.21.15.160:0/1031855176 shutdown_connections 2024-03-07T07:44:11.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.480+0000 7fa7866dd640 1 --2- 172.21.15.160:0/1031855176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7801011f0 0x7fa780101670 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:11.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.480+0000 7fa7866dd640 1 --2- 172.21.15.160:0/1031855176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa780100850 0x7fa780100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:11.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.480+0000 7fa7866dd640 1 --2- 172.21.15.160:0/1031855176 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7800ffef0 0x7fa780100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:11.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.480+0000 7fa7866dd640 1 -- 172.21.15.160:0/1031855176 >> 172.21.15.160:0/1031855176 conn(0x7fa7800fbb10 msgr2=0x7fa7800fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:44:11.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.480+0000 7fa7866dd640 1 -- 172.21.15.160:0/1031855176 shutdown_connections 2024-03-07T07:44:11.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.480+0000 7fa7866dd640 1 -- 172.21.15.160:0/1031855176 wait complete. 2024-03-07T07:44:11.484 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.481+0000 7fa7866dd640 1 Processor -- start 2024-03-07T07:44:11.484 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.481+0000 7fa7866dd640 1 -- start start 2024-03-07T07:44:11.484 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.481+0000 7fa7866dd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7800ffef0 0x7fa780199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:44:11.484 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.481+0000 7fa7866dd640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa780100850 0x7fa78019a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:44:11.484 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.481+0000 7fa7866dd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7801011f0 0x7fa7801a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:44:11.484 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.481+0000 7fa7866dd640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78010f4f0 con 0x7fa7800ffef0 2024-03-07T07:44:11.484 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.481+0000 7fa7866dd640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78010f370 con 0x7fa780100850 2024-03-07T07:44:11.484 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.481+0000 7fa7866dd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78010f670 con 0x7fa7801011f0 2024-03-07T07:44:11.484 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.481+0000 7fa77ffff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7800ffef0 0x7fa780199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:44:11.484 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.481+0000 7fa77ffff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7800ffef0 0x7fa780199b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:36964/0 (socket says 172.21.15.160:36964) 2024-03-07T07:44:11.484 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.481+0000 7fa77ffff640 1 -- 172.21.15.160:0/3037698781 learned_addr learned my addr 172.21.15.160:0/3037698781 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:44:11.484 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.481+0000 7fa77f7fe640 1 --2- 172.21.15.160:0/3037698781 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa780100850 0x7fa78019a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:44:11.484 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.481+0000 7fa784c7a640 1 --2- 172.21.15.160:0/3037698781 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7801011f0 0x7fa7801a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:44:11.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.481+0000 7fa77ffff640 1 -- 172.21.15.160:0/3037698781 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7801011f0 msgr2=0x7fa7801a10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:11.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.481+0000 7fa77ffff640 1 --2- 172.21.15.160:0/3037698781 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7801011f0 0x7fa7801a10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:11.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.481+0000 7fa77ffff640 1 -- 172.21.15.160:0/3037698781 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa780100850 msgr2=0x7fa78019a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:11.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.481+0000 7fa77ffff640 1 --2- 172.21.15.160:0/3037698781 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa780100850 0x7fa78019a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:11.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.481+0000 7fa77ffff640 1 -- 172.21.15.160:0/3037698781 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7801a1750 con 0x7fa7800ffef0 2024-03-07T07:44:11.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.482+0000 7fa77ffff640 1 --2- 172.21.15.160:0/3037698781 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7800ffef0 0x7fa780199b30 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa77400e9f0 tx=0x7fa77400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:44:11.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.482+0000 7fa77d7fa640 1 -- 172.21.15.160:0/3037698781 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77400cd80 con 0x7fa7800ffef0 2024-03-07T07:44:11.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.482+0000 7fa77d7fa640 1 -- 172.21.15.160:0/3037698781 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa774010910 con 0x7fa7800ffef0 2024-03-07T07:44:11.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.482+0000 7fa7866dd640 1 -- 172.21.15.160:0/3037698781 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7801a1a40 con 0x7fa7800ffef0 2024-03-07T07:44:11.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.482+0000 7fa77d7fa640 1 -- 172.21.15.160:0/3037698781 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa774018990 con 0x7fa7800ffef0 2024-03-07T07:44:11.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.482+0000 7fa7866dd640 1 -- 172.21.15.160:0/3037698781 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7801a1f90 con 0x7fa7800ffef0 2024-03-07T07:44:11.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.483+0000 7fa7866dd640 1 -- 172.21.15.160:0/3037698781 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa78010add0 con 0x7fa7800ffef0 2024-03-07T07:44:11.486 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.484+0000 7fa77d7fa640 1 -- 172.21.15.160:0/3037698781 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 20) v1 ==== 136754+0+0 (secure 0 0 0) 0x7fa774010ac0 con 0x7fa7800ffef0 2024-03-07T07:44:11.486 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.484+0000 7fa77d7fa640 1 --2- 172.21.15.160:0/3037698781 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa7680a1d80 0x7fa7680a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:44:11.486 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.484+0000 7fa77f7fe640 1 --2- 172.21.15.160:0/3037698781 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa7680a1d80 0x7fa7680a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:44:11.486 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.484+0000 7fa77d7fa640 1 -- 172.21.15.160:0/3037698781 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa7740ce4b0 con 0x7fa7800ffef0 2024-03-07T07:44:11.486 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.485+0000 7fa77f7fe640 1 --2- 172.21.15.160:0/3037698781 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa7680a1d80 0x7fa7680a4240 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa78019b040 tx=0x7fa770005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:44:11.488 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.488+0000 7fa77d7fa640 1 -- 172.21.15.160:0/3037698781 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa77409b290 con 0x7fa7800ffef0 2024-03-07T07:44:11.664 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.664+0000 7fa7866dd640 1 -- 172.21.15.160:0/3037698781 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa78002d030 con 0x7fa7800ffef0 2024-03-07T07:44:11.665 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.665+0000 7fa77d7fa640 1 -- 172.21.15.160:0/3037698781 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22721 (secure 0 0 0) 0x7fa77409f330 con 0x7fa7800ffef0 2024-03-07T07:44:11.666 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-03-07T07:44:11.666 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":87,"fsid":"826f6582-dc54-11ee-95c6-87774f69a715","created":"2024-03-07T07:31:14.229288+0000","modified":"2024-03-07T07:40:09.354265+0000","last_up_change":"2024-03-07T07:38:20.776225+0000","last_in_change":"2024-03-07T07:38:00.508917+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T07:34:34.529424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T07:39:12.075432+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T07:39:13.367820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bee66c67-fb6e-4717-9685-ca62587d5f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6803","nonce":1368482926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6805","nonce":1368482926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6809","nonce":1368482926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1368482926},{"type":"v1","addr":"172.21.15.160:6807","nonce":1368482926}]},"public_addr":"172.21.15.160:6803/1368482926","cluster_addr":"172.21.15.160:6805/1368482926","heartbeat_back_addr":"172.21.15.160:6809/1368482926","heartbeat_front_addr":"172.21.15.160:6807/1368482926","state":["exists","up"]},{"osd":1,"uuid":"49236251-f750-424e-992b-bddf55762556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6801","nonce":2922391884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6803","nonce":2922391884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6807","nonce":2922391884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2922391884},{"type":"v1","addr":"172.21.15.177:6805","nonce":2922391884}]},"public_addr":"172.21.15.177:6801/2922391884","cluster_addr":"172.21.15.177:6803/2922391884","heartbeat_back_addr":"172.21.15.177:6807/2922391884","heartbeat_front_addr":"172.21.15.177:6805/2922391884","state":["exists","up"]},{"osd":2,"uuid":"7a432b28-f7ee-43fe-9196-7aaeccfecf77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6801","nonce":1217901551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6803","nonce":1217901551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6807","nonce":1217901551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1217901551},{"type":"v1","addr":"172.21.15.178:6805","nonce":1217901551}]},"public_addr":"172.21.15.178:6801/1217901551","cluster_addr":"172.21.15.178:6803/1217901551","heartbeat_back_addr":"172.21.15.178:6807/1217901551","heartbeat_front_addr":"172.21.15.178:6805/1217901551","state":["exists","up"]},{"osd":3,"uuid":"4ad22a20-f1be-4002-a48d-ecd3953e63f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6811","nonce":42543989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6813","nonce":42543989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6817","nonce":42543989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":42543989},{"type":"v1","addr":"172.21.15.160:6815","nonce":42543989}]},"public_addr":"172.21.15.160:6811/42543989","cluster_addr":"172.21.15.160:6813/42543989","heartbeat_back_addr":"172.21.15.160:6817/42543989","heartbeat_front_addr":"172.21.15.160:6815/42543989","state":["exists","up"]},{"osd":4,"uuid":"2b65b104-ac82-4ce3-b6c5-11a399103045","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6809","nonce":88178553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6811","nonce":88178553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6815","nonce":88178553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":88178553},{"type":"v1","addr":"172.21.15.177:6813","nonce":88178553}]},"public_addr":"172.21.15.177:6809/88178553","cluster_addr":"172.21.15.177:6811/88178553","heartbeat_back_addr":"172.21.15.177:6815/88178553","heartbeat_front_addr":"172.21.15.177:6813/88178553","state":["exists","up"]},{"osd":5,"uuid":"30a1e714-08ff-44e8-9264-9ef164604b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6809","nonce":964732891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6811","nonce":964732891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6815","nonce":964732891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":964732891},{"type":"v1","addr":"172.21.15.178:6813","nonce":964732891}]},"public_addr":"172.21.15.178:6809/964732891","cluster_addr":"172.21.15.178:6811/964732891","heartbeat_back_addr":"172.21.15.178:6815/964732891","heartbeat_front_addr":"172.21.15.178:6813/964732891","state":["exists","up"]},{"osd":6,"uuid":"b1e1646e-f8a9-42b8-b62d-9d7b5d6a1ff4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6819","nonce":3251327001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6821","nonce":3251327001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6825","nonce":3251327001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3251327001},{"type":"v1","addr":"172.21.15.160:6823","nonce":3251327001}]},"public_addr":"172.21.15.160:6819/3251327001","cluster_addr":"172.21.15.160:6821/3251327001","heartbeat_back_addr":"172.21.15.160:6825/3251327001","heartbeat_front_addr":"172.21.15.160:6823/3251327001","state":["exists","up"]},{"osd":7,"uuid":"e66b400b-9747-4f6d-b05b-db22468e2750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6817","nonce":1175500850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6819","nonce":1175500850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6823","nonce":1175500850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1175500850},{"type":"v1","addr":"172.21.15.177:6821","nonce":1175500850}]},"public_addr":"172.21.15.177:6817/1175500850","cluster_addr":"172.21.15.177:6819/1175500850","heartbeat_back_addr":"172.21.15.177:6823/1175500850","heartbeat_front_addr":"172.21.15.177:6821/1175500850","state":["exists","up"]},{"osd":8,"uuid":"3ee0f2c4-fca4-49fc-a98b-07aa0fcba0e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6817","nonce":688015531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6819","nonce":688015531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6823","nonce":688015531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":688015531},{"type":"v1","addr":"172.21.15.178:6821","nonce":688015531}]},"public_addr":"172.21.15.178:6817/688015531","cluster_addr":"172.21.15.178:6819/688015531","heartbeat_back_addr":"172.21.15.178:6823/688015531","heartbeat_front_addr":"172.21.15.178:6821/688015531","state":["exists","up"]},{"osd":9,"uuid":"856039d7-f7ee-4dbd-b019-0073e705ce0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6827","nonce":3265797851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6829","nonce":3265797851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6833","nonce":3265797851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":3265797851},{"type":"v1","addr":"172.21.15.160:6831","nonce":3265797851}]},"public_addr":"172.21.15.160:6827/3265797851","cluster_addr":"172.21.15.160:6829/3265797851","heartbeat_back_addr":"172.21.15.160:6833/3265797851","heartbeat_front_addr":"172.21.15.160:6831/3265797851","state":["exists","up"]},{"osd":10,"uuid":"bdc9899d-d720-4429-ad53-4c388a4dbe75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6825","nonce":2867473412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6827","nonce":2867473412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6831","nonce":2867473412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":2867473412},{"type":"v1","addr":"172.21.15.177:6829","nonce":2867473412}]},"public_addr":"172.21.15.177:6825/2867473412","cluster_addr":"172.21.15.177:6827/2867473412","heartbeat_back_addr":"172.21.15.177:6831/2867473412","heartbeat_front_addr":"172.21.15.177:6829/2867473412","state":["exists","up"]},{"osd":11,"uuid":"cc9adb54-ef49-47c6-90e3-fc03bc0f5cda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6825","nonce":4068724731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6827","nonce":4068724731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6831","nonce":4068724731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":4068724731},{"type":"v1","addr":"172.21.15.178:6829","nonce":4068724731}]},"public_addr":"172.21.15.178:6825/4068724731","cluster_addr":"172.21.15.178:6827/4068724731","heartbeat_back_addr":"172.21.15.178:6831/4068724731","heartbeat_front_addr":"172.21.15.178:6829/4068724731","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:33:44.141585+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:07.685585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:31.044386+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:34:56.308925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:35:21.582097+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:35:45.983540+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:36:11.530242+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:36:36.844681+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:02.303437+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:27.629867+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:37:53.289495+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:38:18.536169+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/3027450612":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/874709722":"2024-03-08T07:32:08.190837+0000","172.21.15.160:6800/1130675225":"2024-03-08T07:31:41.166858+0000","172.21.15.160:0/3857193980":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/3861968297":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6800/3859174315":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/3013534743":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6801/3859174315":"2024-03-08T07:32:08.190837+0000","172.21.15.160:0/1310937667":"2024-03-08T07:31:41.166858+0000","172.21.15.160:6801/1130675225":"2024-03-08T07:31:41.166858+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T07:44:11.668 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.668+0000 7fa7866dd640 1 -- 172.21.15.160:0/3037698781 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa7680a1d80 msgr2=0x7fa7680a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:11.668 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.668+0000 7fa7866dd640 1 --2- 172.21.15.160:0/3037698781 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa7680a1d80 0x7fa7680a4240 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa78019b040 tx=0x7fa770005f90 comp rx=0 tx=0).stop 2024-03-07T07:44:11.668 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.668+0000 7fa7866dd640 1 -- 172.21.15.160:0/3037698781 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7800ffef0 msgr2=0x7fa780199b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:11.668 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.668+0000 7fa7866dd640 1 --2- 172.21.15.160:0/3037698781 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7800ffef0 0x7fa780199b30 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa77400e9f0 tx=0x7fa77400eeb0 comp rx=0 tx=0).stop 2024-03-07T07:44:11.668 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.668+0000 7fa7866dd640 1 -- 172.21.15.160:0/3037698781 shutdown_connections 2024-03-07T07:44:11.668 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.668+0000 7fa7866dd640 1 --2- 172.21.15.160:0/3037698781 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7fa7680a1d80 0x7fa7680a4240 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:11.669 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.668+0000 7fa7866dd640 1 --2- 172.21.15.160:0/3037698781 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7801011f0 0x7fa7801a10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:11.669 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.668+0000 7fa7866dd640 1 --2- 172.21.15.160:0/3037698781 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa780100850 0x7fa78019a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:11.669 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.668+0000 7fa7866dd640 1 --2- 172.21.15.160:0/3037698781 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7800ffef0 0x7fa780199b30 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:11.669 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.668+0000 7fa7866dd640 1 -- 172.21.15.160:0/3037698781 >> 172.21.15.160:0/3037698781 conn(0x7fa7800fbb10 msgr2=0x7fa7800fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:44:11.669 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.668+0000 7fa7866dd640 1 -- 172.21.15.160:0/3037698781 shutdown_connections 2024-03-07T07:44:11.669 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.668+0000 7fa7866dd640 1 -- 172.21.15.160:0/3037698781 wait complete. 2024-03-07T07:44:11.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:11 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/2977333709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:44:11.677 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:11 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/3519044429' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:44:11.680 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-07T07:44:11.786 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.783+0000 7f2c9b687640 1 -- 172.21.15.160:0/1512637991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c94104a10 msgr2=0x7f2c94106e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:11.786 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.783+0000 7f2c9b687640 1 --2- 172.21.15.160:0/1512637991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c94104a10 0x7f2c94106e00 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2c84009900 tx=0x7f2c8402bc30 comp rx=0 tx=0).stop 2024-03-07T07:44:11.786 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.783+0000 7f2c9b687640 1 -- 172.21.15.160:0/1512637991 shutdown_connections 2024-03-07T07:44:11.786 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.783+0000 7f2c9b687640 1 --2- 172.21.15.160:0/1512637991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c94107340 0x7f2c94109790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:11.786 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.783+0000 7f2c9b687640 1 --2- 172.21.15.160:0/1512637991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c94104a10 0x7f2c94106e00 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:11.786 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.783+0000 7f2c9b687640 1 --2- 172.21.15.160:0/1512637991 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c941020e0 0x7f2c941044d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:11.786 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.783+0000 7f2c9b687640 1 -- 172.21.15.160:0/1512637991 >> 172.21.15.160:0/1512637991 conn(0x7f2c940fbaf0 msgr2=0x7f2c940fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:44:11.786 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.783+0000 7f2c9b687640 1 -- 172.21.15.160:0/1512637991 shutdown_connections 2024-03-07T07:44:11.786 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.783+0000 7f2c9b687640 1 -- 172.21.15.160:0/1512637991 wait complete. 2024-03-07T07:44:11.787 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.784+0000 7f2c9b687640 1 Processor -- start 2024-03-07T07:44:11.787 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.784+0000 7f2c9b687640 1 -- start start 2024-03-07T07:44:11.787 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.784+0000 7f2c9b687640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c941020e0 0x7f2c941979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:44:11.787 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.784+0000 7f2c9b687640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c94104a10 0x7f2c94197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:44:11.787 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.784+0000 7f2c9b687640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c94107340 0x7f2c9419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:44:11.787 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.784+0000 7f2c9b687640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9410d2d0 con 0x7f2c941020e0 2024-03-07T07:44:11.787 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.784+0000 7f2c9b687640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9410d150 con 0x7f2c94104a10 2024-03-07T07:44:11.787 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.784+0000 7f2c9b687640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9410d450 con 0x7f2c94107340 2024-03-07T07:44:11.787 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.784+0000 7f2c99423640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c941020e0 0x7f2c941979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:44:11.787 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.784+0000 7f2c99423640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c941020e0 0x7f2c941979d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:36984/0 (socket says 172.21.15.160:36984) 2024-03-07T07:44:11.787 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.784+0000 7f2c99423640 1 -- 172.21.15.160:0/4242414636 learned_addr learned my addr 172.21.15.160:0/4242414636 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:44:11.787 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.784+0000 7f2c99c24640 1 --2- 172.21.15.160:0/4242414636 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c94107340 0x7f2c9419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:44:11.787 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.784+0000 7f2c98c22640 1 --2- 172.21.15.160:0/4242414636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c94104a10 0x7f2c94197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:44:11.787 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.785+0000 7f2c99423640 1 -- 172.21.15.160:0/4242414636 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c94107340 msgr2=0x7f2c9419ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:11.788 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.785+0000 7f2c99423640 1 --2- 172.21.15.160:0/4242414636 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c94107340 0x7f2c9419ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:11.788 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.785+0000 7f2c99423640 1 -- 172.21.15.160:0/4242414636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c94104a10 msgr2=0x7f2c94197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:11.788 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.785+0000 7f2c99423640 1 --2- 172.21.15.160:0/4242414636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c94104a10 0x7f2c94197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:11.788 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.785+0000 7f2c99423640 1 -- 172.21.15.160:0/4242414636 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c9419f5f0 con 0x7f2c941020e0 2024-03-07T07:44:11.788 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.785+0000 7f2c99423640 1 --2- 172.21.15.160:0/4242414636 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c941020e0 0x7f2c941979d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2c9000ebc0 tx=0x7f2c9000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:44:11.788 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.785+0000 7f2c8a7fc640 1 -- 172.21.15.160:0/4242414636 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c90010630 con 0x7f2c941020e0 2024-03-07T07:44:11.788 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.785+0000 7f2c8a7fc640 1 -- 172.21.15.160:0/4242414636 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c90010cb0 con 0x7f2c941020e0 2024-03-07T07:44:11.788 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.785+0000 7f2c8a7fc640 1 -- 172.21.15.160:0/4242414636 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c90018bd0 con 0x7f2c941020e0 2024-03-07T07:44:11.788 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.785+0000 7f2c9b687640 1 -- 172.21.15.160:0/4242414636 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c9419f8e0 con 0x7f2c941020e0 2024-03-07T07:44:11.788 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.785+0000 7f2c9b687640 1 -- 172.21.15.160:0/4242414636 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c9419fe00 con 0x7f2c941020e0 2024-03-07T07:44:11.788 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.786+0000 7f2c9b687640 1 -- 172.21.15.160:0/4242414636 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c940fd5e0 con 0x7f2c941020e0 2024-03-07T07:44:11.789 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.787+0000 7f2c8a7fc640 1 -- 172.21.15.160:0/4242414636 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 20) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f2c900107d0 con 0x7f2c941020e0 2024-03-07T07:44:11.789 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.788+0000 7f2c8a7fc640 1 --2- 172.21.15.160:0/4242414636 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2c800a1d80 0x7f2c800a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:44:11.789 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.788+0000 7f2c98c22640 1 --2- 172.21.15.160:0/4242414636 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2c800a1d80 0x7f2c800a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:44:11.789 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.788+0000 7f2c8a7fc640 1 -- 172.21.15.160:0/4242414636 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2c900ce550 con 0x7f2c941020e0 2024-03-07T07:44:11.789 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.788+0000 7f2c98c22640 1 --2- 172.21.15.160:0/4242414636 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2c800a1d80 0x7f2c800a4240 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2c84000c00 tx=0x7f2c84005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:44:11.791 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:11.791+0000 7f2c8a7fc640 1 -- 172.21.15.160:0/4242414636 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f2c9009b330 con 0x7f2c941020e0 2024-03-07T07:44:11.805 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:11 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/2977333709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:44:11.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:11 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/3519044429' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:44:11.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:11 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/2977333709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:44:11.856 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:11 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/3519044429' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-07T07:44:12.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.000+0000 7f2c9b687640 1 -- 172.21.15.160:0/4242414636 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f2c9402d030 con 0x7f2c941020e0 2024-03-07T07:44:12.002 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.002+0000 7f2c8a7fc640 1 -- 172.21.15.160:0/4242414636 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v23) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2c9009f3d0 con 0x7f2c941020e0 2024-03-07T07:44:12.002 INFO:teuthology.orchestra.run.smithi160.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-07T07:44:12.005 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.005+0000 7f2c9b687640 1 -- 172.21.15.160:0/4242414636 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2c800a1d80 msgr2=0x7f2c800a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:12.005 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.005+0000 7f2c9b687640 1 --2- 172.21.15.160:0/4242414636 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2c800a1d80 0x7f2c800a4240 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2c84000c00 tx=0x7f2c84005fd0 comp rx=0 tx=0).stop 2024-03-07T07:44:12.005 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.005+0000 7f2c9b687640 1 -- 172.21.15.160:0/4242414636 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c941020e0 msgr2=0x7f2c941979d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:12.005 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.005+0000 7f2c9b687640 1 --2- 172.21.15.160:0/4242414636 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c941020e0 0x7f2c941979d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2c9000ebc0 tx=0x7f2c9000c6a0 comp rx=0 tx=0).stop 2024-03-07T07:44:12.005 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.005+0000 7f2c9b687640 1 -- 172.21.15.160:0/4242414636 shutdown_connections 2024-03-07T07:44:12.005 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.005+0000 7f2c9b687640 1 --2- 172.21.15.160:0/4242414636 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f2c800a1d80 0x7f2c800a4240 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:12.006 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.005+0000 7f2c9b687640 1 --2- 172.21.15.160:0/4242414636 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c94107340 0x7f2c9419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:12.006 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.005+0000 7f2c9b687640 1 --2- 172.21.15.160:0/4242414636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c94104a10 0x7f2c94197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:12.006 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.005+0000 7f2c9b687640 1 --2- 172.21.15.160:0/4242414636 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c941020e0 0x7f2c941979d0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:12.006 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.005+0000 7f2c9b687640 1 -- 172.21.15.160:0/4242414636 >> 172.21.15.160:0/4242414636 conn(0x7f2c940fbaf0 msgr2=0x7f2c94102ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:44:12.006 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.005+0000 7f2c9b687640 1 -- 172.21.15.160:0/4242414636 shutdown_connections 2024-03-07T07:44:12.006 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.005+0000 7f2c9b687640 1 -- 172.21.15.160:0/4242414636 wait complete. 2024-03-07T07:44:12.016 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-07T07:44:12.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.122+0000 7f25df06b640 1 -- 172.21.15.160:0/911503800 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25d8102860 msgr2=0x7f25d8102ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:12.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.122+0000 7f25df06b640 1 --2- 172.21.15.160:0/911503800 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25d8102860 0x7f25d8102ce0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f25c4009900 tx=0x7f25c402bc30 comp rx=0 tx=0).stop 2024-03-07T07:44:12.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.122+0000 7f25df06b640 1 -- 172.21.15.160:0/911503800 shutdown_connections 2024-03-07T07:44:12.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.122+0000 7f25df06b640 1 --2- 172.21.15.160:0/911503800 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f25d8103220 0x7f25d8109ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:12.127 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.122+0000 7f25df06b640 1 --2- 172.21.15.160:0/911503800 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25d8102860 0x7f25d8102ce0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:12.127 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.122+0000 7f25df06b640 1 --2- 172.21.15.160:0/911503800 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25d8101670 0x7f25d8101a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:12.127 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.122+0000 7f25df06b640 1 -- 172.21.15.160:0/911503800 >> 172.21.15.160:0/911503800 conn(0x7f25d80fcc00 msgr2=0x7f25d80ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:44:12.127 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.122+0000 7f25df06b640 1 -- 172.21.15.160:0/911503800 shutdown_connections 2024-03-07T07:44:12.127 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.122+0000 7f25df06b640 1 -- 172.21.15.160:0/911503800 wait complete. 2024-03-07T07:44:12.127 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.122+0000 7f25df06b640 1 Processor -- start 2024-03-07T07:44:12.127 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.122+0000 7f25df06b640 1 -- start start 2024-03-07T07:44:12.128 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.123+0000 7f25df06b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25d8101670 0x7f25d8197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:44:12.128 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.123+0000 7f25df06b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25d8102860 0x7f25d8197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:44:12.128 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.123+0000 7f25df06b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f25d8103220 0x7f25d819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:44:12.128 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.123+0000 7f25df06b640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d810d300 con 0x7f25d8102860 2024-03-07T07:44:12.128 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.123+0000 7f25df06b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d810d180 con 0x7f25d8103220 2024-03-07T07:44:12.128 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.123+0000 7f25df06b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d810d480 con 0x7f25d8101670 2024-03-07T07:44:12.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.123+0000 7f25d7fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25d8102860 0x7f25d8197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:44:12.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.123+0000 7f25d7fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25d8102860 0x7f25d8197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:37002/0 (socket says 172.21.15.160:37002) 2024-03-07T07:44:12.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.123+0000 7f25dce07640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25d8101670 0x7f25d8197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:44:12.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.123+0000 7f25d7fff640 1 -- 172.21.15.160:0/4085636220 learned_addr learned my addr 172.21.15.160:0/4085636220 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:44:12.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.123+0000 7f25dd608640 1 --2- 172.21.15.160:0/4085636220 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f25d8103220 0x7f25d819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:44:12.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.123+0000 7f25d7fff640 1 -- 172.21.15.160:0/4085636220 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25d8101670 msgr2=0x7f25d8197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:12.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.123+0000 7f25d7fff640 1 --2- 172.21.15.160:0/4085636220 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25d8101670 0x7f25d8197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:12.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.123+0000 7f25d7fff640 1 -- 172.21.15.160:0/4085636220 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f25d8103220 msgr2=0x7f25d819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:12.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.123+0000 7f25d7fff640 1 --2- 172.21.15.160:0/4085636220 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f25d8103220 0x7f25d819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:12.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.123+0000 7f25d7fff640 1 -- 172.21.15.160:0/4085636220 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d819f5b0 con 0x7f25d8102860 2024-03-07T07:44:12.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.123+0000 7f25dce07640 1 --2- 172.21.15.160:0/4085636220 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25d8101670 0x7f25d8197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:44:12.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.123+0000 7f25d7fff640 1 --2- 172.21.15.160:0/4085636220 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25d8102860 0x7f25d8197ed0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f25c4009230 tx=0x7f25c402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:44:12.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.123+0000 7f25d5ffb640 1 -- 172.21.15.160:0/4085636220 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c4041070 con 0x7f25d8102860 2024-03-07T07:44:12.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.123+0000 7f25d5ffb640 1 -- 172.21.15.160:0/4085636220 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f25c4004450 con 0x7f25d8102860 2024-03-07T07:44:12.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.123+0000 7f25df06b640 1 -- 172.21.15.160:0/4085636220 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25d819f840 con 0x7f25d8102860 2024-03-07T07:44:12.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.123+0000 7f25d5ffb640 1 -- 172.21.15.160:0/4085636220 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c4003e20 con 0x7f25d8102860 2024-03-07T07:44:12.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.124+0000 7f25df06b640 1 -- 172.21.15.160:0/4085636220 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25d819fd80 con 0x7f25d8102860 2024-03-07T07:44:12.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.124+0000 7f25df06b640 1 -- 172.21.15.160:0/4085636220 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25d8108a30 con 0x7f25d8102860 2024-03-07T07:44:12.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.126+0000 7f25d5ffb640 1 -- 172.21.15.160:0/4085636220 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 20) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f25c4044070 con 0x7f25d8102860 2024-03-07T07:44:12.132 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.126+0000 7f25d5ffb640 1 --2- 172.21.15.160:0/4085636220 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f25c80a1e00 0x7f25c80a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:44:12.132 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.126+0000 7f25dce07640 1 --2- 172.21.15.160:0/4085636220 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f25c80a1e00 0x7f25c80a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:44:12.132 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.126+0000 7f25d5ffb640 1 -- 172.21.15.160:0/4085636220 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f25c40efd30 con 0x7f25d8102860 2024-03-07T07:44:12.132 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.127+0000 7f25dce07640 1 --2- 172.21.15.160:0/4085636220 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f25c80a1e00 0x7f25c80a42c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f25d81026c0 tx=0x7f25cc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:44:12.132 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.130+0000 7f25d5ffb640 1 -- 172.21.15.160:0/4085636220 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f25c40bca00 con 0x7f25d8102860 2024-03-07T07:44:12.306 INFO:journalctl@ceph.mds.i.smithi178.stdout:Mar 07 07:44:12 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-i[68618]: ignoring --setuser ceph since I am not root 2024-03-07T07:44:12.306 INFO:journalctl@ceph.mds.i.smithi178.stdout:Mar 07 07:44:12 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-i[68618]: ignoring --setgroup ceph since I am not root 2024-03-07T07:44:12.306 INFO:journalctl@ceph.mds.i.smithi178.stdout:Mar 07 07:44:12 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-i[68618]: starting mds.i at 2024-03-07T07:44:12.306 INFO:journalctl@ceph.mds.l.smithi178.stdout:Mar 07 07:44:12 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-l[67685]: ignoring --setuser ceph since I am not root 2024-03-07T07:44:12.307 INFO:journalctl@ceph.mds.l.smithi178.stdout:Mar 07 07:44:12 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-l[67685]: ignoring --setgroup ceph since I am not root 2024-03-07T07:44:12.307 INFO:journalctl@ceph.mds.l.smithi178.stdout:Mar 07 07:44:12 smithi178 ceph-mds[67689]: -- 172.21.15.178:0/3504937122 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ea04847a40 con 0x55ea0481a800 2024-03-07T07:44:12.307 INFO:journalctl@ceph.mds.l.smithi178.stdout:Mar 07 07:44:12 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-l[67685]: starting mds.l at 2024-03-07T07:44:12.307 INFO:journalctl@ceph.mds.c.smithi178.stdout:Mar 07 07:44:12 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-c[68313]: ignoring --setuser ceph since I am not root 2024-03-07T07:44:12.307 INFO:journalctl@ceph.mds.c.smithi178.stdout:Mar 07 07:44:12 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-c[68313]: ignoring --setgroup ceph since I am not root 2024-03-07T07:44:12.307 INFO:journalctl@ceph.mds.c.smithi178.stdout:Mar 07 07:44:12 smithi178 ceph-mds[68317]: -- 172.21.15.178:0/1255393918 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55caf552da40 con 0x55caf5501000 2024-03-07T07:44:12.307 INFO:journalctl@ceph.mds.c.smithi178.stdout:Mar 07 07:44:12 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-c[68313]: starting mds.c at 2024-03-07T07:44:12.307 INFO:journalctl@ceph.mds.f.smithi178.stdout:Mar 07 07:44:12 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-f[68003]: ignoring --setuser ceph since I am not root 2024-03-07T07:44:12.308 INFO:journalctl@ceph.mds.f.smithi178.stdout:Mar 07 07:44:12 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-f[68003]: ignoring --setgroup ceph since I am not root 2024-03-07T07:44:12.308 INFO:journalctl@ceph.mds.f.smithi178.stdout:Mar 07 07:44:12 smithi178 ceph-mds[68007]: -- 172.21.15.178:0/550968334 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555f964fda40 con 0x555f964d0800 2024-03-07T07:44:12.308 INFO:journalctl@ceph.mds.f.smithi178.stdout:Mar 07 07:44:12 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-f[68003]: starting mds.f at 2024-03-07T07:44:12.308 INFO:journalctl@ceph.mds.f.smithi178.stdout:Mar 07 07:44:12 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-f[68003]: 2024-03-07T07:44:12.058+0000 7ff2ff00d700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6833/0: (98) Address already in use 2024-03-07T07:44:12.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:12.344+0000 7f25df06b640 1 -- 172.21.15.160:0/4085636220 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f25d802d280 con 0x7f25d8102860 2024-03-07T07:44:12.357 INFO:journalctl@ceph.mds.b.smithi177.stdout:Mar 07 07:44:12 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-b[67020]: ignoring --setuser ceph since I am not root 2024-03-07T07:44:12.357 INFO:journalctl@ceph.mds.b.smithi177.stdout:Mar 07 07:44:12 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-b[67020]: ignoring --setgroup ceph since I am not root 2024-03-07T07:44:12.357 INFO:journalctl@ceph.mds.b.smithi177.stdout:Mar 07 07:44:12 smithi177 ceph-mds[67024]: -- 172.21.15.177:0/4273649475 <== mon.2 v2:172.21.15.177:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c168aa3a40 con 0x55c168a76800 2024-03-07T07:44:12.357 INFO:journalctl@ceph.mds.b.smithi177.stdout:Mar 07 07:44:12 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-b[67020]: starting mds.b at 2024-03-07T07:44:12.358 INFO:journalctl@ceph.mds.h.smithi177.stdout:Mar 07 07:44:12 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-h[66705]: ignoring --setuser ceph since I am not root 2024-03-07T07:44:12.358 INFO:journalctl@ceph.mds.h.smithi177.stdout:Mar 07 07:44:12 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-h[66705]: ignoring --setgroup ceph since I am not root 2024-03-07T07:44:12.358 INFO:journalctl@ceph.mds.h.smithi177.stdout:Mar 07 07:44:12 smithi177 ceph-mds[66709]: -- 172.21.15.177:0/4153289680 <== mon.2 v2:172.21.15.177:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557153e9ba40 con 0x557153e6e800 2024-03-07T07:44:12.358 INFO:journalctl@ceph.mds.h.smithi177.stdout:Mar 07 07:44:12 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-h[66705]: starting mds.h at 2024-03-07T07:44:12.427 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:12 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mon-a[31959]: 2024-03-07T07:44:12.001+0000 7fc700158700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-07T07:44:12.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:12 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/3037698781' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:44:12.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:12 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/4242414636' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-07T07:44:12.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:12 smithi178 ceph-mon[36650]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-07T07:44:12.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:12 smithi178 ceph-mon[36650]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-07T07:44:12.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:12 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/4242414636' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-07T07:44:12.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:12 smithi178 ceph-mon[36650]: osdmap e88: 12 total, 12 up, 12 in 2024-03-07T07:44:12.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:12 smithi178 ceph-mon[36650]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-03-07T07:44:12.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:12 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/4085636220' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-07T07:44:12.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:12 smithi178 ceph-mon[36650]: pgmap v440: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-07T07:44:12.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:12 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/3037698781' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:44:12.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:12 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/4242414636' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-07T07:44:12.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:12 smithi177 ceph-mon[36578]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-07T07:44:12.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:12 smithi177 ceph-mon[36578]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-07T07:44:12.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:12 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/4242414636' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-07T07:44:12.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:12 smithi177 ceph-mon[36578]: osdmap e88: 12 total, 12 up, 12 in 2024-03-07T07:44:12.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:12 smithi177 ceph-mon[36578]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-03-07T07:44:12.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:12 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/4085636220' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-07T07:44:12.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:12 smithi177 ceph-mon[36578]: pgmap v440: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-07T07:44:12.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:12 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/3037698781' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-07T07:44:12.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:12 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/4242414636' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-07T07:44:12.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:12 smithi160 ceph-mon[31963]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-07T07:44:12.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:12 smithi160 ceph-mon[31963]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-07T07:44:12.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:12 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/4242414636' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-07T07:44:12.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:12 smithi160 ceph-mon[31963]: osdmap e88: 12 total, 12 up, 12 in 2024-03-07T07:44:12.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:12 smithi160 ceph-mon[31963]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-03-07T07:44:12.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:12 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/4085636220' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-07T07:44:12.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:12 smithi160 ceph-mon[31963]: pgmap v440: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-07T07:44:13.007 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.007+0000 7f25d5ffb640 1 -- 172.21.15.160:0/4085636220 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v24) v1 ==== 104+0+0 (secure 0 0 0) 0x7f25c40c0aa0 con 0x7f25d8102860 2024-03-07T07:44:13.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.009+0000 7f25df06b640 1 -- 172.21.15.160:0/4085636220 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f25c80a1e00 msgr2=0x7f25c80a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:13.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.009+0000 7f25df06b640 1 --2- 172.21.15.160:0/4085636220 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f25c80a1e00 0x7f25c80a42c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f25d81026c0 tx=0x7f25cc005f90 comp rx=0 tx=0).stop 2024-03-07T07:44:13.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.009+0000 7f25df06b640 1 -- 172.21.15.160:0/4085636220 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25d8102860 msgr2=0x7f25d8197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:13.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.009+0000 7f25df06b640 1 --2- 172.21.15.160:0/4085636220 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25d8102860 0x7f25d8197ed0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f25c4009230 tx=0x7f25c402c650 comp rx=0 tx=0).stop 2024-03-07T07:44:13.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.009+0000 7f25df06b640 1 -- 172.21.15.160:0/4085636220 shutdown_connections 2024-03-07T07:44:13.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.009+0000 7f25df06b640 1 --2- 172.21.15.160:0/4085636220 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f25c80a1e00 0x7f25c80a42c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:13.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.009+0000 7f25df06b640 1 --2- 172.21.15.160:0/4085636220 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f25d8103220 0x7f25d819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:13.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.009+0000 7f25df06b640 1 --2- 172.21.15.160:0/4085636220 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25d8102860 0x7f25d8197ed0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:13.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.009+0000 7f25df06b640 1 --2- 172.21.15.160:0/4085636220 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25d8101670 0x7f25d8197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:13.010 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.009+0000 7f25df06b640 1 -- 172.21.15.160:0/4085636220 >> 172.21.15.160:0/4085636220 conn(0x7f25d80fcc00 msgr2=0x7f25d8105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:44:13.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.010+0000 7f25df06b640 1 -- 172.21.15.160:0/4085636220 shutdown_connections 2024-03-07T07:44:13.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.010+0000 7f25df06b640 1 -- 172.21.15.160:0/4085636220 wait complete. 2024-03-07T07:44:13.023 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-07T07:44:13.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.131+0000 7f8c2251d640 1 -- 172.21.15.160:0/2659710447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c1c104a80 msgr2=0x7f8c1c106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:13.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.131+0000 7f8c2251d640 1 --2- 172.21.15.160:0/2659710447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c1c104a80 0x7f8c1c106e70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c009900 tx=0x7f8c0c02bc30 comp rx=0 tx=0).stop 2024-03-07T07:44:13.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.131+0000 7f8c2251d640 1 -- 172.21.15.160:0/2659710447 shutdown_connections 2024-03-07T07:44:13.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.131+0000 7f8c2251d640 1 --2- 172.21.15.160:0/2659710447 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c1c1073b0 0x7f8c1c1097a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:13.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.131+0000 7f8c2251d640 1 --2- 172.21.15.160:0/2659710447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c1c104a80 0x7f8c1c106e70 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:13.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.131+0000 7f8c2251d640 1 --2- 172.21.15.160:0/2659710447 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c1c102150 0x7f8c1c104540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:13.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.131+0000 7f8c2251d640 1 -- 172.21.15.160:0/2659710447 >> 172.21.15.160:0/2659710447 conn(0x7f8c1c0fbb00 msgr2=0x7f8c1c0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:44:13.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.131+0000 7f8c2251d640 1 -- 172.21.15.160:0/2659710447 shutdown_connections 2024-03-07T07:44:13.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.131+0000 7f8c2251d640 1 -- 172.21.15.160:0/2659710447 wait complete. 2024-03-07T07:44:13.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.131+0000 7f8c2251d640 1 Processor -- start 2024-03-07T07:44:13.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.131+0000 7f8c2251d640 1 -- start start 2024-03-07T07:44:13.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.132+0000 7f8c2251d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c1c102150 0x7f8c1c1979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:44:13.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.132+0000 7f8c2251d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c1c104a80 0x7f8c1c197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:44:13.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.132+0000 7f8c2251d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c1c1073b0 0x7f8c1c19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:44:13.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.132+0000 7f8c2251d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c10d2d0 con 0x7f8c1c104a80 2024-03-07T07:44:13.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.132+0000 7f8c2251d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c10d150 con 0x7f8c1c102150 2024-03-07T07:44:13.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.132+0000 7f8c2251d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c10d450 con 0x7f8c1c1073b0 2024-03-07T07:44:13.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.132+0000 7f8c1b7fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c1c104a80 0x7f8c1c197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:44:13.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.132+0000 7f8c1b7fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c1c104a80 0x7f8c1c197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:37010/0 (socket says 172.21.15.160:37010) 2024-03-07T07:44:13.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.132+0000 7f8c1b7fe640 1 -- 172.21.15.160:0/4164555570 learned_addr learned my addr 172.21.15.160:0/4164555570 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:44:13.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.132+0000 7f8c1bfff640 1 --2- 172.21.15.160:0/4164555570 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c1c102150 0x7f8c1c1979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:44:13.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.132+0000 7f8c20aba640 1 --2- 172.21.15.160:0/4164555570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c1c1073b0 0x7f8c1c19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:44:13.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.132+0000 7f8c1b7fe640 1 -- 172.21.15.160:0/4164555570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c1c1073b0 msgr2=0x7f8c1c19ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:13.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.132+0000 7f8c1b7fe640 1 --2- 172.21.15.160:0/4164555570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c1c1073b0 0x7f8c1c19ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:13.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.132+0000 7f8c1b7fe640 1 -- 172.21.15.160:0/4164555570 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c1c102150 msgr2=0x7f8c1c1979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:13.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.132+0000 7f8c1b7fe640 1 --2- 172.21.15.160:0/4164555570 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c1c102150 0x7f8c1c1979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:13.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.132+0000 7f8c1b7fe640 1 -- 172.21.15.160:0/4164555570 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c1c19f5f0 con 0x7f8c1c104a80 2024-03-07T07:44:13.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.132+0000 7f8c1b7fe640 1 --2- 172.21.15.160:0/4164555570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c1c104a80 0x7f8c1c197f10 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c005ab0 tx=0x7f8c0c005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:44:13.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.132+0000 7f8c197fa640 1 -- 172.21.15.160:0/4164555570 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c041070 con 0x7f8c1c104a80 2024-03-07T07:44:13.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.132+0000 7f8c197fa640 1 -- 172.21.15.160:0/4164555570 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8c0c0054e0 con 0x7f8c1c104a80 2024-03-07T07:44:13.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.132+0000 7f8c197fa640 1 -- 172.21.15.160:0/4164555570 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c034440 con 0x7f8c1c104a80 2024-03-07T07:44:13.136 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.132+0000 7f8c2251d640 1 -- 172.21.15.160:0/4164555570 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c1c19f880 con 0x7f8c1c104a80 2024-03-07T07:44:13.136 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.132+0000 7f8c2251d640 1 -- 172.21.15.160:0/4164555570 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c1c19fd70 con 0x7f8c1c104a80 2024-03-07T07:44:13.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.133+0000 7f8c2251d640 1 -- 172.21.15.160:0/4164555570 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c1c0fd5f0 con 0x7f8c1c104a80 2024-03-07T07:44:13.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.134+0000 7f8c197fa640 1 -- 172.21.15.160:0/4164555570 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 20) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f8c0c044070 con 0x7f8c1c104a80 2024-03-07T07:44:13.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.135+0000 7f8c197fa640 1 --2- 172.21.15.160:0/4164555570 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f8c040a1d80 0x7f8c040a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:44:13.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.135+0000 7f8c1bfff640 1 --2- 172.21.15.160:0/4164555570 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f8c040a1d80 0x7f8c040a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:44:13.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.135+0000 7f8c197fa640 1 -- 172.21.15.160:0/4164555570 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f8c0c0efae0 con 0x7f8c1c104a80 2024-03-07T07:44:13.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.136+0000 7f8c1bfff640 1 --2- 172.21.15.160:0/4164555570 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f8c040a1d80 0x7f8c040a4240 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8c100058c0 tx=0x7f8c1000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:44:13.139 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.139+0000 7f8c197fa640 1 -- 172.21.15.160:0/4164555570 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8c0c0bce00 con 0x7f8c1c104a80 2024-03-07T07:44:13.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:13.315+0000 7f8c2251d640 1 -- 172.21.15.160:0/4164555570 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8c1c02d030 con 0x7f8c1c104a80 2024-03-07T07:44:13.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:13 smithi178 ceph-mon[36650]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-07T07:44:13.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:13 smithi178 ceph-mon[36650]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-07T07:44:13.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:13 smithi178 ceph-mon[36650]: Cluster is now healthy 2024-03-07T07:44:13.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:13 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/4085636220' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-07T07:44:13.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:13 smithi178 ceph-mon[36650]: mds.? [v2:172.21.15.177:6836/465515072,v1:172.21.15.177:6837/465515072] up:boot 2024-03-07T07:44:13.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:13 smithi178 ceph-mon[36650]: mds.? [v2:172.21.15.178:6833/65463771,v1:172.21.15.178:6837/65463771] up:boot 2024-03-07T07:44:13.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:13 smithi178 ceph-mon[36650]: mds.? [v2:172.21.15.177:6838/3299542484,v1:172.21.15.177:6839/3299542484] up:boot 2024-03-07T07:44:13.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:13 smithi178 ceph-mon[36650]: mds.? [v2:172.21.15.178:6838/2820935841,v1:172.21.15.178:6839/2820935841] up:boot 2024-03-07T07:44:13.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:13 smithi178 ceph-mon[36650]: mds.? [v2:172.21.15.178:6832/3073386766,v1:172.21.15.178:6835/3073386766] up:boot 2024-03-07T07:44:13.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:13 smithi178 ceph-mon[36650]: mds.? [v2:172.21.15.178:6834/2142541754,v1:172.21.15.178:6836/2142541754] up:boot 2024-03-07T07:44:13.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:13 smithi178 ceph-mon[36650]: fsmap 12 up:standby 2024-03-07T07:44:13.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:13 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-07T07:44:13.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:13 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-07T07:44:13.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:13 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-07T07:44:13.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:13 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-07T07:44:13.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:13 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-07T07:44:13.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:13 smithi178 ceph-mon[36650]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-07T07:44:13.808 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:13 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/4164555570' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-07T07:44:13.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:13 smithi177 ceph-mon[36578]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-07T07:44:13.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:13 smithi177 ceph-mon[36578]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-07T07:44:13.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:13 smithi177 ceph-mon[36578]: Cluster is now healthy 2024-03-07T07:44:13.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:13 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/4085636220' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-07T07:44:13.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:13 smithi177 ceph-mon[36578]: mds.? [v2:172.21.15.177:6836/465515072,v1:172.21.15.177:6837/465515072] up:boot 2024-03-07T07:44:13.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:13 smithi177 ceph-mon[36578]: mds.? [v2:172.21.15.178:6833/65463771,v1:172.21.15.178:6837/65463771] up:boot 2024-03-07T07:44:13.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:13 smithi177 ceph-mon[36578]: mds.? [v2:172.21.15.177:6838/3299542484,v1:172.21.15.177:6839/3299542484] up:boot 2024-03-07T07:44:13.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:13 smithi177 ceph-mon[36578]: mds.? [v2:172.21.15.178:6838/2820935841,v1:172.21.15.178:6839/2820935841] up:boot 2024-03-07T07:44:13.857 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:13 smithi177 ceph-mon[36578]: mds.? [v2:172.21.15.178:6832/3073386766,v1:172.21.15.178:6835/3073386766] up:boot 2024-03-07T07:44:13.858 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:13 smithi177 ceph-mon[36578]: mds.? [v2:172.21.15.178:6834/2142541754,v1:172.21.15.178:6836/2142541754] up:boot 2024-03-07T07:44:13.858 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:13 smithi177 ceph-mon[36578]: fsmap 12 up:standby 2024-03-07T07:44:13.858 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:13 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-07T07:44:13.858 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:13 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-07T07:44:13.858 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:13 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-07T07:44:13.858 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:13 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-07T07:44:13.858 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:13 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-07T07:44:13.858 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:13 smithi177 ceph-mon[36578]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-07T07:44:13.858 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:13 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/4164555570' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-07T07:44:13.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:13 smithi160 ceph-mon[31963]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-07T07:44:13.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:13 smithi160 ceph-mon[31963]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-07T07:44:13.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:13 smithi160 ceph-mon[31963]: Cluster is now healthy 2024-03-07T07:44:13.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:13 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/4085636220' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-07T07:44:13.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:13 smithi160 ceph-mon[31963]: mds.? [v2:172.21.15.177:6836/465515072,v1:172.21.15.177:6837/465515072] up:boot 2024-03-07T07:44:13.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:13 smithi160 ceph-mon[31963]: mds.? [v2:172.21.15.178:6833/65463771,v1:172.21.15.178:6837/65463771] up:boot 2024-03-07T07:44:13.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:13 smithi160 ceph-mon[31963]: mds.? [v2:172.21.15.177:6838/3299542484,v1:172.21.15.177:6839/3299542484] up:boot 2024-03-07T07:44:13.929 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:13 smithi160 ceph-mon[31963]: mds.? [v2:172.21.15.178:6838/2820935841,v1:172.21.15.178:6839/2820935841] up:boot 2024-03-07T07:44:13.929 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:13 smithi160 ceph-mon[31963]: mds.? [v2:172.21.15.178:6832/3073386766,v1:172.21.15.178:6835/3073386766] up:boot 2024-03-07T07:44:13.929 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:13 smithi160 ceph-mon[31963]: mds.? [v2:172.21.15.178:6834/2142541754,v1:172.21.15.178:6836/2142541754] up:boot 2024-03-07T07:44:13.929 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:13 smithi160 ceph-mon[31963]: fsmap 12 up:standby 2024-03-07T07:44:13.929 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:13 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-07T07:44:13.929 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:13 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-07T07:44:13.929 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:13 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-07T07:44:13.929 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:13 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-07T07:44:13.929 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:13 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-07T07:44:13.930 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:13 smithi160 ceph-mon[31963]: from='mgr.14152 172.21.15.160:0/3977364074' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-07T07:44:13.930 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:13 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/4164555570' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-07T07:44:14.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.009+0000 7f8c197fa640 1 -- 172.21.15.160:0/4164555570 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v89) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8c0c0bc700 con 0x7f8c1c104a80 2024-03-07T07:44:14.009 INFO:teuthology.orchestra.run.smithi160.stderr:pool 'cephfs_metadata' removed 2024-03-07T07:44:14.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.012+0000 7f8c2251d640 1 -- 172.21.15.160:0/4164555570 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f8c040a1d80 msgr2=0x7f8c040a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:14.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.012+0000 7f8c2251d640 1 --2- 172.21.15.160:0/4164555570 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f8c040a1d80 0x7f8c040a4240 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8c100058c0 tx=0x7f8c1000a2d0 comp rx=0 tx=0).stop 2024-03-07T07:44:14.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.012+0000 7f8c2251d640 1 -- 172.21.15.160:0/4164555570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c1c104a80 msgr2=0x7f8c1c197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:14.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.012+0000 7f8c2251d640 1 --2- 172.21.15.160:0/4164555570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c1c104a80 0x7f8c1c197f10 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c005ab0 tx=0x7f8c0c005dc0 comp rx=0 tx=0).stop 2024-03-07T07:44:14.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.012+0000 7f8c2251d640 1 -- 172.21.15.160:0/4164555570 shutdown_connections 2024-03-07T07:44:14.012 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.012+0000 7f8c2251d640 1 --2- 172.21.15.160:0/4164555570 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f8c040a1d80 0x7f8c040a4240 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:14.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.012+0000 7f8c2251d640 1 --2- 172.21.15.160:0/4164555570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c1c1073b0 0x7f8c1c19ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:14.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.012+0000 7f8c2251d640 1 --2- 172.21.15.160:0/4164555570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c1c104a80 0x7f8c1c197f10 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:14.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.012+0000 7f8c2251d640 1 --2- 172.21.15.160:0/4164555570 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c1c102150 0x7f8c1c1979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:14.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.012+0000 7f8c2251d640 1 -- 172.21.15.160:0/4164555570 >> 172.21.15.160:0/4164555570 conn(0x7f8c1c0fbb00 msgr2=0x7f8c1c102bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:44:14.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.012+0000 7f8c2251d640 1 -- 172.21.15.160:0/4164555570 shutdown_connections 2024-03-07T07:44:14.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.012+0000 7f8c2251d640 1 -- 172.21.15.160:0/4164555570 wait complete. 2024-03-07T07:44:14.028 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-07T07:44:14.144 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.141+0000 7f3a17bd6640 1 -- 172.21.15.160:0/2541818945 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3a10102120 msgr2=0x7f3a101089b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:14.144 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.141+0000 7f3a17bd6640 1 --2- 172.21.15.160:0/2541818945 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3a10102120 0x7f3a101089b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3a08009ca0 tx=0x7f3a0802c0f0 comp rx=0 tx=0).stop 2024-03-07T07:44:14.144 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.141+0000 7f3a17bd6640 1 -- 172.21.15.160:0/2541818945 shutdown_connections 2024-03-07T07:44:14.144 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.141+0000 7f3a17bd6640 1 --2- 172.21.15.160:0/2541818945 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3a10102120 0x7f3a101089b0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:14.144 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.141+0000 7f3a17bd6640 1 --2- 172.21.15.160:0/2541818945 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a10101760 0x7f3a10101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:14.144 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.141+0000 7f3a17bd6640 1 --2- 172.21.15.160:0/2541818945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a10100570 0x7f3a10100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:14.144 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.141+0000 7f3a17bd6640 1 -- 172.21.15.160:0/2541818945 >> 172.21.15.160:0/2541818945 conn(0x7f3a100fbae0 msgr2=0x7f3a100fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:44:14.144 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.141+0000 7f3a17bd6640 1 -- 172.21.15.160:0/2541818945 shutdown_connections 2024-03-07T07:44:14.144 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.141+0000 7f3a17bd6640 1 -- 172.21.15.160:0/2541818945 wait complete. 2024-03-07T07:44:14.144 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.142+0000 7f3a17bd6640 1 Processor -- start 2024-03-07T07:44:14.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.142+0000 7f3a17bd6640 1 -- start start 2024-03-07T07:44:14.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.142+0000 7f3a17bd6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3a10100570 0x7f3a1006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:44:14.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.142+0000 7f3a17bd6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a10101760 0x7f3a1019c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:44:14.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.142+0000 7f3a17bd6640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a10102120 0x7f3a1019ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:44:14.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.142+0000 7f3a17bd6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1010c200 con 0x7f3a10100570 2024-03-07T07:44:14.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.142+0000 7f3a17bd6640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1010c080 con 0x7f3a10102120 2024-03-07T07:44:14.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.142+0000 7f3a17bd6640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1010c380 con 0x7f3a10101760 2024-03-07T07:44:14.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.142+0000 7f3a15972640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3a10100570 0x7f3a1006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:44:14.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.142+0000 7f3a15972640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3a10100570 0x7f3a1006a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:37026/0 (socket says 172.21.15.160:37026) 2024-03-07T07:44:14.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.142+0000 7f3a15972640 1 -- 172.21.15.160:0/2930311263 learned_addr learned my addr 172.21.15.160:0/2930311263 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-03-07T07:44:14.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.142+0000 7f3a16173640 1 --2- 172.21.15.160:0/2930311263 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a10102120 0x7f3a1019ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:44:14.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.142+0000 7f3a15171640 1 --2- 172.21.15.160:0/2930311263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a10101760 0x7f3a1019c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:44:14.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.142+0000 7f3a15972640 1 -- 172.21.15.160:0/2930311263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a10101760 msgr2=0x7f3a1019c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:14.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.142+0000 7f3a15972640 1 --2- 172.21.15.160:0/2930311263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a10101760 0x7f3a1019c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:14.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.142+0000 7f3a15972640 1 -- 172.21.15.160:0/2930311263 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a10102120 msgr2=0x7f3a1019ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:14.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.142+0000 7f3a15972640 1 --2- 172.21.15.160:0/2930311263 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a10102120 0x7f3a1019ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:14.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.142+0000 7f3a15972640 1 -- 172.21.15.160:0/2930311263 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a1019f510 con 0x7f3a10100570 2024-03-07T07:44:14.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.142+0000 7f3a16173640 1 --2- 172.21.15.160:0/2930311263 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a10102120 0x7f3a1019ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:44:14.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.142+0000 7f3a15972640 1 --2- 172.21.15.160:0/2930311263 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3a10100570 0x7f3a1006a840 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c00d770 tx=0x7f3a0c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:44:14.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.143+0000 7f3a06ffd640 1 -- 172.21.15.160:0/2930311263 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c017070 con 0x7f3a10100570 2024-03-07T07:44:14.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.143+0000 7f3a06ffd640 1 -- 172.21.15.160:0/2930311263 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3a0c009c30 con 0x7f3a10100570 2024-03-07T07:44:14.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.143+0000 7f3a17bd6640 1 -- 172.21.15.160:0/2930311263 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a1019f800 con 0x7f3a10100570 2024-03-07T07:44:14.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.143+0000 7f3a06ffd640 1 -- 172.21.15.160:0/2930311263 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c003dd0 con 0x7f3a10100570 2024-03-07T07:44:14.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.143+0000 7f3a17bd6640 1 -- 172.21.15.160:0/2930311263 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a1019fda0 con 0x7f3a10100570 2024-03-07T07:44:14.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.144+0000 7f3a17bd6640 1 -- 172.21.15.160:0/2930311263 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a10107890 con 0x7f3a10100570 2024-03-07T07:44:14.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.145+0000 7f3a06ffd640 1 -- 172.21.15.160:0/2930311263 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 20) v1 ==== 136754+0+0 (secure 0 0 0) 0x7f3a0c003fb0 con 0x7f3a10100570 2024-03-07T07:44:14.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.145+0000 7f3a06ffd640 1 --2- 172.21.15.160:0/2930311263 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f39fc0a1e00 0x7f39fc0a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:44:14.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.146+0000 7f3a15171640 1 --2- 172.21.15.160:0/2930311263 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f39fc0a1e00 0x7f39fc0a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:44:14.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.146+0000 7f3a06ffd640 1 -- 172.21.15.160:0/2930311263 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 7944+0+0 (secure 0 0 0) 0x7f3a0c0cd9f0 con 0x7f3a10100570 2024-03-07T07:44:14.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.146+0000 7f3a15171640 1 --2- 172.21.15.160:0/2930311263 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f39fc0a1e00 0x7f39fc0a42c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3a1006a5b0 tx=0x7f3a00005c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:44:14.149 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.149+0000 7f3a06ffd640 1 -- 172.21.15.160:0/2930311263 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3a0c009de0 con 0x7f3a10100570 2024-03-07T07:44:14.327 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:14.327+0000 7f3a17bd6640 1 -- 172.21.15.160:0/2930311263 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3a1002d030 con 0x7f3a10100570 2024-03-07T07:44:15.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:15.014+0000 7f3a06ffd640 1 -- 172.21.15.160:0/2930311263 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v90) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3a0c09b010 con 0x7f3a10100570 2024-03-07T07:44:15.014 INFO:teuthology.orchestra.run.smithi160.stderr:pool 'cephfs_data' removed 2024-03-07T07:44:15.018 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:15.017+0000 7f3a17bd6640 1 -- 172.21.15.160:0/2930311263 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f39fc0a1e00 msgr2=0x7f39fc0a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:15.018 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:15.017+0000 7f3a17bd6640 1 --2- 172.21.15.160:0/2930311263 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f39fc0a1e00 0x7f39fc0a42c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3a1006a5b0 tx=0x7f3a00005c60 comp rx=0 tx=0).stop 2024-03-07T07:44:15.018 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:15.018+0000 7f3a17bd6640 1 -- 172.21.15.160:0/2930311263 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3a10100570 msgr2=0x7f3a1006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:44:15.018 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:15.018+0000 7f3a17bd6640 1 --2- 172.21.15.160:0/2930311263 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3a10100570 0x7f3a1006a840 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c00d770 tx=0x7f3a0c00dc30 comp rx=0 tx=0).stop 2024-03-07T07:44:15.018 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:15.018+0000 7f3a17bd6640 1 -- 172.21.15.160:0/2930311263 shutdown_connections 2024-03-07T07:44:15.018 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:15.018+0000 7f3a17bd6640 1 --2- 172.21.15.160:0/2930311263 >> [v2:172.21.15.160:6800/419630619,v1:172.21.15.160:6801/419630619] conn(0x7f39fc0a1e00 0x7f39fc0a42c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:15.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:15.018+0000 7f3a17bd6640 1 --2- 172.21.15.160:0/2930311263 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a10102120 0x7f3a1019ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:15.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:15.018+0000 7f3a17bd6640 1 --2- 172.21.15.160:0/2930311263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a10101760 0x7f3a1019c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:44:15.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:15.018+0000 7f3a17bd6640 1 --2- 172.21.15.160:0/2930311263 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3a10100570 0x7f3a1006a840 secure :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c00d770 tx=0x7f3a0c00dc30 comp rx=0 tx=0).stop 2024-03-07T07:44:15.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:15.018+0000 7f3a17bd6640 1 -- 172.21.15.160:0/2930311263 >> 172.21.15.160:0/2930311263 conn(0x7f3a100fbae0 msgr2=0x7f3a10104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:44:15.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:15.018+0000 7f3a17bd6640 1 -- 172.21.15.160:0/2930311263 shutdown_connections 2024-03-07T07:44:15.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-03-07T07:44:15.018+0000 7f3a17bd6640 1 -- 172.21.15.160:0/2930311263 wait complete. 2024-03-07T07:44:15.033 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-07T07:44:15.063 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-07T07:44:15.099 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-07T07:44:15.135 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-07T07:44:15.135 DEBUG:teuthology.orchestra.run.smithi160:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-07T07:44:15.151 DEBUG:teuthology.orchestra.run.smithi177:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-07T07:44:15.169 DEBUG:teuthology.orchestra.run.smithi178:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-07T07:44:15.192 INFO:tasks.cephadm:Stopping all daemons... 2024-03-07T07:44:15.192 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-07T07:44:15.192 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-826f6582-dc54-11ee-95c6-87774f69a715@mon.a 2024-03-07T07:44:15.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:15 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/4164555570' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-07T07:44:15.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:15 smithi178 ceph-mon[36650]: osdmap e89: 12 total, 12 up, 12 in 2024-03-07T07:44:15.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:15 smithi178 ceph-mon[36650]: from='client.? 172.21.15.160:0/2930311263' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-07T07:44:15.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:15 smithi178 ceph-mon[36650]: pgmap v442: 65 pgs: 65 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:44:15.356 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:15 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/4164555570' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-07T07:44:15.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:15 smithi177 ceph-mon[36578]: osdmap e89: 12 total, 12 up, 12 in 2024-03-07T07:44:15.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:15 smithi177 ceph-mon[36578]: from='client.? 172.21.15.160:0/2930311263' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-07T07:44:15.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:15 smithi177 ceph-mon[36578]: pgmap v442: 65 pgs: 65 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:44:15.427 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:15 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/4164555570' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-07T07:44:15.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:15 smithi160 ceph-mon[31963]: osdmap e89: 12 total, 12 up, 12 in 2024-03-07T07:44:15.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:15 smithi160 ceph-mon[31963]: from='client.? 172.21.15.160:0/2930311263' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-07T07:44:15.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:15 smithi160 ceph-mon[31963]: pgmap v442: 65 pgs: 65 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-07T07:44:15.428 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:15 smithi160 systemd[1]: Stopping Ceph mon.a for 826f6582-dc54-11ee-95c6-87774f69a715... 2024-03-07T07:44:15.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:15 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mon-a[31959]: 2024-03-07T07:44:15.513+0000 7fc705963700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-07T07:44:15.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:15 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mon-a[31959]: 2024-03-07T07:44:15.513+0000 7fc705963700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-07T07:44:15.928 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:15 smithi160 podman[87349]: 2024-03-07 07:44:15.554509427 +0000 UTC m=+0.164760550 container died 23e49f05d5ab628af000486507c44ce0bbea95cc185028910c25c1f64531a00e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mon-a, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-8b896a9, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-07T07:44:16.406 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:16 smithi160 podman[87349]: 2024-03-07 07:44:16.154782737 +0000 UTC m=+0.765033912 container cleanup 23e49f05d5ab628af000486507c44ce0bbea95cc185028910c25c1f64531a00e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mon-a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.schema-version=1.0, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-07T07:44:16.407 INFO:journalctl@ceph.mon.a.smithi160.stdout:Mar 07 07:44:16 smithi160 bash[87349]: ceph-826f6582-dc54-11ee-95c6-87774f69a715-mon-a 2024-03-07T07:44:16.538 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@mon.a.service' 2024-03-07T07:44:16.588 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-07T07:44:16.589 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-07T07:44:16.589 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-03-07T07:44:16.589 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-826f6582-dc54-11ee-95c6-87774f69a715@mon.b 2024-03-07T07:44:16.895 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:16 smithi177 systemd[1]: Stopping Ceph mon.b for 826f6582-dc54-11ee-95c6-87774f69a715... 2024-03-07T07:44:17.262 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:16 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mon-b[36574]: 2024-03-07T07:44:16.894+0000 7f0dbaf9f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-07T07:44:17.263 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:16 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mon-b[36574]: 2024-03-07T07:44:16.894+0000 7f0dbaf9f700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-03-07T07:44:17.263 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:16 smithi177 podman[69847]: 2024-03-07 07:44:16.966290048 +0000 UTC m=+0.179486271 container died f09afba66d4cb77222599abbc6a0d8c34649cc183f119c5eed0e8617651aad1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mon-b, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-8b896a9, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-07T07:44:17.606 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:17 smithi177 podman[69847]: 2024-03-07 07:44:17.471159318 +0000 UTC m=+0.684355538 container cleanup f09afba66d4cb77222599abbc6a0d8c34649cc183f119c5eed0e8617651aad1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mon-b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, RELEASE=wip-lusov-qdb-8b896a9, maintainer=Guillaume Abrioux ) 2024-03-07T07:44:17.607 INFO:journalctl@ceph.mon.b.smithi177.stdout:Mar 07 07:44:17 smithi177 bash[69847]: ceph-826f6582-dc54-11ee-95c6-87774f69a715-mon-b 2024-03-07T07:44:17.844 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@mon.b.service' 2024-03-07T07:44:17.896 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-07T07:44:17.896 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-03-07T07:44:17.897 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-03-07T07:44:17.897 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-826f6582-dc54-11ee-95c6-87774f69a715@mon.c 2024-03-07T07:44:18.241 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:17 smithi178 systemd[1]: Stopping Ceph mon.c for 826f6582-dc54-11ee-95c6-87774f69a715... 2024-03-07T07:44:18.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:18 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mon-c[36632]: 2024-03-07T07:44:18.240+0000 7f1e58fe9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-07T07:44:18.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:18 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mon-c[36632]: 2024-03-07T07:44:18.240+0000 7f1e58fe9700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-03-07T07:44:18.556 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:18 smithi178 podman[71605]: 2024-03-07 07:44:18.414984167 +0000 UTC m=+0.315067231 container died 3f6302e362f68b96182490ae1b07cd441c759ef8692f689b4acb6c18e7bb4ca0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mon-c, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.build-date=20240222, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-07T07:44:19.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:18 smithi178 podman[71605]: 2024-03-07 07:44:18.80863942 +0000 UTC m=+0.708722492 container cleanup 3f6302e362f68b96182490ae1b07cd441c759ef8692f689b4acb6c18e7bb4ca0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-8b896a9, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0) 2024-03-07T07:44:19.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Mar 07 07:44:18 smithi178 bash[71605]: ceph-826f6582-dc54-11ee-95c6-87774f69a715-mon-c 2024-03-07T07:44:19.192 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@mon.c.service' 2024-03-07T07:44:19.244 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-07T07:44:19.244 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-03-07T07:44:19.244 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-07T07:44:19.244 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-826f6582-dc54-11ee-95c6-87774f69a715@mgr.x 2024-03-07T07:44:19.570 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:44:19 smithi160 systemd[1]: Stopping Ceph mgr.x for 826f6582-dc54-11ee-95c6-87774f69a715... 2024-03-07T07:44:19.884 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:44:19 smithi160 podman[87473]: 2024-03-07 07:44:19.573905646 +0000 UTC m=+0.134602577 container died c6ae33bdae0b8dd26272801a4ce3d3885e0323d0da99aafb66ad680f6825ff38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-8b896a9, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=) 2024-03-07T07:44:20.136 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:44:19 smithi160 podman[87473]: 2024-03-07 07:44:19.885242078 +0000 UTC m=+0.445939004 container cleanup c6ae33bdae0b8dd26272801a4ce3d3885e0323d0da99aafb66ad680f6825ff38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8b896a9e145796119f0451201dc4d53ddac97db0, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-8b896a9, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b) 2024-03-07T07:44:20.136 INFO:journalctl@ceph.mgr.x.smithi160.stdout:Mar 07 07:44:19 smithi160 bash[87473]: ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-x 2024-03-07T07:44:20.305 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@mgr.x.service' 2024-03-07T07:44:20.343 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-07T07:44:20.344 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-07T07:44:20.344 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-07T07:44:20.344 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-826f6582-dc54-11ee-95c6-87774f69a715@mgr.y 2024-03-07T07:44:20.648 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Mar 07 07:44:20 smithi177 systemd[1]: Stopping Ceph mgr.y for 826f6582-dc54-11ee-95c6-87774f69a715... 2024-03-07T07:44:20.944 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Mar 07 07:44:20 smithi177 podman[69970]: 2024-03-07 07:44:20.66437979 +0000 UTC m=+0.118642980 container died 44e7a1cb82b18b91af95635cea4807e8f3b7ae37c4746bd0f75645d6e3cecd7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-y, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-8b896a9, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-07T07:44:21.311 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Mar 07 07:44:21 smithi177 podman[69970]: 2024-03-07 07:44:21.052275578 +0000 UTC m=+0.506538759 container cleanup 44e7a1cb82b18b91af95635cea4807e8f3b7ae37c4746bd0f75645d6e3cecd7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-8b896a9, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240222) 2024-03-07T07:44:21.311 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Mar 07 07:44:21 smithi177 bash[69970]: ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-y 2024-03-07T07:44:21.429 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@mgr.y.service' 2024-03-07T07:44:21.469 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-07T07:44:21.469 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-07T07:44:21.469 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-03-07T07:44:21.469 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-826f6582-dc54-11ee-95c6-87774f69a715@mgr.z 2024-03-07T07:44:21.806 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Mar 07 07:44:21 smithi178 systemd[1]: Stopping Ceph mgr.z for 826f6582-dc54-11ee-95c6-87774f69a715... 2024-03-07T07:44:22.081 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Mar 07 07:44:22 smithi178 podman[71727]: 2024-03-07 07:44:22.001847647 +0000 UTC m=+0.336705522 container died e1f247c6e191f52a6ac942cbc8dc93251793412c717309519ed6424207bc58ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-z, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-03-07T07:44:22.650 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Mar 07 07:44:22 smithi178 podman[71727]: 2024-03-07 07:44:22.390399936 +0000 UTC m=+0.725257780 container cleanup e1f247c6e191f52a6ac942cbc8dc93251793412c717309519ed6424207bc58ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-z, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-07T07:44:22.650 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Mar 07 07:44:22 smithi178 bash[71727]: ceph-826f6582-dc54-11ee-95c6-87774f69a715-mgr-z 2024-03-07T07:44:22.815 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@mgr.z.service' 2024-03-07T07:44:22.855 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-07T07:44:22.855 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-03-07T07:44:22.855 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-07T07:44:22.855 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-826f6582-dc54-11ee-95c6-87774f69a715@osd.0 2024-03-07T07:44:23.148 INFO:journalctl@ceph.osd.0.smithi160.stdout:Mar 07 07:44:22 smithi160 systemd[1]: Stopping Ceph osd.0 for 826f6582-dc54-11ee-95c6-87774f69a715... 2024-03-07T07:44:23.427 INFO:journalctl@ceph.osd.0.smithi160.stdout:Mar 07 07:44:23 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-0[46912]: 2024-03-07T07:44:23.147+0000 7f3055fdd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-07T07:44:23.428 INFO:journalctl@ceph.osd.0.smithi160.stdout:Mar 07 07:44:23 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-0[46912]: 2024-03-07T07:44:23.147+0000 7f3055fdd700 -1 osd.0 90 *** Got signal Terminated *** 2024-03-07T07:44:23.428 INFO:journalctl@ceph.osd.0.smithi160.stdout:Mar 07 07:44:23 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-0[46912]: 2024-03-07T07:44:23.147+0000 7f3055fdd700 -1 osd.0 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-07T07:44:28.741 INFO:journalctl@ceph.osd.0.smithi160.stdout:Mar 07 07:44:28 smithi160 podman[87594]: 2024-03-07 07:44:28.46352247 +0000 UTC m=+5.417709071 container died 75b6c5128948e4d58cccba6d01793fb5510cec2a083ec3c6efade3ded18b5c72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222) 2024-03-07T07:44:29.177 INFO:journalctl@ceph.osd.0.smithi160.stdout:Mar 07 07:44:28 smithi160 podman[87594]: 2024-03-07 07:44:28.932323366 +0000 UTC m=+5.886510012 container cleanup 75b6c5128948e4d58cccba6d01793fb5510cec2a083ec3c6efade3ded18b5c72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-0, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-8b896a9, ceph=True, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-07T07:44:29.178 INFO:journalctl@ceph.osd.0.smithi160.stdout:Mar 07 07:44:28 smithi160 bash[87594]: ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-0 2024-03-07T07:44:29.677 INFO:journalctl@ceph.osd.0.smithi160.stdout:Mar 07 07:44:29 smithi160 podman[87611]: 2024-03-07 07:44:29.368004398 +0000 UTC m=+0.897667782 container remove 75b6c5128948e4d58cccba6d01793fb5510cec2a083ec3c6efade3ded18b5c72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.build-date=20240222, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-07T07:44:29.946 INFO:journalctl@ceph.osd.0.smithi160.stdout:Mar 07 07:44:29 smithi160 podman[87704]: 2024-03-07 07:44:29.614141123 +0000 UTC m=+0.018564046 image pull e362cd7cebfee51077fb90b7f01895811140c9a4272edd83ad9910b05e3a46f6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2 2024-03-07T07:44:29.946 INFO:journalctl@ceph.osd.0.smithi160.stdout:Mar 07 07:44:29 smithi160 podman[87704]: 2024-03-07 07:44:29.718754166 +0000 UTC m=+0.123177075 container create 449d3237307ceb9617906b77a551ded143313130e315614c5269169518b564b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, RELEASE=wip-lusov-qdb-8b896a9, ceph=True) 2024-03-07T07:44:30.251 INFO:journalctl@ceph.osd.0.smithi160.stdout:Mar 07 07:44:29 smithi160 podman[87704]: 2024-03-07 07:44:29.946825339 +0000 UTC m=+0.351248261 container init 449d3237307ceb9617906b77a551ded143313130e315614c5269169518b564b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, GIT_CLEAN=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-8b896a9, GIT_BRANCH=HEAD) 2024-03-07T07:44:30.251 INFO:journalctl@ceph.osd.0.smithi160.stdout:Mar 07 07:44:29 smithi160 podman[87704]: 2024-03-07 07:44:29.950588679 +0000 UTC m=+0.355011590 container start 449d3237307ceb9617906b77a551ded143313130e315614c5269169518b564b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, RELEASE=wip-lusov-qdb-8b896a9, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-07T07:44:30.251 INFO:journalctl@ceph.osd.0.smithi160.stdout:Mar 07 07:44:30 smithi160 podman[87704]: 2024-03-07 07:44:30.038538266 +0000 UTC m=+0.442961182 container attach 449d3237307ceb9617906b77a551ded143313130e315614c5269169518b564b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2) 2024-03-07T07:44:30.677 INFO:journalctl@ceph.osd.0.smithi160.stdout:Mar 07 07:44:30 smithi160 podman[87704]: 2024-03-07 07:44:30.251377424 +0000 UTC m=+0.655800347 container died 449d3237307ceb9617906b77a551ded143313130e315614c5269169518b564b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-0-deactivate, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-07T07:44:30.817 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@osd.0.service' 2024-03-07T07:44:30.861 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-07T07:44:30.861 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-07T07:44:30.861 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-07T07:44:30.861 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-826f6582-dc54-11ee-95c6-87774f69a715@osd.1 2024-03-07T07:44:31.173 INFO:journalctl@ceph.osd.1.smithi177.stdout:Mar 07 07:44:30 smithi177 systemd[1]: Stopping Ceph osd.1 for 826f6582-dc54-11ee-95c6-87774f69a715... 2024-03-07T07:44:31.607 INFO:journalctl@ceph.osd.1.smithi177.stdout:Mar 07 07:44:31 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-1[41162]: 2024-03-07T07:44:31.172+0000 7fed8638c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-07T07:44:31.607 INFO:journalctl@ceph.osd.1.smithi177.stdout:Mar 07 07:44:31 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-1[41162]: 2024-03-07T07:44:31.172+0000 7fed8638c700 -1 osd.1 90 *** Got signal Terminated *** 2024-03-07T07:44:31.607 INFO:journalctl@ceph.osd.1.smithi177.stdout:Mar 07 07:44:31 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-1[41162]: 2024-03-07T07:44:31.172+0000 7fed8638c700 -1 osd.1 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-07T07:44:36.882 INFO:journalctl@ceph.osd.1.smithi177.stdout:Mar 07 07:44:36 smithi177 podman[70103]: 2024-03-07 07:44:36.612651812 +0000 UTC m=+5.554452630 container died 65b9067952e32517e29cf4998ccd92862e5d422a5773eb5a506d5abad450613a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, RELEASE=wip-lusov-qdb-8b896a9, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-07T07:44:37.299 INFO:journalctl@ceph.osd.1.smithi177.stdout:Mar 07 07:44:36 smithi177 podman[70103]: 2024-03-07 07:44:36.981337026 +0000 UTC m=+5.923137834 container cleanup 65b9067952e32517e29cf4998ccd92862e5d422a5773eb5a506d5abad450613a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-1, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-8b896a9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240222) 2024-03-07T07:44:37.299 INFO:journalctl@ceph.osd.1.smithi177.stdout:Mar 07 07:44:36 smithi177 bash[70103]: ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-1 2024-03-07T07:44:37.606 INFO:journalctl@ceph.osd.1.smithi177.stdout:Mar 07 07:44:37 smithi177 podman[70118]: 2024-03-07 07:44:37.299824294 +0000 UTC m=+0.683453095 container remove 65b9067952e32517e29cf4998ccd92862e5d422a5773eb5a506d5abad450613a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-1, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.vendor=CentOS, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-07T07:44:37.895 INFO:journalctl@ceph.osd.1.smithi177.stdout:Mar 07 07:44:37 smithi177 podman[70212]: 2024-03-07 07:44:37.548231937 +0000 UTC m=+0.018083659 image pull e362cd7cebfee51077fb90b7f01895811140c9a4272edd83ad9910b05e3a46f6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2 2024-03-07T07:44:37.895 INFO:journalctl@ceph.osd.1.smithi177.stdout:Mar 07 07:44:37 smithi177 podman[70212]: 2024-03-07 07:44:37.650851007 +0000 UTC m=+0.120702757 container create 9d6f61254524144d1b39874572d35d235290ffb1ad829538e168cb38381c15cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-1-deactivate, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-07T07:44:37.895 INFO:journalctl@ceph.osd.1.smithi177.stdout:Mar 07 07:44:37 smithi177 podman[70212]: 2024-03-07 07:44:37.895067201 +0000 UTC m=+0.364918909 container init 9d6f61254524144d1b39874572d35d235290ffb1ad829538e168cb38381c15cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-1-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240222, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True) 2024-03-07T07:44:38.189 INFO:journalctl@ceph.osd.1.smithi177.stdout:Mar 07 07:44:37 smithi177 podman[70212]: 2024-03-07 07:44:37.899341581 +0000 UTC m=+0.369193289 container start 9d6f61254524144d1b39874572d35d235290ffb1ad829538e168cb38381c15cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-1-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-07T07:44:38.189 INFO:journalctl@ceph.osd.1.smithi177.stdout:Mar 07 07:44:37 smithi177 podman[70212]: 2024-03-07 07:44:37.986885463 +0000 UTC m=+0.456737188 container attach 9d6f61254524144d1b39874572d35d235290ffb1ad829538e168cb38381c15cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, RELEASE=wip-lusov-qdb-8b896a9, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, ceph=True) 2024-03-07T07:44:38.454 INFO:journalctl@ceph.osd.1.smithi177.stdout:Mar 07 07:44:38 smithi177 podman[70212]: 2024-03-07 07:44:38.190081069 +0000 UTC m=+0.659932794 container died 9d6f61254524144d1b39874572d35d235290ffb1ad829538e168cb38381c15cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, io.buildah.version=1.33.5) 2024-03-07T07:44:38.825 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@osd.1.service' 2024-03-07T07:44:38.866 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-07T07:44:38.866 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-07T07:44:38.866 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-07T07:44:38.867 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-826f6582-dc54-11ee-95c6-87774f69a715@osd.2 2024-03-07T07:44:39.186 INFO:journalctl@ceph.osd.2.smithi178.stdout:Mar 07 07:44:38 smithi178 systemd[1]: Stopping Ceph osd.2 for 826f6582-dc54-11ee-95c6-87774f69a715... 2024-03-07T07:44:39.556 INFO:journalctl@ceph.osd.2.smithi178.stdout:Mar 07 07:44:39 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-2[41777]: 2024-03-07T07:44:39.185+0000 7fa74988d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-07T07:44:39.556 INFO:journalctl@ceph.osd.2.smithi178.stdout:Mar 07 07:44:39 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-2[41777]: 2024-03-07T07:44:39.185+0000 7fa74988d700 -1 osd.2 90 *** Got signal Terminated *** 2024-03-07T07:44:39.556 INFO:journalctl@ceph.osd.2.smithi178.stdout:Mar 07 07:44:39 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-2[41777]: 2024-03-07T07:44:39.185+0000 7fa74988d700 -1 osd.2 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-07T07:44:44.953 INFO:journalctl@ceph.osd.2.smithi178.stdout:Mar 07 07:44:44 smithi178 podman[71864]: 2024-03-07 07:44:44.645857928 +0000 UTC m=+5.585521243 container died 6ce198dedec2ff38472b0c2a8e1c255656d6ca594c38d9a026d588f22f1ab9d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-2, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-8b896a9, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222) 2024-03-07T07:44:45.270 INFO:journalctl@ceph.osd.2.smithi178.stdout:Mar 07 07:44:45 smithi178 podman[71864]: 2024-03-07 07:44:45.044835155 +0000 UTC m=+5.984498489 container cleanup 6ce198dedec2ff38472b0c2a8e1c255656d6ca594c38d9a026d588f22f1ab9d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.vendor=CentOS, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2) 2024-03-07T07:44:45.271 INFO:journalctl@ceph.osd.2.smithi178.stdout:Mar 07 07:44:45 smithi178 bash[71864]: ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-2 2024-03-07T07:44:45.556 INFO:journalctl@ceph.osd.2.smithi178.stdout:Mar 07 07:44:45 smithi178 podman[71881]: 2024-03-07 07:44:45.271321967 +0000 UTC m=+0.625504677 container remove 6ce198dedec2ff38472b0c2a8e1c255656d6ca594c38d9a026d588f22f1ab9d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-8b896a9, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS) 2024-03-07T07:44:45.875 INFO:journalctl@ceph.osd.2.smithi178.stdout:Mar 07 07:44:45 smithi178 podman[71974]: 2024-03-07 07:44:45.50615603 +0000 UTC m=+0.018060535 image pull e362cd7cebfee51077fb90b7f01895811140c9a4272edd83ad9910b05e3a46f6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2 2024-03-07T07:44:45.875 INFO:journalctl@ceph.osd.2.smithi178.stdout:Mar 07 07:44:45 smithi178 podman[71974]: 2024-03-07 07:44:45.630747047 +0000 UTC m=+0.142651581 container create d0644bbda663bd8ef5b1eb06c2f5fdef607040cc95aaea58c6e472b993923242 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-8b896a9, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-07T07:44:45.875 INFO:journalctl@ceph.osd.2.smithi178.stdout:Mar 07 07:44:45 smithi178 podman[71974]: 2024-03-07 07:44:45.875050593 +0000 UTC m=+0.386955085 container init d0644bbda663bd8ef5b1eb06c2f5fdef607040cc95aaea58c6e472b993923242 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-8b896a9, ceph=True) 2024-03-07T07:44:46.163 INFO:journalctl@ceph.osd.2.smithi178.stdout:Mar 07 07:44:45 smithi178 podman[71974]: 2024-03-07 07:44:45.878433228 +0000 UTC m=+0.390337720 container start d0644bbda663bd8ef5b1eb06c2f5fdef607040cc95aaea58c6e472b993923242 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-8b896a9, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.schema-version=1.0) 2024-03-07T07:44:46.164 INFO:journalctl@ceph.osd.2.smithi178.stdout:Mar 07 07:44:45 smithi178 podman[71974]: 2024-03-07 07:44:45.975291914 +0000 UTC m=+0.487196410 container attach d0644bbda663bd8ef5b1eb06c2f5fdef607040cc95aaea58c6e472b993923242 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-8b896a9, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-07T07:44:46.435 INFO:journalctl@ceph.osd.2.smithi178.stdout:Mar 07 07:44:46 smithi178 podman[71974]: 2024-03-07 07:44:46.164588936 +0000 UTC m=+0.676493430 container died d0644bbda663bd8ef5b1eb06c2f5fdef607040cc95aaea58c6e472b993923242 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-2-deactivate, RELEASE=wip-lusov-qdb-8b896a9, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-07T07:44:46.814 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@osd.2.service' 2024-03-07T07:44:46.856 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-07T07:44:46.856 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-07T07:44:46.856 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-07T07:44:46.857 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-826f6582-dc54-11ee-95c6-87774f69a715@osd.3 2024-03-07T07:44:47.177 INFO:journalctl@ceph.osd.3.smithi160.stdout:Mar 07 07:44:46 smithi160 systemd[1]: Stopping Ceph osd.3 for 826f6582-dc54-11ee-95c6-87774f69a715... 2024-03-07T07:44:47.439 INFO:journalctl@ceph.osd.3.smithi160.stdout:Mar 07 07:44:47 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-3[53426]: 2024-03-07T07:44:47.177+0000 7fb69e868700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-07T07:44:47.439 INFO:journalctl@ceph.osd.3.smithi160.stdout:Mar 07 07:44:47 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-3[53426]: 2024-03-07T07:44:47.177+0000 7fb69e868700 -1 osd.3 90 *** Got signal Terminated *** 2024-03-07T07:44:47.439 INFO:journalctl@ceph.osd.3.smithi160.stdout:Mar 07 07:44:47 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-3[53426]: 2024-03-07T07:44:47.177+0000 7fb69e868700 -1 osd.3 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-07T07:44:49.556 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:44:49 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-5[48766]: 2024-03-07T07:44:49.110+0000 7f38c24f8700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:25.514679+0000 front 2024-03-07T07:44:25.514805+0000 (oldest deadline 2024-03-07T07:44:49.014605+0000) 2024-03-07T07:44:50.556 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:44:50 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-5[48766]: 2024-03-07T07:44:50.132+0000 7f38c24f8700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:25.514679+0000 front 2024-03-07T07:44:25.514805+0000 (oldest deadline 2024-03-07T07:44:49.014605+0000) 2024-03-07T07:44:50.677 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:44:50 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:44:50.254+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.270498+0000 front 2024-03-07T07:44:26.270653+0000 (oldest deadline 2024-03-07T07:44:49.770684+0000) 2024-03-07T07:44:51.457 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:44:51 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-5[48766]: 2024-03-07T07:44:51.123+0000 7f38c24f8700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:25.514679+0000 front 2024-03-07T07:44:25.514805+0000 (oldest deadline 2024-03-07T07:44:49.014605+0000) 2024-03-07T07:44:51.677 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:44:51 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:44:51.252+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.270498+0000 front 2024-03-07T07:44:26.270653+0000 (oldest deadline 2024-03-07T07:44:49.770684+0000) 2024-03-07T07:44:52.356 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:44:51 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:44:51.883+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:44:52.427 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:44:52 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:44:52.287+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.270498+0000 front 2024-03-07T07:44:26.270653+0000 (oldest deadline 2024-03-07T07:44:49.770684+0000) 2024-03-07T07:44:52.428 INFO:journalctl@ceph.osd.3.smithi160.stdout:Mar 07 07:44:52 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-3[53426]: 2024-03-07T07:44:52.125+0000 7fb69ae5c700 -1 osd.3 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:23.483520+0000 front 2024-03-07T07:44:23.483499+0000 (oldest deadline 2024-03-07T07:44:51.684344+0000) 2024-03-07T07:44:52.556 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:44:52 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-5[48766]: 2024-03-07T07:44:52.086+0000 7f38c24f8700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:25.514679+0000 front 2024-03-07T07:44:25.514805+0000 (oldest deadline 2024-03-07T07:44:49.014605+0000) 2024-03-07T07:44:52.812 INFO:journalctl@ceph.osd.3.smithi160.stdout:Mar 07 07:44:52 smithi160 podman[87810]: 2024-03-07 07:44:52.507683652 +0000 UTC m=+5.458974412 container died ecbaaa7faf76aaaac69fbe5067f053739e8bf1ba1df52ffac4c1234b071ec08f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-3, RELEASE=wip-lusov-qdb-8b896a9, ceph=True, org.label-schema.build-date=20240222, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-07T07:44:53.178 INFO:journalctl@ceph.osd.3.smithi160.stdout:Mar 07 07:44:53 smithi160 podman[87810]: 2024-03-07 07:44:53.00339709 +0000 UTC m=+5.954687851 container cleanup ecbaaa7faf76aaaac69fbe5067f053739e8bf1ba1df52ffac4c1234b071ec08f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-3, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-8b896a9, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, GIT_CLEAN=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-07T07:44:53.178 INFO:journalctl@ceph.osd.3.smithi160.stdout:Mar 07 07:44:53 smithi160 bash[87810]: ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-3 2024-03-07T07:44:53.306 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:44:53 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-5[48766]: 2024-03-07T07:44:53.106+0000 7f38c24f8700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:25.514679+0000 front 2024-03-07T07:44:25.514805+0000 (oldest deadline 2024-03-07T07:44:49.014605+0000) 2024-03-07T07:44:53.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:44:52 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:44:52.887+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:44:53.356 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:44:52 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:44:52.880+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:44:53.672 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:44:53 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:44:53.254+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.270498+0000 front 2024-03-07T07:44:26.270653+0000 (oldest deadline 2024-03-07T07:44:49.770684+0000) 2024-03-07T07:44:53.672 INFO:journalctl@ceph.osd.3.smithi160.stdout:Mar 07 07:44:53 smithi160 podman[87838]: 2024-03-07 07:44:53.355532267 +0000 UTC m=+0.849546266 container remove ecbaaa7faf76aaaac69fbe5067f053739e8bf1ba1df52ffac4c1234b071ec08f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-3, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-8b896a9, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, ceph=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-07T07:44:53.927 INFO:journalctl@ceph.osd.3.smithi160.stdout:Mar 07 07:44:53 smithi160 podman[87926]: 2024-03-07 07:44:53.576643629 +0000 UTC m=+0.018819293 image pull e362cd7cebfee51077fb90b7f01895811140c9a4272edd83ad9910b05e3a46f6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2 2024-03-07T07:44:53.928 INFO:journalctl@ceph.osd.3.smithi160.stdout:Mar 07 07:44:53 smithi160 podman[87926]: 2024-03-07 07:44:53.723120614 +0000 UTC m=+0.165296319 container create 734da0c2e5cfe30ef436f0fd123660faf998077232bf831ce87a545be57c1f72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-3-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b) 2024-03-07T07:44:54.255 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:44:54 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:44:54.170+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.876690+0000 front 2024-03-07T07:44:27.876716+0000 (oldest deadline 2024-03-07T07:44:53.776918+0000) 2024-03-07T07:44:54.255 INFO:journalctl@ceph.osd.3.smithi160.stdout:Mar 07 07:44:53 smithi160 podman[87926]: 2024-03-07 07:44:53.967467462 +0000 UTC m=+0.409643126 container init 734da0c2e5cfe30ef436f0fd123660faf998077232bf831ce87a545be57c1f72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-8b896a9, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b) 2024-03-07T07:44:54.256 INFO:journalctl@ceph.osd.3.smithi160.stdout:Mar 07 07:44:53 smithi160 podman[87926]: 2024-03-07 07:44:53.970626583 +0000 UTC m=+0.412802253 container start 734da0c2e5cfe30ef436f0fd123660faf998077232bf831ce87a545be57c1f72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-3-deactivate, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, CEPH_POINT_RELEASE=) 2024-03-07T07:44:54.256 INFO:journalctl@ceph.osd.3.smithi160.stdout:Mar 07 07:44:54 smithi160 podman[87926]: 2024-03-07 07:44:54.075755209 +0000 UTC m=+0.517930887 container attach 734da0c2e5cfe30ef436f0fd123660faf998077232bf831ce87a545be57c1f72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-3-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-03-07T07:44:54.306 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:44:54 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-5[48766]: 2024-03-07T07:44:54.103+0000 7f38c24f8700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:25.514679+0000 front 2024-03-07T07:44:25.514805+0000 (oldest deadline 2024-03-07T07:44:49.014605+0000) 2024-03-07T07:44:54.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:44:53 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:44:53.924+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:44:54.356 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:44:53 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:44:53.891+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:44:54.570 INFO:journalctl@ceph.osd.3.smithi160.stdout:Mar 07 07:44:54 smithi160 podman[87926]: 2024-03-07 07:44:54.261917843 +0000 UTC m=+0.704093505 container died 734da0c2e5cfe30ef436f0fd123660faf998077232bf831ce87a545be57c1f72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-3-deactivate, RELEASE=wip-lusov-qdb-8b896a9, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-07T07:44:54.570 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:44:54 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:44:54.254+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.270498+0000 front 2024-03-07T07:44:26.270653+0000 (oldest deadline 2024-03-07T07:44:49.770684+0000) 2024-03-07T07:44:55.030 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@osd.3.service' 2024-03-07T07:44:55.073 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-07T07:44:55.073 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-07T07:44:55.073 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-07T07:44:55.073 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-826f6582-dc54-11ee-95c6-87774f69a715@osd.4 2024-03-07T07:44:55.279 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:44:55 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:44:55.166+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.876690+0000 front 2024-03-07T07:44:27.876716+0000 (oldest deadline 2024-03-07T07:44:53.776918+0000) 2024-03-07T07:44:55.306 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:44:55 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-5[48766]: 2024-03-07T07:44:55.129+0000 7f38c24f8700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:25.514679+0000 front 2024-03-07T07:44:25.514805+0000 (oldest deadline 2024-03-07T07:44:49.014605+0000) 2024-03-07T07:44:55.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:44:54 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:44:54.949+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:44:55.357 INFO:journalctl@ceph.osd.4.smithi177.stdout:Mar 07 07:44:55 smithi177 systemd[1]: Stopping Ceph osd.4 for 826f6582-dc54-11ee-95c6-87774f69a715... 2024-03-07T07:44:55.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:44:54 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:44:54.896+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:44:55.677 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:44:55 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:44:55.278+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.270498+0000 front 2024-03-07T07:44:26.270653+0000 (oldest deadline 2024-03-07T07:44:49.770684+0000) 2024-03-07T07:44:55.856 INFO:journalctl@ceph.osd.4.smithi177.stdout:Mar 07 07:44:55 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-4[48119]: 2024-03-07T07:44:55.401+0000 7f2635790700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-07T07:44:55.857 INFO:journalctl@ceph.osd.4.smithi177.stdout:Mar 07 07:44:55 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-4[48119]: 2024-03-07T07:44:55.401+0000 7f2635790700 -1 osd.4 90 *** Got signal Terminated *** 2024-03-07T07:44:55.857 INFO:journalctl@ceph.osd.4.smithi177.stdout:Mar 07 07:44:55 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-4[48119]: 2024-03-07T07:44:55.401+0000 7f2635790700 -1 osd.4 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-07T07:44:56.161 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:44:55 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:44:55.901+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:44:56.356 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:44:55 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:44:55.945+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:44:56.427 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:44:56 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:44:56.274+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.270498+0000 front 2024-03-07T07:44:26.270653+0000 (oldest deadline 2024-03-07T07:44:49.770684+0000) 2024-03-07T07:44:56.428 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:44:56 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:44:56.161+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.876690+0000 front 2024-03-07T07:44:27.876716+0000 (oldest deadline 2024-03-07T07:44:53.776918+0000) 2024-03-07T07:44:56.556 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:44:56 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-5[48766]: 2024-03-07T07:44:56.161+0000 7f38c24f8700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:25.514679+0000 front 2024-03-07T07:44:25.514805+0000 (oldest deadline 2024-03-07T07:44:49.014605+0000) 2024-03-07T07:44:57.306 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:44:57 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-5[48766]: 2024-03-07T07:44:57.121+0000 7f38c24f8700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:25.514679+0000 front 2024-03-07T07:44:25.514805+0000 (oldest deadline 2024-03-07T07:44:49.014605+0000) 2024-03-07T07:44:57.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:44:56 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:44:56.898+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:44:57.356 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:44:56 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:44:56.926+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:44:57.428 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:44:57 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:44:57.259+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.270498+0000 front 2024-03-07T07:44:26.270653+0000 (oldest deadline 2024-03-07T07:44:49.770684+0000) 2024-03-07T07:44:57.428 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:44:57 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:44:57.145+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.876690+0000 front 2024-03-07T07:44:27.876716+0000 (oldest deadline 2024-03-07T07:44:53.776918+0000) 2024-03-07T07:44:58.128 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:44:57 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:44:57.871+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:32.457358+0000 front 2024-03-07T07:44:32.457424+0000 (oldest deadline 2024-03-07T07:44:57.757460+0000) 2024-03-07T07:44:58.129 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:44:57 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:44:57.896+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:44:58.129 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:44:57 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:44:57.896+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:44:58.359 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:44:57 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:44:57.878+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:44:58.427 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:44:58 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:44:58.162+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.876690+0000 front 2024-03-07T07:44:27.876716+0000 (oldest deadline 2024-03-07T07:44:53.776918+0000) 2024-03-07T07:44:58.428 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:44:58 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:44:58.263+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.270498+0000 front 2024-03-07T07:44:26.270653+0000 (oldest deadline 2024-03-07T07:44:49.770684+0000) 2024-03-07T07:44:58.428 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:44:58 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:44:58.263+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:35.671620+0000 front 2024-03-07T07:44:35.671862+0000 (oldest deadline 2024-03-07T07:44:57.971608+0000) 2024-03-07T07:44:58.556 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:44:58 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-5[48766]: 2024-03-07T07:44:58.128+0000 7f38c24f8700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:25.514679+0000 front 2024-03-07T07:44:25.514805+0000 (oldest deadline 2024-03-07T07:44:49.014605+0000) 2024-03-07T07:44:59.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:44:58 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:44:58.849+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:32.457358+0000 front 2024-03-07T07:44:32.457424+0000 (oldest deadline 2024-03-07T07:44:57.757460+0000) 2024-03-07T07:44:59.306 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:44:59 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-5[48766]: 2024-03-07T07:44:59.093+0000 7f38c24f8700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:25.514679+0000 front 2024-03-07T07:44:25.514805+0000 (oldest deadline 2024-03-07T07:44:49.014605+0000) 2024-03-07T07:44:59.307 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:44:59 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-5[48766]: 2024-03-07T07:44:59.093+0000 7f38c24f8700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:34.816361+0000 front 2024-03-07T07:44:34.816179+0000 (oldest deadline 2024-03-07T07:44:58.916130+0000) 2024-03-07T07:44:59.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:44:58 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:44:58.923+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:44:59.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:44:58 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:44:58.923+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:44:59.356 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:44:58 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:44:58.922+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:44:59.428 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:44:59 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:44:59.177+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.876690+0000 front 2024-03-07T07:44:27.876716+0000 (oldest deadline 2024-03-07T07:44:53.776918+0000) 2024-03-07T07:44:59.428 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:44:59 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:44:59.270+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.270498+0000 front 2024-03-07T07:44:26.270653+0000 (oldest deadline 2024-03-07T07:44:49.770684+0000) 2024-03-07T07:44:59.428 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:44:59 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:44:59.270+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:35.671620+0000 front 2024-03-07T07:44:35.671862+0000 (oldest deadline 2024-03-07T07:44:57.971608+0000) 2024-03-07T07:45:00.211 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:44:59 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:44:59.944+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:45:00.212 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:44:59 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:44:59.966+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:35.976112+0000 front 2024-03-07T07:44:35.976134+0000 (oldest deadline 2024-03-07T07:44:59.476430+0000) 2024-03-07T07:45:00.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:44:59 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:44:59.892+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:32.457358+0000 front 2024-03-07T07:44:32.457424+0000 (oldest deadline 2024-03-07T07:44:57.757460+0000) 2024-03-07T07:45:00.306 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:45:00 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-5[48766]: 2024-03-07T07:45:00.056+0000 7f38c24f8700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:25.514679+0000 front 2024-03-07T07:44:25.514805+0000 (oldest deadline 2024-03-07T07:44:49.014605+0000) 2024-03-07T07:45:00.306 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:45:00 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-5[48766]: 2024-03-07T07:45:00.056+0000 7f38c24f8700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:34.816361+0000 front 2024-03-07T07:44:34.816179+0000 (oldest deadline 2024-03-07T07:44:58.916130+0000) 2024-03-07T07:45:00.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:44:59 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:44:59.925+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:00.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:44:59 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:44:59.925+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:00.427 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:00 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:00.167+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.876690+0000 front 2024-03-07T07:44:27.876716+0000 (oldest deadline 2024-03-07T07:44:53.776918+0000) 2024-03-07T07:45:00.428 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:00 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:45:00.283+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.270498+0000 front 2024-03-07T07:44:26.270653+0000 (oldest deadline 2024-03-07T07:44:49.770684+0000) 2024-03-07T07:45:00.428 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:00 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:45:00.283+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:35.671620+0000 front 2024-03-07T07:44:35.671862+0000 (oldest deadline 2024-03-07T07:44:57.971608+0000) 2024-03-07T07:45:01.093 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:00 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:00.978+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:45:01.093 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:00 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:00.978+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:31.095363+0000 front 2024-03-07T07:44:31.095381+0000 (oldest deadline 2024-03-07T07:45:00.495899+0000) 2024-03-07T07:45:01.093 INFO:journalctl@ceph.osd.4.smithi177.stdout:Mar 07 07:45:00 smithi177 podman[70334]: 2024-03-07 07:45:00.789830418 +0000 UTC m=+5.523743452 container died 3a198df1c446799043b8809ac27d10b2962ea4d07f8fcee43aae75d554a6ccf5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-4, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-8b896a9, io.buildah.version=1.33.5) 2024-03-07T07:45:01.093 INFO:journalctl@ceph.osd.4.smithi177.stdout:Mar 07 07:45:01 smithi177 podman[70334]: 2024-03-07 07:45:01.093440936 +0000 UTC m=+5.827353961 container cleanup 3a198df1c446799043b8809ac27d10b2962ea4d07f8fcee43aae75d554a6ccf5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-4, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-03-07T07:45:01.093 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:01 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:01.011+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:35.976112+0000 front 2024-03-07T07:44:35.976134+0000 (oldest deadline 2024-03-07T07:44:59.476430+0000) 2024-03-07T07:45:01.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:00 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:00.893+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:32.457358+0000 front 2024-03-07T07:44:32.457424+0000 (oldest deadline 2024-03-07T07:44:57.757460+0000) 2024-03-07T07:45:01.307 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:45:01 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-5[48766]: 2024-03-07T07:45:01.071+0000 7f38c24f8700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:25.514679+0000 front 2024-03-07T07:44:25.514805+0000 (oldest deadline 2024-03-07T07:44:49.014605+0000) 2024-03-07T07:45:01.307 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:45:01 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-5[48766]: 2024-03-07T07:45:01.071+0000 7f38c24f8700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:34.816361+0000 front 2024-03-07T07:44:34.816179+0000 (oldest deadline 2024-03-07T07:44:58.916130+0000) 2024-03-07T07:45:01.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:00 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:00.912+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:01.308 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:00 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:00.912+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:01.356 INFO:journalctl@ceph.osd.4.smithi177.stdout:Mar 07 07:45:01 smithi177 bash[70334]: ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-4 2024-03-07T07:45:01.357 INFO:journalctl@ceph.osd.4.smithi177.stdout:Mar 07 07:45:01 smithi177 podman[70351]: 2024-03-07 07:45:01.328559851 +0000 UTC m=+0.539942185 container remove 3a198df1c446799043b8809ac27d10b2962ea4d07f8fcee43aae75d554a6ccf5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-4, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-03-07T07:45:01.428 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:01 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:01.172+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.876690+0000 front 2024-03-07T07:44:27.876716+0000 (oldest deadline 2024-03-07T07:44:53.776918+0000) 2024-03-07T07:45:01.428 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:01 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:45:01.333+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.270498+0000 front 2024-03-07T07:44:26.270653+0000 (oldest deadline 2024-03-07T07:44:49.770684+0000) 2024-03-07T07:45:01.428 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:01 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:45:01.333+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:35.671620+0000 front 2024-03-07T07:44:35.671862+0000 (oldest deadline 2024-03-07T07:44:57.971608+0000) 2024-03-07T07:45:01.906 INFO:journalctl@ceph.osd.4.smithi177.stdout:Mar 07 07:45:01 smithi177 podman[70442]: 2024-03-07 07:45:01.55871541 +0000 UTC m=+0.017758092 image pull e362cd7cebfee51077fb90b7f01895811140c9a4272edd83ad9910b05e3a46f6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2 2024-03-07T07:45:01.907 INFO:journalctl@ceph.osd.4.smithi177.stdout:Mar 07 07:45:01 smithi177 podman[70442]: 2024-03-07 07:45:01.662743219 +0000 UTC m=+0.121785907 container create a44cc8388295b8ac0d1de0ba164ddd1c35843716771d486aaa9af5b8181966e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-4-deactivate, ceph=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, RELEASE=wip-lusov-qdb-8b896a9, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-07T07:45:01.907 INFO:journalctl@ceph.osd.4.smithi177.stdout:Mar 07 07:45:01 smithi177 podman[70442]: 2024-03-07 07:45:01.906863905 +0000 UTC m=+0.365906574 container init a44cc8388295b8ac0d1de0ba164ddd1c35843716771d486aaa9af5b8181966e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-4-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-8b896a9, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-07T07:45:02.205 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:01 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:01.946+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:45:02.206 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:01 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:01.946+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:31.095363+0000 front 2024-03-07T07:44:31.095381+0000 (oldest deadline 2024-03-07T07:45:00.495899+0000) 2024-03-07T07:45:02.206 INFO:journalctl@ceph.osd.4.smithi177.stdout:Mar 07 07:45:01 smithi177 podman[70442]: 2024-03-07 07:45:01.910253791 +0000 UTC m=+0.369296454 container start a44cc8388295b8ac0d1de0ba164ddd1c35843716771d486aaa9af5b8181966e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-4-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-07T07:45:02.206 INFO:journalctl@ceph.osd.4.smithi177.stdout:Mar 07 07:45:01 smithi177 podman[70442]: 2024-03-07 07:45:01.998686561 +0000 UTC m=+0.457729248 container attach a44cc8388295b8ac0d1de0ba164ddd1c35843716771d486aaa9af5b8181966e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-4-deactivate, RELEASE=wip-lusov-qdb-8b896a9, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-07T07:45:02.206 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:01 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:01.969+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:35.976112+0000 front 2024-03-07T07:44:35.976134+0000 (oldest deadline 2024-03-07T07:44:59.476430+0000) 2024-03-07T07:45:02.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:01 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:01.922+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:32.457358+0000 front 2024-03-07T07:44:32.457424+0000 (oldest deadline 2024-03-07T07:44:57.757460+0000) 2024-03-07T07:45:02.306 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:45:02 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-5[48766]: 2024-03-07T07:45:02.115+0000 7f38c24f8700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:25.514679+0000 front 2024-03-07T07:44:25.514805+0000 (oldest deadline 2024-03-07T07:44:49.014605+0000) 2024-03-07T07:45:02.306 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:45:02 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-5[48766]: 2024-03-07T07:45:02.115+0000 7f38c24f8700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:34.816361+0000 front 2024-03-07T07:44:34.816179+0000 (oldest deadline 2024-03-07T07:44:58.916130+0000) 2024-03-07T07:45:02.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:01 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:01.905+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:02.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:01 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:01.905+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:02.467 INFO:journalctl@ceph.osd.4.smithi177.stdout:Mar 07 07:45:02 smithi177 podman[70442]: 2024-03-07 07:45:02.206442584 +0000 UTC m=+0.665485254 container died a44cc8388295b8ac0d1de0ba164ddd1c35843716771d486aaa9af5b8181966e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240222, GIT_CLEAN=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-8b896a9, CEPH_POINT_RELEASE=) 2024-03-07T07:45:02.678 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:02 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:45:02.369+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.270498+0000 front 2024-03-07T07:44:26.270653+0000 (oldest deadline 2024-03-07T07:44:49.770684+0000) 2024-03-07T07:45:02.678 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:02 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:45:02.369+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:35.671620+0000 front 2024-03-07T07:44:35.671862+0000 (oldest deadline 2024-03-07T07:44:57.971608+0000) 2024-03-07T07:45:02.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:02 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:02.207+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.876690+0000 front 2024-03-07T07:44:27.876716+0000 (oldest deadline 2024-03-07T07:44:53.776918+0000) 2024-03-07T07:45:02.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:02 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:02.207+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:36.078333+0000 front 2024-03-07T07:44:36.078350+0000 (oldest deadline 2024-03-07T07:45:01.978091+0000) 2024-03-07T07:45:02.812 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@osd.4.service' 2024-03-07T07:45:02.852 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-07T07:45:02.852 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-07T07:45:02.853 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-07T07:45:02.853 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-826f6582-dc54-11ee-95c6-87774f69a715@osd.5 2024-03-07T07:45:03.107 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:02 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:02.986+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:35.976112+0000 front 2024-03-07T07:44:35.976134+0000 (oldest deadline 2024-03-07T07:44:59.476430+0000) 2024-03-07T07:45:03.107 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:02 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:02.969+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:45:03.107 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:02 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:02.969+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:31.095363+0000 front 2024-03-07T07:44:31.095381+0000 (oldest deadline 2024-03-07T07:45:00.495899+0000) 2024-03-07T07:45:03.173 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:02 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:02.935+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:32.457358+0000 front 2024-03-07T07:44:32.457424+0000 (oldest deadline 2024-03-07T07:44:57.757460+0000) 2024-03-07T07:45:03.174 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:45:02 smithi178 systemd[1]: Stopping Ceph osd.5 for 826f6582-dc54-11ee-95c6-87774f69a715... 2024-03-07T07:45:03.174 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:45:03 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-5[48766]: 2024-03-07T07:45:03.083+0000 7f38c24f8700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:25.514679+0000 front 2024-03-07T07:44:25.514805+0000 (oldest deadline 2024-03-07T07:44:49.014605+0000) 2024-03-07T07:45:03.174 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:45:03 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-5[48766]: 2024-03-07T07:45:03.083+0000 7f38c24f8700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:34.816361+0000 front 2024-03-07T07:44:34.816179+0000 (oldest deadline 2024-03-07T07:44:58.916130+0000) 2024-03-07T07:45:03.174 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:02 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:02.938+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:03.174 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:02 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:02.938+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:03.556 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:45:03 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-5[48766]: 2024-03-07T07:45:03.173+0000 7f38c5f04700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-07T07:45:03.556 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:45:03 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-5[48766]: 2024-03-07T07:45:03.173+0000 7f38c5f04700 -1 osd.5 90 *** Got signal Terminated *** 2024-03-07T07:45:03.556 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:45:03 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-5[48766]: 2024-03-07T07:45:03.173+0000 7f38c5f04700 -1 osd.5 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-07T07:45:03.677 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:03 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:45:03.321+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.270498+0000 front 2024-03-07T07:44:26.270653+0000 (oldest deadline 2024-03-07T07:44:49.770684+0000) 2024-03-07T07:45:03.678 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:03 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:45:03.321+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:35.671620+0000 front 2024-03-07T07:44:35.671862+0000 (oldest deadline 2024-03-07T07:44:57.971608+0000) 2024-03-07T07:45:03.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:03 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:03.220+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.876690+0000 front 2024-03-07T07:44:27.876716+0000 (oldest deadline 2024-03-07T07:44:53.776918+0000) 2024-03-07T07:45:03.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:03 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:03.220+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:36.078333+0000 front 2024-03-07T07:44:36.078350+0000 (oldest deadline 2024-03-07T07:45:01.978091+0000) 2024-03-07T07:45:04.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:03 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:03.924+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:32.457358+0000 front 2024-03-07T07:44:32.457424+0000 (oldest deadline 2024-03-07T07:44:57.757460+0000) 2024-03-07T07:45:04.307 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:45:04 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-5[48766]: 2024-03-07T07:45:04.068+0000 7f38c24f8700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:25.514679+0000 front 2024-03-07T07:44:25.514805+0000 (oldest deadline 2024-03-07T07:44:49.014605+0000) 2024-03-07T07:45:04.307 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:45:04 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-5[48766]: 2024-03-07T07:45:04.068+0000 7f38c24f8700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:34.816361+0000 front 2024-03-07T07:44:34.816179+0000 (oldest deadline 2024-03-07T07:44:58.916130+0000) 2024-03-07T07:45:04.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:03 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:03.949+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:04.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:03 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:03.949+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:04.357 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:04 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:04.034+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:35.976112+0000 front 2024-03-07T07:44:35.976134+0000 (oldest deadline 2024-03-07T07:44:59.476430+0000) 2024-03-07T07:45:04.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:03 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:03.964+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:45:04.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:03 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:03.964+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:31.095363+0000 front 2024-03-07T07:44:31.095381+0000 (oldest deadline 2024-03-07T07:45:00.495899+0000) 2024-03-07T07:45:04.677 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:04 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:45:04.298+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.270498+0000 front 2024-03-07T07:44:26.270653+0000 (oldest deadline 2024-03-07T07:44:49.770684+0000) 2024-03-07T07:45:04.678 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:04 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:45:04.298+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:35.671620+0000 front 2024-03-07T07:44:35.671862+0000 (oldest deadline 2024-03-07T07:44:57.971608+0000) 2024-03-07T07:45:04.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:04 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:04.207+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.876690+0000 front 2024-03-07T07:44:27.876716+0000 (oldest deadline 2024-03-07T07:44:53.776918+0000) 2024-03-07T07:45:04.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:04 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:04.207+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:36.078333+0000 front 2024-03-07T07:44:36.078350+0000 (oldest deadline 2024-03-07T07:45:01.978091+0000) 2024-03-07T07:45:05.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:04 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:04.882+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:32.457358+0000 front 2024-03-07T07:44:32.457424+0000 (oldest deadline 2024-03-07T07:44:57.757460+0000) 2024-03-07T07:45:05.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:04 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:04.942+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:05.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:04 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:04.942+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:05.307 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:45:05 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-5[48766]: 2024-03-07T07:45:05.026+0000 7f38c24f8700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:25.514679+0000 front 2024-03-07T07:44:25.514805+0000 (oldest deadline 2024-03-07T07:44:49.014605+0000) 2024-03-07T07:45:05.307 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:45:05 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-5[48766]: 2024-03-07T07:45:05.026+0000 7f38c24f8700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:34.816361+0000 front 2024-03-07T07:44:34.816179+0000 (oldest deadline 2024-03-07T07:44:58.916130+0000) 2024-03-07T07:45:05.307 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:45:05 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-5[48766]: 2024-03-07T07:45:05.026+0000 7f38c24f8700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:40.017008+0000 front 2024-03-07T07:44:40.017031+0000 (oldest deadline 2024-03-07T07:45:04.717220+0000) 2024-03-07T07:45:05.356 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:05 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:05.046+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:35.976112+0000 front 2024-03-07T07:44:35.976134+0000 (oldest deadline 2024-03-07T07:44:59.476430+0000) 2024-03-07T07:45:05.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:05 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:05.002+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:45:05.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:05 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:05.002+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:31.095363+0000 front 2024-03-07T07:44:31.095381+0000 (oldest deadline 2024-03-07T07:45:00.495899+0000) 2024-03-07T07:45:05.677 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:05 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:45:05.269+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.270498+0000 front 2024-03-07T07:44:26.270653+0000 (oldest deadline 2024-03-07T07:44:49.770684+0000) 2024-03-07T07:45:05.678 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:05 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:45:05.269+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:35.671620+0000 front 2024-03-07T07:44:35.671862+0000 (oldest deadline 2024-03-07T07:44:57.971608+0000) 2024-03-07T07:45:05.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:05 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:05.235+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.876690+0000 front 2024-03-07T07:44:27.876716+0000 (oldest deadline 2024-03-07T07:44:53.776918+0000) 2024-03-07T07:45:05.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:05 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:05.235+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:36.078333+0000 front 2024-03-07T07:44:36.078350+0000 (oldest deadline 2024-03-07T07:45:01.978091+0000) 2024-03-07T07:45:06.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:05 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:05.912+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:32.457358+0000 front 2024-03-07T07:44:32.457424+0000 (oldest deadline 2024-03-07T07:44:57.757460+0000) 2024-03-07T07:45:06.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:05 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:05.899+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:06.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:05 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:05.899+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:06.307 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:45:06 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-5[48766]: 2024-03-07T07:45:06.063+0000 7f38c24f8700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:25.514679+0000 front 2024-03-07T07:44:25.514805+0000 (oldest deadline 2024-03-07T07:44:49.014605+0000) 2024-03-07T07:45:06.307 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:45:06 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-5[48766]: 2024-03-07T07:45:06.063+0000 7f38c24f8700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:34.816361+0000 front 2024-03-07T07:44:34.816179+0000 (oldest deadline 2024-03-07T07:44:58.916130+0000) 2024-03-07T07:45:06.307 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:45:06 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-5[48766]: 2024-03-07T07:45:06.063+0000 7f38c24f8700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:40.017008+0000 front 2024-03-07T07:44:40.017031+0000 (oldest deadline 2024-03-07T07:45:04.717220+0000) 2024-03-07T07:45:06.356 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:06 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:06.066+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:35.976112+0000 front 2024-03-07T07:44:35.976134+0000 (oldest deadline 2024-03-07T07:44:59.476430+0000) 2024-03-07T07:45:06.357 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:06 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:06.066+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:44.177363+0000 front 2024-03-07T07:44:44.177376+0000 (oldest deadline 2024-03-07T07:45:05.877409+0000) 2024-03-07T07:45:06.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:06 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:06.049+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:45:06.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:06 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:06.049+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:31.095363+0000 front 2024-03-07T07:44:31.095381+0000 (oldest deadline 2024-03-07T07:45:00.495899+0000) 2024-03-07T07:45:06.678 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:06 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:45:06.309+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.270498+0000 front 2024-03-07T07:44:26.270653+0000 (oldest deadline 2024-03-07T07:44:49.770684+0000) 2024-03-07T07:45:06.678 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:06 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:45:06.309+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:35.671620+0000 front 2024-03-07T07:44:35.671862+0000 (oldest deadline 2024-03-07T07:44:57.971608+0000) 2024-03-07T07:45:06.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:06 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:06.236+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.876690+0000 front 2024-03-07T07:44:27.876716+0000 (oldest deadline 2024-03-07T07:44:53.776918+0000) 2024-03-07T07:45:06.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:06 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:06.236+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:36.078333+0000 front 2024-03-07T07:44:36.078350+0000 (oldest deadline 2024-03-07T07:45:01.978091+0000) 2024-03-07T07:45:07.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:06 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:06.896+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:32.457358+0000 front 2024-03-07T07:44:32.457424+0000 (oldest deadline 2024-03-07T07:44:57.757460+0000) 2024-03-07T07:45:07.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:06 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:06.896+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:07.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:06 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:06.896+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:07.307 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:45:07 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-5[48766]: 2024-03-07T07:45:07.030+0000 7f38c24f8700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:25.514679+0000 front 2024-03-07T07:44:25.514805+0000 (oldest deadline 2024-03-07T07:44:49.014605+0000) 2024-03-07T07:45:07.307 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:45:07 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-5[48766]: 2024-03-07T07:45:07.030+0000 7f38c24f8700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:34.816361+0000 front 2024-03-07T07:44:34.816179+0000 (oldest deadline 2024-03-07T07:44:58.916130+0000) 2024-03-07T07:45:07.307 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:45:07 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-5[48766]: 2024-03-07T07:45:07.030+0000 7f38c24f8700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:40.017008+0000 front 2024-03-07T07:44:40.017031+0000 (oldest deadline 2024-03-07T07:45:04.717220+0000) 2024-03-07T07:45:07.356 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:07 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:07.027+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:35.976112+0000 front 2024-03-07T07:44:35.976134+0000 (oldest deadline 2024-03-07T07:44:59.476430+0000) 2024-03-07T07:45:07.357 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:07 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:07.027+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:44.177363+0000 front 2024-03-07T07:44:44.177376+0000 (oldest deadline 2024-03-07T07:45:05.877409+0000) 2024-03-07T07:45:07.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:07 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:07.040+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:45:07.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:07 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:07.040+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:31.095363+0000 front 2024-03-07T07:44:31.095381+0000 (oldest deadline 2024-03-07T07:45:00.495899+0000) 2024-03-07T07:45:07.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:07 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:07.206+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.876690+0000 front 2024-03-07T07:44:27.876716+0000 (oldest deadline 2024-03-07T07:44:53.776918+0000) 2024-03-07T07:45:07.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:07 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:07.206+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:36.078333+0000 front 2024-03-07T07:44:36.078350+0000 (oldest deadline 2024-03-07T07:45:01.978091+0000) 2024-03-07T07:45:07.678 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:07 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:45:07.305+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.270498+0000 front 2024-03-07T07:44:26.270653+0000 (oldest deadline 2024-03-07T07:44:49.770684+0000) 2024-03-07T07:45:07.678 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:07 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:45:07.305+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:35.671620+0000 front 2024-03-07T07:44:35.671862+0000 (oldest deadline 2024-03-07T07:44:57.971608+0000) 2024-03-07T07:45:07.678 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:07 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:45:07.305+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:41.373181+0000 front 2024-03-07T07:44:41.373203+0000 (oldest deadline 2024-03-07T07:45:06.673042+0000) 2024-03-07T07:45:08.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:07 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:07.870+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:32.457358+0000 front 2024-03-07T07:44:32.457424+0000 (oldest deadline 2024-03-07T07:44:57.757460+0000) 2024-03-07T07:45:08.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:07 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:07.870+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:44.159176+0000 front 2024-03-07T07:44:44.159293+0000 (oldest deadline 2024-03-07T07:45:07.059342+0000) 2024-03-07T07:45:08.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:07 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:07.919+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:08.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:07 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:07.919+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:08.307 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:45:08 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-5[48766]: 2024-03-07T07:45:08.019+0000 7f38c24f8700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:25.514679+0000 front 2024-03-07T07:44:25.514805+0000 (oldest deadline 2024-03-07T07:44:49.014605+0000) 2024-03-07T07:45:08.307 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:45:08 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-5[48766]: 2024-03-07T07:45:08.019+0000 7f38c24f8700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:34.816361+0000 front 2024-03-07T07:44:34.816179+0000 (oldest deadline 2024-03-07T07:44:58.916130+0000) 2024-03-07T07:45:08.307 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:45:08 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-5[48766]: 2024-03-07T07:45:08.019+0000 7f38c24f8700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:40.017008+0000 front 2024-03-07T07:44:40.017031+0000 (oldest deadline 2024-03-07T07:45:04.717220+0000) 2024-03-07T07:45:08.356 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:08 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:08.065+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:35.976112+0000 front 2024-03-07T07:44:35.976134+0000 (oldest deadline 2024-03-07T07:44:59.476430+0000) 2024-03-07T07:45:08.357 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:08 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:08.065+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:44.177363+0000 front 2024-03-07T07:44:44.177376+0000 (oldest deadline 2024-03-07T07:45:05.877409+0000) 2024-03-07T07:45:08.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:08 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:08.022+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:45:08.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:08 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:08.022+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:31.095363+0000 front 2024-03-07T07:44:31.095381+0000 (oldest deadline 2024-03-07T07:45:00.495899+0000) 2024-03-07T07:45:08.678 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:08 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:45:08.329+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.270498+0000 front 2024-03-07T07:44:26.270653+0000 (oldest deadline 2024-03-07T07:44:49.770684+0000) 2024-03-07T07:45:08.678 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:08 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:45:08.329+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:35.671620+0000 front 2024-03-07T07:44:35.671862+0000 (oldest deadline 2024-03-07T07:44:57.971608+0000) 2024-03-07T07:45:08.678 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:08 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:45:08.329+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:41.373181+0000 front 2024-03-07T07:44:41.373203+0000 (oldest deadline 2024-03-07T07:45:06.673042+0000) 2024-03-07T07:45:08.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:08 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:08.249+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.876690+0000 front 2024-03-07T07:44:27.876716+0000 (oldest deadline 2024-03-07T07:44:53.776918+0000) 2024-03-07T07:45:08.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:08 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:08.249+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:36.078333+0000 front 2024-03-07T07:44:36.078350+0000 (oldest deadline 2024-03-07T07:45:01.978091+0000) 2024-03-07T07:45:08.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:08 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:08.249+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:41.978705+0000 front 2024-03-07T07:44:41.978717+0000 (oldest deadline 2024-03-07T07:45:07.278468+0000) 2024-03-07T07:45:08.887 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:45:08 smithi178 podman[72092]: 2024-03-07 07:45:08.629936292 +0000 UTC m=+5.573682414 container died b4fa9bf9b2b46c64a07db57bad2470b02e40a21e2b18edf42964d946ab169d91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-5, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-lusov-qdb-8b896a9, CEPH_POINT_RELEASE=) 2024-03-07T07:45:09.258 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:45:09 smithi178 podman[72092]: 2024-03-07 07:45:09.032257375 +0000 UTC m=+5.976003489 container cleanup b4fa9bf9b2b46c64a07db57bad2470b02e40a21e2b18edf42964d946ab169d91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.build-date=20240222, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-07T07:45:09.258 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:45:09 smithi178 bash[72092]: ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-5 2024-03-07T07:45:09.258 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:08 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:08.886+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:32.457358+0000 front 2024-03-07T07:44:32.457424+0000 (oldest deadline 2024-03-07T07:44:57.757460+0000) 2024-03-07T07:45:09.258 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:08 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:08.886+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:44.159176+0000 front 2024-03-07T07:44:44.159293+0000 (oldest deadline 2024-03-07T07:45:07.059342+0000) 2024-03-07T07:45:09.259 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:08 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:08.951+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:09.259 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:08 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:08.951+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:09.259 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:08 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:08.951+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:42.970262+0000 front 2024-03-07T07:44:42.970339+0000 (oldest deadline 2024-03-07T07:45:08.870388+0000) 2024-03-07T07:45:09.356 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:09 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:09.092+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:35.976112+0000 front 2024-03-07T07:44:35.976134+0000 (oldest deadline 2024-03-07T07:44:59.476430+0000) 2024-03-07T07:45:09.357 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:09 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:09.092+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:44.177363+0000 front 2024-03-07T07:44:44.177376+0000 (oldest deadline 2024-03-07T07:45:05.877409+0000) 2024-03-07T07:45:09.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:09 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:09.003+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:45:09.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:09 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:09.003+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:31.095363+0000 front 2024-03-07T07:44:31.095381+0000 (oldest deadline 2024-03-07T07:45:00.495899+0000) 2024-03-07T07:45:09.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:09 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:09.003+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:40.496222+0000 front 2024-03-07T07:44:40.496309+0000 (oldest deadline 2024-03-07T07:45:08.696933+0000) 2024-03-07T07:45:09.556 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:45:09 smithi178 podman[72109]: 2024-03-07 07:45:09.258661561 +0000 UTC m=+0.628352876 container remove b4fa9bf9b2b46c64a07db57bad2470b02e40a21e2b18edf42964d946ab169d91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-5, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-8b896a9) 2024-03-07T07:45:09.678 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:09 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:45:09.354+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.270498+0000 front 2024-03-07T07:44:26.270653+0000 (oldest deadline 2024-03-07T07:44:49.770684+0000) 2024-03-07T07:45:09.678 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:09 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:45:09.354+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:35.671620+0000 front 2024-03-07T07:44:35.671862+0000 (oldest deadline 2024-03-07T07:44:57.971608+0000) 2024-03-07T07:45:09.678 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:09 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:45:09.354+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:41.373181+0000 front 2024-03-07T07:44:41.373203+0000 (oldest deadline 2024-03-07T07:45:06.673042+0000) 2024-03-07T07:45:09.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:09 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:09.294+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.876690+0000 front 2024-03-07T07:44:27.876716+0000 (oldest deadline 2024-03-07T07:44:53.776918+0000) 2024-03-07T07:45:09.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:09 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:09.294+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:36.078333+0000 front 2024-03-07T07:44:36.078350+0000 (oldest deadline 2024-03-07T07:45:01.978091+0000) 2024-03-07T07:45:09.679 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:09 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:09.294+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:41.978705+0000 front 2024-03-07T07:44:41.978717+0000 (oldest deadline 2024-03-07T07:45:07.278468+0000) 2024-03-07T07:45:09.854 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:45:09 smithi178 podman[72201]: 2024-03-07 07:45:09.493849157 +0000 UTC m=+0.018026352 image pull e362cd7cebfee51077fb90b7f01895811140c9a4272edd83ad9910b05e3a46f6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2 2024-03-07T07:45:09.854 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:45:09 smithi178 podman[72201]: 2024-03-07 07:45:09.601398328 +0000 UTC m=+0.125575520 container create 1e3598ae0a0ad8ac99de6a09650513dc02d6669962641cad6a52c53cc7572c30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-8b896a9) 2024-03-07T07:45:10.152 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:09 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:09.863+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:32.457358+0000 front 2024-03-07T07:44:32.457424+0000 (oldest deadline 2024-03-07T07:44:57.757460+0000) 2024-03-07T07:45:10.152 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:09 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:09.863+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:44.159176+0000 front 2024-03-07T07:44:44.159293+0000 (oldest deadline 2024-03-07T07:45:07.059342+0000) 2024-03-07T07:45:10.153 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:09 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:09.940+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:10.153 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:09 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:09.940+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:10.153 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:09 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:09.940+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:42.970262+0000 front 2024-03-07T07:44:42.970339+0000 (oldest deadline 2024-03-07T07:45:08.870388+0000) 2024-03-07T07:45:10.153 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:45:09 smithi178 podman[72201]: 2024-03-07 07:45:09.854321173 +0000 UTC m=+0.378498370 container init 1e3598ae0a0ad8ac99de6a09650513dc02d6669962641cad6a52c53cc7572c30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-5-deactivate, GIT_CLEAN=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-8b896a9, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-07T07:45:10.153 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:45:09 smithi178 podman[72201]: 2024-03-07 07:45:09.857516047 +0000 UTC m=+0.381693233 container start 1e3598ae0a0ad8ac99de6a09650513dc02d6669962641cad6a52c53cc7572c30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-5-deactivate, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b) 2024-03-07T07:45:10.153 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:45:09 smithi178 podman[72201]: 2024-03-07 07:45:09.945959601 +0000 UTC m=+0.470136805 container attach 1e3598ae0a0ad8ac99de6a09650513dc02d6669962641cad6a52c53cc7572c30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240222, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.vendor=CentOS, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, io.buildah.version=1.33.5) 2024-03-07T07:45:10.357 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:10 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:10.099+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:35.976112+0000 front 2024-03-07T07:44:35.976134+0000 (oldest deadline 2024-03-07T07:44:59.476430+0000) 2024-03-07T07:45:10.357 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:10 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:10.099+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:44.177363+0000 front 2024-03-07T07:44:44.177376+0000 (oldest deadline 2024-03-07T07:45:05.877409+0000) 2024-03-07T07:45:10.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:10 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:10.026+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:45:10.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:10 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:10.026+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:31.095363+0000 front 2024-03-07T07:44:31.095381+0000 (oldest deadline 2024-03-07T07:45:00.495899+0000) 2024-03-07T07:45:10.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:10 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:10.026+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:40.496222+0000 front 2024-03-07T07:44:40.496309+0000 (oldest deadline 2024-03-07T07:45:08.696933+0000) 2024-03-07T07:45:10.440 INFO:journalctl@ceph.osd.5.smithi178.stdout:Mar 07 07:45:10 smithi178 podman[72201]: 2024-03-07 07:45:10.153195758 +0000 UTC m=+0.677372944 container died 1e3598ae0a0ad8ac99de6a09650513dc02d6669962641cad6a52c53cc7572c30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240222, RELEASE=wip-lusov-qdb-8b896a9, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-07T07:45:10.678 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:10 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:45:10.317+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.270498+0000 front 2024-03-07T07:44:26.270653+0000 (oldest deadline 2024-03-07T07:44:49.770684+0000) 2024-03-07T07:45:10.678 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:10 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:45:10.317+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:35.671620+0000 front 2024-03-07T07:44:35.671862+0000 (oldest deadline 2024-03-07T07:44:57.971608+0000) 2024-03-07T07:45:10.678 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:10 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:45:10.317+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:41.373181+0000 front 2024-03-07T07:44:41.373203+0000 (oldest deadline 2024-03-07T07:45:06.673042+0000) 2024-03-07T07:45:10.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:10 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:10.284+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.876690+0000 front 2024-03-07T07:44:27.876716+0000 (oldest deadline 2024-03-07T07:44:53.776918+0000) 2024-03-07T07:45:10.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:10 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:10.284+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:36.078333+0000 front 2024-03-07T07:44:36.078350+0000 (oldest deadline 2024-03-07T07:45:01.978091+0000) 2024-03-07T07:45:10.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:10 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:10.284+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:41.978705+0000 front 2024-03-07T07:44:41.978717+0000 (oldest deadline 2024-03-07T07:45:07.278468+0000) 2024-03-07T07:45:10.721 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@osd.5.service' 2024-03-07T07:45:10.761 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-07T07:45:10.761 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-07T07:45:10.761 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-07T07:45:10.761 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-826f6582-dc54-11ee-95c6-87774f69a715@osd.6 2024-03-07T07:45:11.056 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:10 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:10.914+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:32.457358+0000 front 2024-03-07T07:44:32.457424+0000 (oldest deadline 2024-03-07T07:44:57.757460+0000) 2024-03-07T07:45:11.056 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:10 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:10.914+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:44.159176+0000 front 2024-03-07T07:44:44.159293+0000 (oldest deadline 2024-03-07T07:45:07.059342+0000) 2024-03-07T07:45:11.057 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:10 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:10.893+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:11.057 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:10 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:10.893+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:11.057 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:10 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:10.893+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:42.970262+0000 front 2024-03-07T07:44:42.970339+0000 (oldest deadline 2024-03-07T07:45:08.870388+0000) 2024-03-07T07:45:11.062 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:10 smithi160 systemd[1]: Stopping Ceph osd.6 for 826f6582-dc54-11ee-95c6-87774f69a715... 2024-03-07T07:45:11.357 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:11 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:11.082+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:35.976112+0000 front 2024-03-07T07:44:35.976134+0000 (oldest deadline 2024-03-07T07:44:59.476430+0000) 2024-03-07T07:45:11.357 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:11 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:11.082+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:44.177363+0000 front 2024-03-07T07:44:44.177376+0000 (oldest deadline 2024-03-07T07:45:05.877409+0000) 2024-03-07T07:45:11.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:11 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:11.003+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:45:11.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:11 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:11.003+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:31.095363+0000 front 2024-03-07T07:44:31.095381+0000 (oldest deadline 2024-03-07T07:45:00.495899+0000) 2024-03-07T07:45:11.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:11 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:11.003+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:40.496222+0000 front 2024-03-07T07:44:40.496309+0000 (oldest deadline 2024-03-07T07:45:08.696933+0000) 2024-03-07T07:45:11.361 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:11 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:11.281+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.876690+0000 front 2024-03-07T07:44:27.876716+0000 (oldest deadline 2024-03-07T07:44:53.776918+0000) 2024-03-07T07:45:11.361 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:11 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:11.281+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:36.078333+0000 front 2024-03-07T07:44:36.078350+0000 (oldest deadline 2024-03-07T07:45:01.978091+0000) 2024-03-07T07:45:11.361 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:11 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:11.281+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:41.978705+0000 front 2024-03-07T07:44:41.978717+0000 (oldest deadline 2024-03-07T07:45:07.278468+0000) 2024-03-07T07:45:11.361 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:11 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:45:11.062+0000 7f0cc9c2e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-07T07:45:11.361 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:11 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:45:11.062+0000 7f0cc9c2e700 -1 osd.6 90 *** Got signal Terminated *** 2024-03-07T07:45:11.361 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:11 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:45:11.062+0000 7f0cc9c2e700 -1 osd.6 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-07T07:45:11.361 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:11 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:45:11.361+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.270498+0000 front 2024-03-07T07:44:26.270653+0000 (oldest deadline 2024-03-07T07:44:49.770684+0000) 2024-03-07T07:45:11.362 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:11 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:45:11.361+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:35.671620+0000 front 2024-03-07T07:44:35.671862+0000 (oldest deadline 2024-03-07T07:44:57.971608+0000) 2024-03-07T07:45:11.362 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:11 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:45:11.361+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:41.373181+0000 front 2024-03-07T07:44:41.373203+0000 (oldest deadline 2024-03-07T07:45:06.673042+0000) 2024-03-07T07:45:12.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:11 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:11.866+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:32.457358+0000 front 2024-03-07T07:44:32.457424+0000 (oldest deadline 2024-03-07T07:44:57.757460+0000) 2024-03-07T07:45:12.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:11 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:11.866+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:44.159176+0000 front 2024-03-07T07:44:44.159293+0000 (oldest deadline 2024-03-07T07:45:07.059342+0000) 2024-03-07T07:45:12.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:11 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:11.916+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:12.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:11 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:11.916+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:12.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:11 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:11.916+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:42.970262+0000 front 2024-03-07T07:44:42.970339+0000 (oldest deadline 2024-03-07T07:45:08.870388+0000) 2024-03-07T07:45:12.357 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:12 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:12.094+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:35.976112+0000 front 2024-03-07T07:44:35.976134+0000 (oldest deadline 2024-03-07T07:44:59.476430+0000) 2024-03-07T07:45:12.357 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:12 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:12.094+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:44.177363+0000 front 2024-03-07T07:44:44.177376+0000 (oldest deadline 2024-03-07T07:45:05.877409+0000) 2024-03-07T07:45:12.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:12 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:12.048+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:45:12.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:12 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:12.048+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:31.095363+0000 front 2024-03-07T07:44:31.095381+0000 (oldest deadline 2024-03-07T07:45:00.495899+0000) 2024-03-07T07:45:12.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:12 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:12.048+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:40.496222+0000 front 2024-03-07T07:44:40.496309+0000 (oldest deadline 2024-03-07T07:45:08.696933+0000) 2024-03-07T07:45:12.677 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:12 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:45:12.377+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.270498+0000 front 2024-03-07T07:44:26.270653+0000 (oldest deadline 2024-03-07T07:44:49.770684+0000) 2024-03-07T07:45:12.678 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:12 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:45:12.377+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:35.671620+0000 front 2024-03-07T07:44:35.671862+0000 (oldest deadline 2024-03-07T07:44:57.971608+0000) 2024-03-07T07:45:12.678 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:12 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:45:12.377+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:41.373181+0000 front 2024-03-07T07:44:41.373203+0000 (oldest deadline 2024-03-07T07:45:06.673042+0000) 2024-03-07T07:45:12.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:12 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:12.256+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.876690+0000 front 2024-03-07T07:44:27.876716+0000 (oldest deadline 2024-03-07T07:44:53.776918+0000) 2024-03-07T07:45:12.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:12 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:12.256+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:36.078333+0000 front 2024-03-07T07:44:36.078350+0000 (oldest deadline 2024-03-07T07:45:01.978091+0000) 2024-03-07T07:45:12.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:12 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:12.256+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:41.978705+0000 front 2024-03-07T07:44:41.978717+0000 (oldest deadline 2024-03-07T07:45:07.278468+0000) 2024-03-07T07:45:13.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:12 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:12.893+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:32.457358+0000 front 2024-03-07T07:44:32.457424+0000 (oldest deadline 2024-03-07T07:44:57.757460+0000) 2024-03-07T07:45:13.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:12 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:12.893+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:44.159176+0000 front 2024-03-07T07:44:44.159293+0000 (oldest deadline 2024-03-07T07:45:07.059342+0000) 2024-03-07T07:45:13.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:12 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:12.937+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:13.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:12 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:12.937+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:13.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:12 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:12.937+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:42.970262+0000 front 2024-03-07T07:44:42.970339+0000 (oldest deadline 2024-03-07T07:45:08.870388+0000) 2024-03-07T07:45:13.357 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:13 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:13.063+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:35.976112+0000 front 2024-03-07T07:44:35.976134+0000 (oldest deadline 2024-03-07T07:44:59.476430+0000) 2024-03-07T07:45:13.357 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:13 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:13.063+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:44.177363+0000 front 2024-03-07T07:44:44.177376+0000 (oldest deadline 2024-03-07T07:45:05.877409+0000) 2024-03-07T07:45:13.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:13 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:13.077+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:45:13.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:13 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:13.077+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:31.095363+0000 front 2024-03-07T07:44:31.095381+0000 (oldest deadline 2024-03-07T07:45:00.495899+0000) 2024-03-07T07:45:13.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:13 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:13.077+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:40.496222+0000 front 2024-03-07T07:44:40.496309+0000 (oldest deadline 2024-03-07T07:45:08.696933+0000) 2024-03-07T07:45:13.678 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:13 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:45:13.404+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.270498+0000 front 2024-03-07T07:44:26.270653+0000 (oldest deadline 2024-03-07T07:44:49.770684+0000) 2024-03-07T07:45:13.678 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:13 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:45:13.404+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:35.671620+0000 front 2024-03-07T07:44:35.671862+0000 (oldest deadline 2024-03-07T07:44:57.971608+0000) 2024-03-07T07:45:13.678 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:13 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:45:13.404+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:41.373181+0000 front 2024-03-07T07:44:41.373203+0000 (oldest deadline 2024-03-07T07:45:06.673042+0000) 2024-03-07T07:45:13.678 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:13 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:45:13.404+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:48.973793+0000 front 2024-03-07T07:44:48.973820+0000 (oldest deadline 2024-03-07T07:45:13.073922+0000) 2024-03-07T07:45:13.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:13 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:13.298+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.876690+0000 front 2024-03-07T07:44:27.876716+0000 (oldest deadline 2024-03-07T07:44:53.776918+0000) 2024-03-07T07:45:13.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:13 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:13.298+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:36.078333+0000 front 2024-03-07T07:44:36.078350+0000 (oldest deadline 2024-03-07T07:45:01.978091+0000) 2024-03-07T07:45:13.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:13 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:13.298+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:41.978705+0000 front 2024-03-07T07:44:41.978717+0000 (oldest deadline 2024-03-07T07:45:07.278468+0000) 2024-03-07T07:45:13.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:13 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:13.298+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:50.179200+0000 front 2024-03-07T07:44:50.179222+0000 (oldest deadline 2024-03-07T07:45:13.079347+0000) 2024-03-07T07:45:14.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:13 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:13.895+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:32.457358+0000 front 2024-03-07T07:44:32.457424+0000 (oldest deadline 2024-03-07T07:44:57.757460+0000) 2024-03-07T07:45:14.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:13 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:13.895+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:44.159176+0000 front 2024-03-07T07:44:44.159293+0000 (oldest deadline 2024-03-07T07:45:07.059342+0000) 2024-03-07T07:45:14.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:13 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:13.895+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.960471+0000 front 2024-03-07T07:44:49.960386+0000 (oldest deadline 2024-03-07T07:45:13.360864+0000) 2024-03-07T07:45:14.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:13 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:13.966+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:14.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:13 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:13.967+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:14.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:13 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:13.967+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:42.970262+0000 front 2024-03-07T07:44:42.970339+0000 (oldest deadline 2024-03-07T07:45:08.870388+0000) 2024-03-07T07:45:14.356 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:14 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:14.059+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:45:14.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:14 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:14.059+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:31.095363+0000 front 2024-03-07T07:44:31.095381+0000 (oldest deadline 2024-03-07T07:45:00.495899+0000) 2024-03-07T07:45:14.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:14 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:14.059+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:40.496222+0000 front 2024-03-07T07:44:40.496309+0000 (oldest deadline 2024-03-07T07:45:08.696933+0000) 2024-03-07T07:45:14.357 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:14 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:14.086+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:35.976112+0000 front 2024-03-07T07:44:35.976134+0000 (oldest deadline 2024-03-07T07:44:59.476430+0000) 2024-03-07T07:45:14.357 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:14 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:14.086+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:44.177363+0000 front 2024-03-07T07:44:44.177376+0000 (oldest deadline 2024-03-07T07:45:05.877409+0000) 2024-03-07T07:45:14.357 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:14 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:14.086+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:51.678875+0000 front 2024-03-07T07:44:51.679003+0000 (oldest deadline 2024-03-07T07:45:13.379016+0000) 2024-03-07T07:45:14.678 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:14 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:45:14.385+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.270498+0000 front 2024-03-07T07:44:26.270653+0000 (oldest deadline 2024-03-07T07:44:49.770684+0000) 2024-03-07T07:45:14.678 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:14 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:45:14.385+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:35.671620+0000 front 2024-03-07T07:44:35.671862+0000 (oldest deadline 2024-03-07T07:44:57.971608+0000) 2024-03-07T07:45:14.678 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:14 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:45:14.385+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:41.373181+0000 front 2024-03-07T07:44:41.373203+0000 (oldest deadline 2024-03-07T07:45:06.673042+0000) 2024-03-07T07:45:14.678 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:14 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:45:14.385+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:48.973793+0000 front 2024-03-07T07:44:48.973820+0000 (oldest deadline 2024-03-07T07:45:13.073922+0000) 2024-03-07T07:45:14.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:14 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:14.327+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.876690+0000 front 2024-03-07T07:44:27.876716+0000 (oldest deadline 2024-03-07T07:44:53.776918+0000) 2024-03-07T07:45:14.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:14 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:14.327+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:36.078333+0000 front 2024-03-07T07:44:36.078350+0000 (oldest deadline 2024-03-07T07:45:01.978091+0000) 2024-03-07T07:45:14.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:14 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:14.327+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:41.978705+0000 front 2024-03-07T07:44:41.978717+0000 (oldest deadline 2024-03-07T07:45:07.278468+0000) 2024-03-07T07:45:14.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:14 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:14.327+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:50.179200+0000 front 2024-03-07T07:44:50.179222+0000 (oldest deadline 2024-03-07T07:45:13.079347+0000) 2024-03-07T07:45:15.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:14 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:14.919+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:32.457358+0000 front 2024-03-07T07:44:32.457424+0000 (oldest deadline 2024-03-07T07:44:57.757460+0000) 2024-03-07T07:45:15.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:14 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:14.919+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:44.159176+0000 front 2024-03-07T07:44:44.159293+0000 (oldest deadline 2024-03-07T07:45:07.059342+0000) 2024-03-07T07:45:15.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:14 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:14.919+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.960471+0000 front 2024-03-07T07:44:49.960386+0000 (oldest deadline 2024-03-07T07:45:13.360864+0000) 2024-03-07T07:45:15.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:15 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:15.008+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:15.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:15 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:15.008+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:15.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:15 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:15.008+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:42.970262+0000 front 2024-03-07T07:44:42.970339+0000 (oldest deadline 2024-03-07T07:45:08.870388+0000) 2024-03-07T07:45:15.357 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:15 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:15.132+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:35.976112+0000 front 2024-03-07T07:44:35.976134+0000 (oldest deadline 2024-03-07T07:44:59.476430+0000) 2024-03-07T07:45:15.357 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:15 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:15.132+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:44.177363+0000 front 2024-03-07T07:44:44.177376+0000 (oldest deadline 2024-03-07T07:45:05.877409+0000) 2024-03-07T07:45:15.357 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:15 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:15.132+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:51.678875+0000 front 2024-03-07T07:44:51.679003+0000 (oldest deadline 2024-03-07T07:45:13.379016+0000) 2024-03-07T07:45:15.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:15 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:15.009+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:45:15.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:15 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:15.009+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:31.095363+0000 front 2024-03-07T07:44:31.095381+0000 (oldest deadline 2024-03-07T07:45:00.495899+0000) 2024-03-07T07:45:15.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:15 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:15.009+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:40.496222+0000 front 2024-03-07T07:44:40.496309+0000 (oldest deadline 2024-03-07T07:45:08.696933+0000) 2024-03-07T07:45:15.677 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:15 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:45:15.358+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.270498+0000 front 2024-03-07T07:44:26.270653+0000 (oldest deadline 2024-03-07T07:44:49.770684+0000) 2024-03-07T07:45:15.678 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:15 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:45:15.358+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:35.671620+0000 front 2024-03-07T07:44:35.671862+0000 (oldest deadline 2024-03-07T07:44:57.971608+0000) 2024-03-07T07:45:15.678 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:15 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:45:15.358+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:41.373181+0000 front 2024-03-07T07:44:41.373203+0000 (oldest deadline 2024-03-07T07:45:06.673042+0000) 2024-03-07T07:45:15.678 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:15 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6[60423]: 2024-03-07T07:45:15.358+0000 7f0cc221a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:48.973793+0000 front 2024-03-07T07:44:48.973820+0000 (oldest deadline 2024-03-07T07:45:13.073922+0000) 2024-03-07T07:45:15.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:15 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:15.284+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.876690+0000 front 2024-03-07T07:44:27.876716+0000 (oldest deadline 2024-03-07T07:44:53.776918+0000) 2024-03-07T07:45:15.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:15 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:15.284+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:36.078333+0000 front 2024-03-07T07:44:36.078350+0000 (oldest deadline 2024-03-07T07:45:01.978091+0000) 2024-03-07T07:45:15.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:15 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:15.284+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:41.978705+0000 front 2024-03-07T07:44:41.978717+0000 (oldest deadline 2024-03-07T07:45:07.278468+0000) 2024-03-07T07:45:15.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:15 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:15.284+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:50.179200+0000 front 2024-03-07T07:44:50.179222+0000 (oldest deadline 2024-03-07T07:45:13.079347+0000) 2024-03-07T07:45:16.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:15 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:15.942+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:32.457358+0000 front 2024-03-07T07:44:32.457424+0000 (oldest deadline 2024-03-07T07:44:57.757460+0000) 2024-03-07T07:45:16.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:15 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:15.942+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:44.159176+0000 front 2024-03-07T07:44:44.159293+0000 (oldest deadline 2024-03-07T07:45:07.059342+0000) 2024-03-07T07:45:16.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:15 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:15.942+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.960471+0000 front 2024-03-07T07:44:49.960386+0000 (oldest deadline 2024-03-07T07:45:13.360864+0000) 2024-03-07T07:45:16.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:15 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:15.995+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:16.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:15 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:15.995+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:16.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:15 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:15.995+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:42.970262+0000 front 2024-03-07T07:44:42.970339+0000 (oldest deadline 2024-03-07T07:45:08.870388+0000) 2024-03-07T07:45:16.356 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:16 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:16.118+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:35.976112+0000 front 2024-03-07T07:44:35.976134+0000 (oldest deadline 2024-03-07T07:44:59.476430+0000) 2024-03-07T07:45:16.357 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:16 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:16.118+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:44.177363+0000 front 2024-03-07T07:44:44.177376+0000 (oldest deadline 2024-03-07T07:45:05.877409+0000) 2024-03-07T07:45:16.357 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:16 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:16.118+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:51.678875+0000 front 2024-03-07T07:44:51.679003+0000 (oldest deadline 2024-03-07T07:45:13.379016+0000) 2024-03-07T07:45:16.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:15 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:15.975+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:45:16.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:15 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:15.975+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:31.095363+0000 front 2024-03-07T07:44:31.095381+0000 (oldest deadline 2024-03-07T07:45:00.495899+0000) 2024-03-07T07:45:16.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:15 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:15.975+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:40.496222+0000 front 2024-03-07T07:44:40.496309+0000 (oldest deadline 2024-03-07T07:45:08.696933+0000) 2024-03-07T07:45:16.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:15 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:15.975+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.798043+0000 front 2024-03-07T07:44:49.797936+0000 (oldest deadline 2024-03-07T07:45:15.097839+0000) 2024-03-07T07:45:16.677 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:16 smithi160 podman[88032]: 2024-03-07 07:45:16.508545925 +0000 UTC m=+5.547362953 container died 76f2218e15b28a71ffc47e368a0833164454e5d92f41e3b08cc28f14758295f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222) 2024-03-07T07:45:16.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:16 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:16.314+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.876690+0000 front 2024-03-07T07:44:27.876716+0000 (oldest deadline 2024-03-07T07:44:53.776918+0000) 2024-03-07T07:45:16.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:16 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:16.314+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:36.078333+0000 front 2024-03-07T07:44:36.078350+0000 (oldest deadline 2024-03-07T07:45:01.978091+0000) 2024-03-07T07:45:16.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:16 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:16.314+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:41.978705+0000 front 2024-03-07T07:44:41.978717+0000 (oldest deadline 2024-03-07T07:45:07.278468+0000) 2024-03-07T07:45:16.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:16 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:16.314+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:50.179200+0000 front 2024-03-07T07:44:50.179222+0000 (oldest deadline 2024-03-07T07:45:13.079347+0000) 2024-03-07T07:45:17.178 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:16 smithi160 podman[88032]: 2024-03-07 07:45:16.979381914 +0000 UTC m=+6.018198987 container cleanup 76f2218e15b28a71ffc47e368a0833164454e5d92f41e3b08cc28f14758295f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, RELEASE=wip-lusov-qdb-8b896a9, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222) 2024-03-07T07:45:17.178 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:16 smithi160 bash[88032]: ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6 2024-03-07T07:45:17.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:16 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:16.960+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:32.457358+0000 front 2024-03-07T07:44:32.457424+0000 (oldest deadline 2024-03-07T07:44:57.757460+0000) 2024-03-07T07:45:17.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:16 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:16.960+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:44.159176+0000 front 2024-03-07T07:44:44.159293+0000 (oldest deadline 2024-03-07T07:45:07.059342+0000) 2024-03-07T07:45:17.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:16 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:16.960+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.960471+0000 front 2024-03-07T07:44:49.960386+0000 (oldest deadline 2024-03-07T07:45:13.360864+0000) 2024-03-07T07:45:17.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:17 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:17.029+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:17.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:17 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:17.029+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:17.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:17 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:17.029+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:42.970262+0000 front 2024-03-07T07:44:42.970339+0000 (oldest deadline 2024-03-07T07:45:08.870388+0000) 2024-03-07T07:45:17.357 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:17 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:17.123+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:35.976112+0000 front 2024-03-07T07:44:35.976134+0000 (oldest deadline 2024-03-07T07:44:59.476430+0000) 2024-03-07T07:45:17.357 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:17 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:17.123+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:44.177363+0000 front 2024-03-07T07:44:44.177376+0000 (oldest deadline 2024-03-07T07:45:05.877409+0000) 2024-03-07T07:45:17.357 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:17 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:17.123+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:51.678875+0000 front 2024-03-07T07:44:51.679003+0000 (oldest deadline 2024-03-07T07:45:13.379016+0000) 2024-03-07T07:45:17.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:16 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:16.931+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:45:17.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:16 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:16.931+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:31.095363+0000 front 2024-03-07T07:44:31.095381+0000 (oldest deadline 2024-03-07T07:45:00.495899+0000) 2024-03-07T07:45:17.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:16 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:16.931+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:40.496222+0000 front 2024-03-07T07:44:40.496309+0000 (oldest deadline 2024-03-07T07:45:08.696933+0000) 2024-03-07T07:45:17.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:16 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:16.931+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.798043+0000 front 2024-03-07T07:44:49.797936+0000 (oldest deadline 2024-03-07T07:45:15.097839+0000) 2024-03-07T07:45:17.678 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:17 smithi160 podman[88049]: 2024-03-07 07:45:17.373005964 +0000 UTC m=+0.853002252 container remove 76f2218e15b28a71ffc47e368a0833164454e5d92f41e3b08cc28f14758295f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-8b896a9, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-07T07:45:17.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:17 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:17.265+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.876690+0000 front 2024-03-07T07:44:27.876716+0000 (oldest deadline 2024-03-07T07:44:53.776918+0000) 2024-03-07T07:45:17.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:17 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:17.265+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:36.078333+0000 front 2024-03-07T07:44:36.078350+0000 (oldest deadline 2024-03-07T07:45:01.978091+0000) 2024-03-07T07:45:17.679 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:17 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:17.265+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:41.978705+0000 front 2024-03-07T07:44:41.978717+0000 (oldest deadline 2024-03-07T07:45:07.278468+0000) 2024-03-07T07:45:17.679 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:17 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:17.265+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:50.179200+0000 front 2024-03-07T07:44:50.179222+0000 (oldest deadline 2024-03-07T07:45:13.079347+0000) 2024-03-07T07:45:18.051 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:17 smithi160 podman[88143]: 2024-03-07 07:45:17.643121524 +0000 UTC m=+0.019410014 image pull e362cd7cebfee51077fb90b7f01895811140c9a4272edd83ad9910b05e3a46f6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2 2024-03-07T07:45:18.051 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:17 smithi160 podman[88143]: 2024-03-07 07:45:17.785171498 +0000 UTC m=+0.161459969 container create 298d2eaa7f74ea46fad6766ac657b3ce517def1256237f2875d0d95276f64eca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-07T07:45:18.051 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:18 smithi160 podman[88143]: 2024-03-07 07:45:18.051935826 +0000 UTC m=+0.428224297 container init 298d2eaa7f74ea46fad6766ac657b3ce517def1256237f2875d0d95276f64eca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, GIT_CLEAN=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-8b896a9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-07T07:45:18.167 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:17 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:17.893+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:45:18.168 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:17 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:17.893+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:31.095363+0000 front 2024-03-07T07:44:31.095381+0000 (oldest deadline 2024-03-07T07:45:00.495899+0000) 2024-03-07T07:45:18.168 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:17 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:17.893+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:40.496222+0000 front 2024-03-07T07:44:40.496309+0000 (oldest deadline 2024-03-07T07:45:08.696933+0000) 2024-03-07T07:45:18.168 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:17 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:17.893+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.798043+0000 front 2024-03-07T07:44:49.797936+0000 (oldest deadline 2024-03-07T07:45:15.097839+0000) 2024-03-07T07:45:18.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:17 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:17.945+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:32.457358+0000 front 2024-03-07T07:44:32.457424+0000 (oldest deadline 2024-03-07T07:44:57.757460+0000) 2024-03-07T07:45:18.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:17 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:17.945+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:44.159176+0000 front 2024-03-07T07:44:44.159293+0000 (oldest deadline 2024-03-07T07:45:07.059342+0000) 2024-03-07T07:45:18.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:17 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:17.945+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.960471+0000 front 2024-03-07T07:44:49.960386+0000 (oldest deadline 2024-03-07T07:45:13.360864+0000) 2024-03-07T07:45:18.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:18 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:18.026+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:18.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:18 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:18.026+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:18.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:18 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:18.026+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:42.970262+0000 front 2024-03-07T07:44:42.970339+0000 (oldest deadline 2024-03-07T07:45:08.870388+0000) 2024-03-07T07:45:18.347 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:18 smithi160 podman[88143]: 2024-03-07 07:45:18.055544511 +0000 UTC m=+0.431832988 container start 298d2eaa7f74ea46fad6766ac657b3ce517def1256237f2875d0d95276f64eca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, RELEASE=wip-lusov-qdb-8b896a9, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-07T07:45:18.347 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:18 smithi160 podman[88143]: 2024-03-07 07:45:18.143553554 +0000 UTC m=+0.519842038 container attach 298d2eaa7f74ea46fad6766ac657b3ce517def1256237f2875d0d95276f64eca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-8b896a9, maintainer=Guillaume Abrioux ) 2024-03-07T07:45:18.347 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:18 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:18.280+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.876690+0000 front 2024-03-07T07:44:27.876716+0000 (oldest deadline 2024-03-07T07:44:53.776918+0000) 2024-03-07T07:45:18.348 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:18 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:18.280+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:36.078333+0000 front 2024-03-07T07:44:36.078350+0000 (oldest deadline 2024-03-07T07:45:01.978091+0000) 2024-03-07T07:45:18.348 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:18 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:18.280+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:41.978705+0000 front 2024-03-07T07:44:41.978717+0000 (oldest deadline 2024-03-07T07:45:07.278468+0000) 2024-03-07T07:45:18.348 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:18 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:18.280+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:50.179200+0000 front 2024-03-07T07:44:50.179222+0000 (oldest deadline 2024-03-07T07:45:13.079347+0000) 2024-03-07T07:45:18.606 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:18 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:18.167+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:35.976112+0000 front 2024-03-07T07:44:35.976134+0000 (oldest deadline 2024-03-07T07:44:59.476430+0000) 2024-03-07T07:45:18.607 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:18 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:18.167+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:44.177363+0000 front 2024-03-07T07:44:44.177376+0000 (oldest deadline 2024-03-07T07:45:05.877409+0000) 2024-03-07T07:45:18.607 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:18 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:18.167+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:51.678875+0000 front 2024-03-07T07:44:51.679003+0000 (oldest deadline 2024-03-07T07:45:13.379016+0000) 2024-03-07T07:45:18.629 INFO:journalctl@ceph.osd.6.smithi160.stdout:Mar 07 07:45:18 smithi160 podman[88143]: 2024-03-07 07:45:18.348197633 +0000 UTC m=+0.724486106 container died 298d2eaa7f74ea46fad6766ac657b3ce517def1256237f2875d0d95276f64eca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-6-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-8b896a9, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b) 2024-03-07T07:45:19.066 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@osd.6.service' 2024-03-07T07:45:19.108 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-07T07:45:19.108 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-07T07:45:19.108 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-07T07:45:19.109 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-826f6582-dc54-11ee-95c6-87774f69a715@osd.7 2024-03-07T07:45:19.110 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:18 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:18.856+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:45:19.110 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:18 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:18.856+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:31.095363+0000 front 2024-03-07T07:44:31.095381+0000 (oldest deadline 2024-03-07T07:45:00.495899+0000) 2024-03-07T07:45:19.110 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:18 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:18.856+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:40.496222+0000 front 2024-03-07T07:44:40.496309+0000 (oldest deadline 2024-03-07T07:45:08.696933+0000) 2024-03-07T07:45:19.110 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:18 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:18.856+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.798043+0000 front 2024-03-07T07:44:49.797936+0000 (oldest deadline 2024-03-07T07:45:15.097839+0000) 2024-03-07T07:45:19.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:18 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:18.931+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:32.457358+0000 front 2024-03-07T07:44:32.457424+0000 (oldest deadline 2024-03-07T07:44:57.757460+0000) 2024-03-07T07:45:19.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:18 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:18.931+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:44.159176+0000 front 2024-03-07T07:44:44.159293+0000 (oldest deadline 2024-03-07T07:45:07.059342+0000) 2024-03-07T07:45:19.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:18 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:18.931+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.960471+0000 front 2024-03-07T07:44:49.960386+0000 (oldest deadline 2024-03-07T07:45:13.360864+0000) 2024-03-07T07:45:19.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:18 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:18.994+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:19.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:18 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:18.994+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:19.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:18 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:18.994+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:42.970262+0000 front 2024-03-07T07:44:42.970339+0000 (oldest deadline 2024-03-07T07:45:08.870388+0000) 2024-03-07T07:45:19.423 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:19 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:19.126+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:35.976112+0000 front 2024-03-07T07:44:35.976134+0000 (oldest deadline 2024-03-07T07:44:59.476430+0000) 2024-03-07T07:45:19.423 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:19 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:19.126+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:44.177363+0000 front 2024-03-07T07:44:44.177376+0000 (oldest deadline 2024-03-07T07:45:05.877409+0000) 2024-03-07T07:45:19.423 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:19 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:19.126+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:51.678875+0000 front 2024-03-07T07:44:51.679003+0000 (oldest deadline 2024-03-07T07:45:13.379016+0000) 2024-03-07T07:45:19.423 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:19 smithi177 systemd[1]: Stopping Ceph osd.7 for 826f6582-dc54-11ee-95c6-87774f69a715... 2024-03-07T07:45:19.580 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:19 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:19.321+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.876690+0000 front 2024-03-07T07:44:27.876716+0000 (oldest deadline 2024-03-07T07:44:53.776918+0000) 2024-03-07T07:45:19.580 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:19 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:19.321+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:36.078333+0000 front 2024-03-07T07:44:36.078350+0000 (oldest deadline 2024-03-07T07:45:01.978091+0000) 2024-03-07T07:45:19.580 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:19 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:19.321+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:41.978705+0000 front 2024-03-07T07:44:41.978717+0000 (oldest deadline 2024-03-07T07:45:07.278468+0000) 2024-03-07T07:45:19.580 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:19 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:19.321+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:50.179200+0000 front 2024-03-07T07:44:50.179222+0000 (oldest deadline 2024-03-07T07:45:13.079347+0000) 2024-03-07T07:45:19.857 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:19 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:19.453+0000 7f975004f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-07T07:45:19.858 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:19 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:19.453+0000 7f975004f700 -1 osd.7 90 *** Got signal Terminated *** 2024-03-07T07:45:19.858 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:19 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:19.453+0000 7f975004f700 -1 osd.7 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-07T07:45:20.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:19 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:19.890+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:32.457358+0000 front 2024-03-07T07:44:32.457424+0000 (oldest deadline 2024-03-07T07:44:57.757460+0000) 2024-03-07T07:45:20.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:19 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:19.890+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:44.159176+0000 front 2024-03-07T07:44:44.159293+0000 (oldest deadline 2024-03-07T07:45:07.059342+0000) 2024-03-07T07:45:20.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:19 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:19.890+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.960471+0000 front 2024-03-07T07:44:49.960386+0000 (oldest deadline 2024-03-07T07:45:13.360864+0000) 2024-03-07T07:45:20.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:20 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:20.030+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:20.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:20 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:20.030+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:20.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:20 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:20.030+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:42.970262+0000 front 2024-03-07T07:44:42.970339+0000 (oldest deadline 2024-03-07T07:45:08.870388+0000) 2024-03-07T07:45:20.352 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:20 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:20.117+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:35.976112+0000 front 2024-03-07T07:44:35.976134+0000 (oldest deadline 2024-03-07T07:44:59.476430+0000) 2024-03-07T07:45:20.352 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:20 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:20.117+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:44.177363+0000 front 2024-03-07T07:44:44.177376+0000 (oldest deadline 2024-03-07T07:45:05.877409+0000) 2024-03-07T07:45:20.352 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:20 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:20.117+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:51.678875+0000 front 2024-03-07T07:44:51.679003+0000 (oldest deadline 2024-03-07T07:45:13.379016+0000) 2024-03-07T07:45:20.352 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:19 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:19.875+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:45:20.352 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:19 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:19.875+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:31.095363+0000 front 2024-03-07T07:44:31.095381+0000 (oldest deadline 2024-03-07T07:45:00.495899+0000) 2024-03-07T07:45:20.352 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:19 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:19.875+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:40.496222+0000 front 2024-03-07T07:44:40.496309+0000 (oldest deadline 2024-03-07T07:45:08.696933+0000) 2024-03-07T07:45:20.352 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:19 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:19.875+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.798043+0000 front 2024-03-07T07:44:49.797936+0000 (oldest deadline 2024-03-07T07:45:15.097839+0000) 2024-03-07T07:45:20.677 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:20 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:20.302+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.876690+0000 front 2024-03-07T07:44:27.876716+0000 (oldest deadline 2024-03-07T07:44:53.776918+0000) 2024-03-07T07:45:20.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:20 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:20.302+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:36.078333+0000 front 2024-03-07T07:44:36.078350+0000 (oldest deadline 2024-03-07T07:45:01.978091+0000) 2024-03-07T07:45:20.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:20 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:20.302+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:41.978705+0000 front 2024-03-07T07:44:41.978717+0000 (oldest deadline 2024-03-07T07:45:07.278468+0000) 2024-03-07T07:45:20.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:20 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:20.302+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:50.179200+0000 front 2024-03-07T07:44:50.179222+0000 (oldest deadline 2024-03-07T07:45:13.079347+0000) 2024-03-07T07:45:21.207 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:21 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:21.059+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:21.207 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:21 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:21.059+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:21.208 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:21 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:21.059+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:42.970262+0000 front 2024-03-07T07:44:42.970339+0000 (oldest deadline 2024-03-07T07:45:08.870388+0000) 2024-03-07T07:45:21.208 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:20 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:20.938+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:32.457358+0000 front 2024-03-07T07:44:32.457424+0000 (oldest deadline 2024-03-07T07:44:57.757460+0000) 2024-03-07T07:45:21.208 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:20 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:20.938+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:44.159176+0000 front 2024-03-07T07:44:44.159293+0000 (oldest deadline 2024-03-07T07:45:07.059342+0000) 2024-03-07T07:45:21.208 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:20 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:20.938+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.960471+0000 front 2024-03-07T07:44:49.960386+0000 (oldest deadline 2024-03-07T07:45:13.360864+0000) 2024-03-07T07:45:21.356 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:21 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:21.127+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:35.976112+0000 front 2024-03-07T07:44:35.976134+0000 (oldest deadline 2024-03-07T07:44:59.476430+0000) 2024-03-07T07:45:21.357 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:21 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:21.127+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:44.177363+0000 front 2024-03-07T07:44:44.177376+0000 (oldest deadline 2024-03-07T07:45:05.877409+0000) 2024-03-07T07:45:21.357 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:21 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:21.127+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:51.678875+0000 front 2024-03-07T07:44:51.679003+0000 (oldest deadline 2024-03-07T07:45:13.379016+0000) 2024-03-07T07:45:21.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:20 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:20.901+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:45:21.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:20 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:20.901+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:31.095363+0000 front 2024-03-07T07:44:31.095381+0000 (oldest deadline 2024-03-07T07:45:00.495899+0000) 2024-03-07T07:45:21.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:20 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:20.901+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:40.496222+0000 front 2024-03-07T07:44:40.496309+0000 (oldest deadline 2024-03-07T07:45:08.696933+0000) 2024-03-07T07:45:21.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:20 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:20.901+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.798043+0000 front 2024-03-07T07:44:49.797936+0000 (oldest deadline 2024-03-07T07:45:15.097839+0000) 2024-03-07T07:45:21.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:20 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:20.901+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:58.598925+0000 front 2024-03-07T07:44:58.598939+0000 (oldest deadline 2024-03-07T07:45:20.899124+0000) 2024-03-07T07:45:21.677 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:21 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:21.339+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.876690+0000 front 2024-03-07T07:44:27.876716+0000 (oldest deadline 2024-03-07T07:44:53.776918+0000) 2024-03-07T07:45:21.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:21 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:21.339+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:36.078333+0000 front 2024-03-07T07:44:36.078350+0000 (oldest deadline 2024-03-07T07:45:01.978091+0000) 2024-03-07T07:45:21.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:21 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:21.339+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:41.978705+0000 front 2024-03-07T07:44:41.978717+0000 (oldest deadline 2024-03-07T07:45:07.278468+0000) 2024-03-07T07:45:21.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:21 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:21.339+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:50.179200+0000 front 2024-03-07T07:44:50.179222+0000 (oldest deadline 2024-03-07T07:45:13.079347+0000) 2024-03-07T07:45:21.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:21 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:21.339+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:56.580422+0000 front 2024-03-07T07:44:56.580566+0000 (oldest deadline 2024-03-07T07:45:21.280325+0000) 2024-03-07T07:45:22.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:21 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:21.962+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:32.457358+0000 front 2024-03-07T07:44:32.457424+0000 (oldest deadline 2024-03-07T07:44:57.757460+0000) 2024-03-07T07:45:22.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:21 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:21.963+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:44.159176+0000 front 2024-03-07T07:44:44.159293+0000 (oldest deadline 2024-03-07T07:45:07.059342+0000) 2024-03-07T07:45:22.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:21 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:21.963+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.960471+0000 front 2024-03-07T07:44:49.960386+0000 (oldest deadline 2024-03-07T07:45:13.360864+0000) 2024-03-07T07:45:22.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:22 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:22.107+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:22.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:22 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:22.107+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:22.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:22 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:22.107+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:42.970262+0000 front 2024-03-07T07:44:42.970339+0000 (oldest deadline 2024-03-07T07:45:08.870388+0000) 2024-03-07T07:45:22.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:22 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:22.107+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:57.071861+0000 front 2024-03-07T07:44:57.071845+0000 (oldest deadline 2024-03-07T07:45:21.171805+0000) 2024-03-07T07:45:22.357 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:22 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:22.139+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:35.976112+0000 front 2024-03-07T07:44:35.976134+0000 (oldest deadline 2024-03-07T07:44:59.476430+0000) 2024-03-07T07:45:22.357 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:22 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:22.139+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:44.177363+0000 front 2024-03-07T07:44:44.177376+0000 (oldest deadline 2024-03-07T07:45:05.877409+0000) 2024-03-07T07:45:22.357 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:22 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:22.139+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:51.678875+0000 front 2024-03-07T07:44:51.679003+0000 (oldest deadline 2024-03-07T07:45:13.379016+0000) 2024-03-07T07:45:22.357 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:22 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:22.139+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:55.580380+0000 front 2024-03-07T07:44:55.580408+0000 (oldest deadline 2024-03-07T07:45:21.480494+0000) 2024-03-07T07:45:22.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:21 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:21.949+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:45:22.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:21 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:21.949+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:31.095363+0000 front 2024-03-07T07:44:31.095381+0000 (oldest deadline 2024-03-07T07:45:00.495899+0000) 2024-03-07T07:45:22.358 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:21 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:21.949+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:40.496222+0000 front 2024-03-07T07:44:40.496309+0000 (oldest deadline 2024-03-07T07:45:08.696933+0000) 2024-03-07T07:45:22.358 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:21 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:21.949+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.798043+0000 front 2024-03-07T07:44:49.797936+0000 (oldest deadline 2024-03-07T07:45:15.097839+0000) 2024-03-07T07:45:22.358 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:21 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:21.949+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:58.598925+0000 front 2024-03-07T07:44:58.598939+0000 (oldest deadline 2024-03-07T07:45:20.899124+0000) 2024-03-07T07:45:22.677 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:22 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:22.291+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.876690+0000 front 2024-03-07T07:44:27.876716+0000 (oldest deadline 2024-03-07T07:44:53.776918+0000) 2024-03-07T07:45:22.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:22 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:22.291+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:36.078333+0000 front 2024-03-07T07:44:36.078350+0000 (oldest deadline 2024-03-07T07:45:01.978091+0000) 2024-03-07T07:45:22.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:22 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:22.291+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:41.978705+0000 front 2024-03-07T07:44:41.978717+0000 (oldest deadline 2024-03-07T07:45:07.278468+0000) 2024-03-07T07:45:22.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:22 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:22.291+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:50.179200+0000 front 2024-03-07T07:44:50.179222+0000 (oldest deadline 2024-03-07T07:45:13.079347+0000) 2024-03-07T07:45:22.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:22 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:22.291+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:56.580422+0000 front 2024-03-07T07:44:56.580566+0000 (oldest deadline 2024-03-07T07:45:21.280325+0000) 2024-03-07T07:45:23.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:22 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:22.927+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:32.457358+0000 front 2024-03-07T07:44:32.457424+0000 (oldest deadline 2024-03-07T07:44:57.757460+0000) 2024-03-07T07:45:23.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:22 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:22.927+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:44.159176+0000 front 2024-03-07T07:44:44.159293+0000 (oldest deadline 2024-03-07T07:45:07.059342+0000) 2024-03-07T07:45:23.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:22 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:22.927+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.960471+0000 front 2024-03-07T07:44:49.960386+0000 (oldest deadline 2024-03-07T07:45:13.360864+0000) 2024-03-07T07:45:23.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:23 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:23.062+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:23.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:23 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:23.062+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:23.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:23 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:23.062+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:42.970262+0000 front 2024-03-07T07:44:42.970339+0000 (oldest deadline 2024-03-07T07:45:08.870388+0000) 2024-03-07T07:45:23.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:23 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:23.062+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:57.071861+0000 front 2024-03-07T07:44:57.071845+0000 (oldest deadline 2024-03-07T07:45:21.171805+0000) 2024-03-07T07:45:23.357 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:23 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:23.160+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:35.976112+0000 front 2024-03-07T07:44:35.976134+0000 (oldest deadline 2024-03-07T07:44:59.476430+0000) 2024-03-07T07:45:23.357 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:23 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:23.160+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:44.177363+0000 front 2024-03-07T07:44:44.177376+0000 (oldest deadline 2024-03-07T07:45:05.877409+0000) 2024-03-07T07:45:23.357 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:23 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:23.160+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:51.678875+0000 front 2024-03-07T07:44:51.679003+0000 (oldest deadline 2024-03-07T07:45:13.379016+0000) 2024-03-07T07:45:23.357 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:23 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:23.160+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:55.580380+0000 front 2024-03-07T07:44:55.580408+0000 (oldest deadline 2024-03-07T07:45:21.480494+0000) 2024-03-07T07:45:23.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:22 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:22.993+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:45:23.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:22 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:22.993+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:31.095363+0000 front 2024-03-07T07:44:31.095381+0000 (oldest deadline 2024-03-07T07:45:00.495899+0000) 2024-03-07T07:45:23.358 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:22 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:22.993+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:40.496222+0000 front 2024-03-07T07:44:40.496309+0000 (oldest deadline 2024-03-07T07:45:08.696933+0000) 2024-03-07T07:45:23.358 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:22 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:22.993+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.798043+0000 front 2024-03-07T07:44:49.797936+0000 (oldest deadline 2024-03-07T07:45:15.097839+0000) 2024-03-07T07:45:23.358 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:22 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:22.993+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:58.598925+0000 front 2024-03-07T07:44:58.598939+0000 (oldest deadline 2024-03-07T07:45:20.899124+0000) 2024-03-07T07:45:23.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:23 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:23.319+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.876690+0000 front 2024-03-07T07:44:27.876716+0000 (oldest deadline 2024-03-07T07:44:53.776918+0000) 2024-03-07T07:45:23.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:23 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:23.319+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:36.078333+0000 front 2024-03-07T07:44:36.078350+0000 (oldest deadline 2024-03-07T07:45:01.978091+0000) 2024-03-07T07:45:23.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:23 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:23.319+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:41.978705+0000 front 2024-03-07T07:44:41.978717+0000 (oldest deadline 2024-03-07T07:45:07.278468+0000) 2024-03-07T07:45:23.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:23 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:23.319+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:50.179200+0000 front 2024-03-07T07:44:50.179222+0000 (oldest deadline 2024-03-07T07:45:13.079347+0000) 2024-03-07T07:45:23.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:23 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:23.319+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:56.580422+0000 front 2024-03-07T07:44:56.580566+0000 (oldest deadline 2024-03-07T07:45:21.280325+0000) 2024-03-07T07:45:24.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:24 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:24.091+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:24.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:24 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:24.091+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:24.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:24 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:24.091+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:42.970262+0000 front 2024-03-07T07:44:42.970339+0000 (oldest deadline 2024-03-07T07:45:08.870388+0000) 2024-03-07T07:45:24.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:24 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:24.091+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:57.071861+0000 front 2024-03-07T07:44:57.071845+0000 (oldest deadline 2024-03-07T07:45:21.171805+0000) 2024-03-07T07:45:24.307 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:23 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:23.928+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:32.457358+0000 front 2024-03-07T07:44:32.457424+0000 (oldest deadline 2024-03-07T07:44:57.757460+0000) 2024-03-07T07:45:24.307 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:23 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:23.928+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:44.159176+0000 front 2024-03-07T07:44:44.159293+0000 (oldest deadline 2024-03-07T07:45:07.059342+0000) 2024-03-07T07:45:24.307 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:23 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:23.928+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.960471+0000 front 2024-03-07T07:44:49.960386+0000 (oldest deadline 2024-03-07T07:45:13.360864+0000) 2024-03-07T07:45:24.307 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:23 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:23.928+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:58.562587+0000 front 2024-03-07T07:44:58.562651+0000 (oldest deadline 2024-03-07T07:45:23.262492+0000) 2024-03-07T07:45:24.357 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:24 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:24.192+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:35.976112+0000 front 2024-03-07T07:44:35.976134+0000 (oldest deadline 2024-03-07T07:44:59.476430+0000) 2024-03-07T07:45:24.357 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:24 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:24.192+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:44.177363+0000 front 2024-03-07T07:44:44.177376+0000 (oldest deadline 2024-03-07T07:45:05.877409+0000) 2024-03-07T07:45:24.357 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:24 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:24.192+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:51.678875+0000 front 2024-03-07T07:44:51.679003+0000 (oldest deadline 2024-03-07T07:45:13.379016+0000) 2024-03-07T07:45:24.357 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:24 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7[54853]: 2024-03-07T07:45:24.192+0000 7f974863b700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:55.580380+0000 front 2024-03-07T07:44:55.580408+0000 (oldest deadline 2024-03-07T07:45:21.480494+0000) 2024-03-07T07:45:24.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:24 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:24.004+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:45:24.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:24 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:24.004+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:31.095363+0000 front 2024-03-07T07:44:31.095381+0000 (oldest deadline 2024-03-07T07:45:00.495899+0000) 2024-03-07T07:45:24.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:24 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:24.004+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:40.496222+0000 front 2024-03-07T07:44:40.496309+0000 (oldest deadline 2024-03-07T07:45:08.696933+0000) 2024-03-07T07:45:24.358 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:24 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:24.004+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.798043+0000 front 2024-03-07T07:44:49.797936+0000 (oldest deadline 2024-03-07T07:45:15.097839+0000) 2024-03-07T07:45:24.358 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:24 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:24.005+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:58.598925+0000 front 2024-03-07T07:44:58.598939+0000 (oldest deadline 2024-03-07T07:45:20.899124+0000) 2024-03-07T07:45:24.677 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:24 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:24.298+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.876690+0000 front 2024-03-07T07:44:27.876716+0000 (oldest deadline 2024-03-07T07:44:53.776918+0000) 2024-03-07T07:45:24.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:24 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:24.298+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:36.078333+0000 front 2024-03-07T07:44:36.078350+0000 (oldest deadline 2024-03-07T07:45:01.978091+0000) 2024-03-07T07:45:24.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:24 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:24.298+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:41.978705+0000 front 2024-03-07T07:44:41.978717+0000 (oldest deadline 2024-03-07T07:45:07.278468+0000) 2024-03-07T07:45:24.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:24 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:24.298+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:50.179200+0000 front 2024-03-07T07:44:50.179222+0000 (oldest deadline 2024-03-07T07:45:13.079347+0000) 2024-03-07T07:45:24.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:24 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:24.298+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:56.580422+0000 front 2024-03-07T07:44:56.580566+0000 (oldest deadline 2024-03-07T07:45:21.280325+0000) 2024-03-07T07:45:25.062 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:24 smithi177 podman[70549]: 2024-03-07 07:45:24.804503209 +0000 UTC m=+5.481989767 container died ea563134bafcc6c8c102962852542a30e9cad1f9deaa594d52dd66cb7e6bd49c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-8b896a9, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-07T07:45:25.063 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:25 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:25.012+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:45:25.063 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:25 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:25.012+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:31.095363+0000 front 2024-03-07T07:44:31.095381+0000 (oldest deadline 2024-03-07T07:45:00.495899+0000) 2024-03-07T07:45:25.063 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:25 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:25.012+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:40.496222+0000 front 2024-03-07T07:44:40.496309+0000 (oldest deadline 2024-03-07T07:45:08.696933+0000) 2024-03-07T07:45:25.064 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:25 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:25.012+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.798043+0000 front 2024-03-07T07:44:49.797936+0000 (oldest deadline 2024-03-07T07:45:15.097839+0000) 2024-03-07T07:45:25.064 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:25 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:25.012+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:58.598925+0000 front 2024-03-07T07:44:58.598939+0000 (oldest deadline 2024-03-07T07:45:20.899124+0000) 2024-03-07T07:45:25.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:24 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:24.931+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:32.457358+0000 front 2024-03-07T07:44:32.457424+0000 (oldest deadline 2024-03-07T07:44:57.757460+0000) 2024-03-07T07:45:25.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:24 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:24.931+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:44.159176+0000 front 2024-03-07T07:44:44.159293+0000 (oldest deadline 2024-03-07T07:45:07.059342+0000) 2024-03-07T07:45:25.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:24 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:24.931+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.960471+0000 front 2024-03-07T07:44:49.960386+0000 (oldest deadline 2024-03-07T07:45:13.360864+0000) 2024-03-07T07:45:25.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:24 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:24.932+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:58.562587+0000 front 2024-03-07T07:44:58.562651+0000 (oldest deadline 2024-03-07T07:45:23.262492+0000) 2024-03-07T07:45:25.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:25 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:25.077+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:25.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:25 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:25.077+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:25.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:25 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:25.077+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:42.970262+0000 front 2024-03-07T07:44:42.970339+0000 (oldest deadline 2024-03-07T07:45:08.870388+0000) 2024-03-07T07:45:25.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:25 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:25.077+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:57.071861+0000 front 2024-03-07T07:44:57.071845+0000 (oldest deadline 2024-03-07T07:45:21.171805+0000) 2024-03-07T07:45:25.329 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:25 smithi177 podman[70549]: 2024-03-07 07:45:25.153457457 +0000 UTC m=+5.830943986 container cleanup ea563134bafcc6c8c102962852542a30e9cad1f9deaa594d52dd66cb7e6bd49c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-8b896a9, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-07T07:45:25.329 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:25 smithi177 bash[70549]: ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7 2024-03-07T07:45:25.606 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:25 smithi177 podman[70578]: 2024-03-07 07:45:25.329840253 +0000 UTC m=+0.518504971 container remove ea563134bafcc6c8c102962852542a30e9cad1f9deaa594d52dd66cb7e6bd49c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240222, RELEASE=wip-lusov-qdb-8b896a9, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b) 2024-03-07T07:45:25.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:25 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:25.283+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.876690+0000 front 2024-03-07T07:44:27.876716+0000 (oldest deadline 2024-03-07T07:44:53.776918+0000) 2024-03-07T07:45:25.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:25 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:25.283+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:36.078333+0000 front 2024-03-07T07:44:36.078350+0000 (oldest deadline 2024-03-07T07:45:01.978091+0000) 2024-03-07T07:45:25.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:25 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:25.283+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:41.978705+0000 front 2024-03-07T07:44:41.978717+0000 (oldest deadline 2024-03-07T07:45:07.278468+0000) 2024-03-07T07:45:25.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:25 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:25.283+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:50.179200+0000 front 2024-03-07T07:44:50.179222+0000 (oldest deadline 2024-03-07T07:45:13.079347+0000) 2024-03-07T07:45:25.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:25 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:25.283+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:56.580422+0000 front 2024-03-07T07:44:56.580566+0000 (oldest deadline 2024-03-07T07:45:21.280325+0000) 2024-03-07T07:45:26.032 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:25 smithi177 podman[70669]: 2024-03-07 07:45:25.664262329 +0000 UTC m=+0.017405647 image pull e362cd7cebfee51077fb90b7f01895811140c9a4272edd83ad9910b05e3a46f6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2 2024-03-07T07:45:26.033 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:25 smithi177 podman[70669]: 2024-03-07 07:45:25.788989641 +0000 UTC m=+0.142132990 container create a55de3d5c73fca95bfe2baea52ed3b0cf481704af1a59bccea9f3dea475598c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.schema-version=1.0, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-07T07:45:26.033 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:26 smithi177 podman[70669]: 2024-03-07 07:45:26.033277622 +0000 UTC m=+0.386420928 container init a55de3d5c73fca95bfe2baea52ed3b0cf481704af1a59bccea9f3dea475598c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.license=GPLv2, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0) 2024-03-07T07:45:26.033 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:25 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:25.992+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:45:26.033 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:25 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:25.992+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:31.095363+0000 front 2024-03-07T07:44:31.095381+0000 (oldest deadline 2024-03-07T07:45:00.495899+0000) 2024-03-07T07:45:26.033 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:25 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:25.992+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:40.496222+0000 front 2024-03-07T07:44:40.496309+0000 (oldest deadline 2024-03-07T07:45:08.696933+0000) 2024-03-07T07:45:26.033 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:25 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:25.992+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.798043+0000 front 2024-03-07T07:44:49.797936+0000 (oldest deadline 2024-03-07T07:45:15.097839+0000) 2024-03-07T07:45:26.034 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:25 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:25.992+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:58.598925+0000 front 2024-03-07T07:44:58.598939+0000 (oldest deadline 2024-03-07T07:45:20.899124+0000) 2024-03-07T07:45:26.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:26 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:26.107+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:26.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:26 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:26.107+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:26.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:26 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:26.107+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:42.970262+0000 front 2024-03-07T07:44:42.970339+0000 (oldest deadline 2024-03-07T07:45:08.870388+0000) 2024-03-07T07:45:26.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:26 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:26.107+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:57.071861+0000 front 2024-03-07T07:44:57.071845+0000 (oldest deadline 2024-03-07T07:45:21.171805+0000) 2024-03-07T07:45:26.307 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:25 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:25.882+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:32.457358+0000 front 2024-03-07T07:44:32.457424+0000 (oldest deadline 2024-03-07T07:44:57.757460+0000) 2024-03-07T07:45:26.307 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:25 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:25.882+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:44.159176+0000 front 2024-03-07T07:44:44.159293+0000 (oldest deadline 2024-03-07T07:45:07.059342+0000) 2024-03-07T07:45:26.307 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:25 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:25.882+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.960471+0000 front 2024-03-07T07:44:49.960386+0000 (oldest deadline 2024-03-07T07:45:13.360864+0000) 2024-03-07T07:45:26.307 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:25 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:25.882+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:58.562587+0000 front 2024-03-07T07:44:58.562651+0000 (oldest deadline 2024-03-07T07:45:23.262492+0000) 2024-03-07T07:45:26.318 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:26 smithi177 podman[70669]: 2024-03-07 07:45:26.036888695 +0000 UTC m=+0.390032007 container start a55de3d5c73fca95bfe2baea52ed3b0cf481704af1a59bccea9f3dea475598c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-8b896a9, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240222, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True) 2024-03-07T07:45:26.319 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:26 smithi177 podman[70669]: 2024-03-07 07:45:26.158601914 +0000 UTC m=+0.511745236 container attach a55de3d5c73fca95bfe2baea52ed3b0cf481704af1a59bccea9f3dea475598c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7-deactivate, ceph=True, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-07T07:45:26.607 INFO:journalctl@ceph.osd.7.smithi177.stdout:Mar 07 07:45:26 smithi177 podman[70669]: 2024-03-07 07:45:26.319458542 +0000 UTC m=+0.672601845 container died a55de3d5c73fca95bfe2baea52ed3b0cf481704af1a59bccea9f3dea475598c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-7-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-07T07:45:26.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:26 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:26.242+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.876690+0000 front 2024-03-07T07:44:27.876716+0000 (oldest deadline 2024-03-07T07:44:53.776918+0000) 2024-03-07T07:45:26.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:26 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:26.242+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:36.078333+0000 front 2024-03-07T07:44:36.078350+0000 (oldest deadline 2024-03-07T07:45:01.978091+0000) 2024-03-07T07:45:26.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:26 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:26.242+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:41.978705+0000 front 2024-03-07T07:44:41.978717+0000 (oldest deadline 2024-03-07T07:45:07.278468+0000) 2024-03-07T07:45:26.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:26 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:26.242+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:50.179200+0000 front 2024-03-07T07:44:50.179222+0000 (oldest deadline 2024-03-07T07:45:13.079347+0000) 2024-03-07T07:45:26.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:26 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:26.242+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:56.580422+0000 front 2024-03-07T07:44:56.580566+0000 (oldest deadline 2024-03-07T07:45:21.280325+0000) 2024-03-07T07:45:26.896 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@osd.7.service' 2024-03-07T07:45:26.938 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-07T07:45:26.938 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-07T07:45:26.938 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-03-07T07:45:26.938 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-826f6582-dc54-11ee-95c6-87774f69a715@osd.8 2024-03-07T07:45:27.252 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:26 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:26.912+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:32.457358+0000 front 2024-03-07T07:44:32.457424+0000 (oldest deadline 2024-03-07T07:44:57.757460+0000) 2024-03-07T07:45:27.252 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:26 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:26.912+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:44.159176+0000 front 2024-03-07T07:44:44.159293+0000 (oldest deadline 2024-03-07T07:45:07.059342+0000) 2024-03-07T07:45:27.253 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:26 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:26.912+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.960471+0000 front 2024-03-07T07:44:49.960386+0000 (oldest deadline 2024-03-07T07:45:13.360864+0000) 2024-03-07T07:45:27.253 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:26 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:26.912+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:58.562587+0000 front 2024-03-07T07:44:58.562651+0000 (oldest deadline 2024-03-07T07:45:23.262492+0000) 2024-03-07T07:45:27.253 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:26 smithi178 systemd[1]: Stopping Ceph osd.8 for 826f6582-dc54-11ee-95c6-87774f69a715... 2024-03-07T07:45:27.253 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:27 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:27.251+0000 7fb4cf317700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-07T07:45:27.253 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:27 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:27.251+0000 7fb4cf317700 -1 osd.8 90 *** Got signal Terminated *** 2024-03-07T07:45:27.253 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:27 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:27.251+0000 7fb4cf317700 -1 osd.8 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-07T07:45:27.253 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:27 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:27.063+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:27.253 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:27 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:27.063+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:27.253 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:27 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:27.063+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:42.970262+0000 front 2024-03-07T07:44:42.970339+0000 (oldest deadline 2024-03-07T07:45:08.870388+0000) 2024-03-07T07:45:27.253 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:27 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:27.063+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:57.071861+0000 front 2024-03-07T07:44:57.071845+0000 (oldest deadline 2024-03-07T07:45:21.171805+0000) 2024-03-07T07:45:27.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:26 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:26.971+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:45:27.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:26 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:26.971+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:31.095363+0000 front 2024-03-07T07:44:31.095381+0000 (oldest deadline 2024-03-07T07:45:00.495899+0000) 2024-03-07T07:45:27.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:26 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:26.971+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:40.496222+0000 front 2024-03-07T07:44:40.496309+0000 (oldest deadline 2024-03-07T07:45:08.696933+0000) 2024-03-07T07:45:27.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:26 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:26.971+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.798043+0000 front 2024-03-07T07:44:49.797936+0000 (oldest deadline 2024-03-07T07:45:15.097839+0000) 2024-03-07T07:45:27.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:26 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:26.971+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:58.598925+0000 front 2024-03-07T07:44:58.598939+0000 (oldest deadline 2024-03-07T07:45:20.899124+0000) 2024-03-07T07:45:27.677 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:27 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:27.291+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.876690+0000 front 2024-03-07T07:44:27.876716+0000 (oldest deadline 2024-03-07T07:44:53.776918+0000) 2024-03-07T07:45:27.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:27 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:27.291+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:36.078333+0000 front 2024-03-07T07:44:36.078350+0000 (oldest deadline 2024-03-07T07:45:01.978091+0000) 2024-03-07T07:45:27.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:27 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:27.291+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:41.978705+0000 front 2024-03-07T07:44:41.978717+0000 (oldest deadline 2024-03-07T07:45:07.278468+0000) 2024-03-07T07:45:27.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:27 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:27.291+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:50.179200+0000 front 2024-03-07T07:44:50.179222+0000 (oldest deadline 2024-03-07T07:45:13.079347+0000) 2024-03-07T07:45:27.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:27 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:27.291+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:56.580422+0000 front 2024-03-07T07:44:56.580566+0000 (oldest deadline 2024-03-07T07:45:21.280325+0000) 2024-03-07T07:45:28.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:27 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:27.915+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:32.457358+0000 front 2024-03-07T07:44:32.457424+0000 (oldest deadline 2024-03-07T07:44:57.757460+0000) 2024-03-07T07:45:28.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:27 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:27.915+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:44.159176+0000 front 2024-03-07T07:44:44.159293+0000 (oldest deadline 2024-03-07T07:45:07.059342+0000) 2024-03-07T07:45:28.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:27 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:27.915+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.960471+0000 front 2024-03-07T07:44:49.960386+0000 (oldest deadline 2024-03-07T07:45:13.360864+0000) 2024-03-07T07:45:28.307 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:27 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:27.915+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:58.562587+0000 front 2024-03-07T07:44:58.562651+0000 (oldest deadline 2024-03-07T07:45:23.262492+0000) 2024-03-07T07:45:28.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:28 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:28.038+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:28.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:28 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:28.038+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:28.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:28 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:28.038+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:42.970262+0000 front 2024-03-07T07:44:42.970339+0000 (oldest deadline 2024-03-07T07:45:08.870388+0000) 2024-03-07T07:45:28.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:28 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:28.038+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:57.071861+0000 front 2024-03-07T07:44:57.071845+0000 (oldest deadline 2024-03-07T07:45:21.171805+0000) 2024-03-07T07:45:28.356 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:28 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:28.009+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:45:28.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:28 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:28.009+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:31.095363+0000 front 2024-03-07T07:44:31.095381+0000 (oldest deadline 2024-03-07T07:45:00.495899+0000) 2024-03-07T07:45:28.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:28 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:28.009+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:40.496222+0000 front 2024-03-07T07:44:40.496309+0000 (oldest deadline 2024-03-07T07:45:08.696933+0000) 2024-03-07T07:45:28.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:28 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:28.009+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.798043+0000 front 2024-03-07T07:44:49.797936+0000 (oldest deadline 2024-03-07T07:45:15.097839+0000) 2024-03-07T07:45:28.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:28 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:28.009+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:58.598925+0000 front 2024-03-07T07:44:58.598939+0000 (oldest deadline 2024-03-07T07:45:20.899124+0000) 2024-03-07T07:45:28.677 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:28 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:28.309+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.876690+0000 front 2024-03-07T07:44:27.876716+0000 (oldest deadline 2024-03-07T07:44:53.776918+0000) 2024-03-07T07:45:28.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:28 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:28.309+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:36.078333+0000 front 2024-03-07T07:44:36.078350+0000 (oldest deadline 2024-03-07T07:45:01.978091+0000) 2024-03-07T07:45:28.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:28 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:28.309+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:41.978705+0000 front 2024-03-07T07:44:41.978717+0000 (oldest deadline 2024-03-07T07:45:07.278468+0000) 2024-03-07T07:45:28.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:28 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:28.309+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:50.179200+0000 front 2024-03-07T07:44:50.179222+0000 (oldest deadline 2024-03-07T07:45:13.079347+0000) 2024-03-07T07:45:28.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:28 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:28.309+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:56.580422+0000 front 2024-03-07T07:44:56.580566+0000 (oldest deadline 2024-03-07T07:45:21.280325+0000) 2024-03-07T07:45:29.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:29 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:29.072+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:29.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:29 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:29.072+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:29.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:29 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:29.072+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:42.970262+0000 front 2024-03-07T07:44:42.970339+0000 (oldest deadline 2024-03-07T07:45:08.870388+0000) 2024-03-07T07:45:29.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:29 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:29.072+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:57.071861+0000 front 2024-03-07T07:44:57.071845+0000 (oldest deadline 2024-03-07T07:45:21.171805+0000) 2024-03-07T07:45:29.307 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:28 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:28.943+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:32.457358+0000 front 2024-03-07T07:44:32.457424+0000 (oldest deadline 2024-03-07T07:44:57.757460+0000) 2024-03-07T07:45:29.307 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:28 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:28.943+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:44.159176+0000 front 2024-03-07T07:44:44.159293+0000 (oldest deadline 2024-03-07T07:45:07.059342+0000) 2024-03-07T07:45:29.307 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:28 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:28.943+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.960471+0000 front 2024-03-07T07:44:49.960386+0000 (oldest deadline 2024-03-07T07:45:13.360864+0000) 2024-03-07T07:45:29.307 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:28 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:28.943+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:58.562587+0000 front 2024-03-07T07:44:58.562651+0000 (oldest deadline 2024-03-07T07:45:23.262492+0000) 2024-03-07T07:45:29.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:29 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:29.019+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:45:29.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:29 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:29.019+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:31.095363+0000 front 2024-03-07T07:44:31.095381+0000 (oldest deadline 2024-03-07T07:45:00.495899+0000) 2024-03-07T07:45:29.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:29 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:29.019+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:40.496222+0000 front 2024-03-07T07:44:40.496309+0000 (oldest deadline 2024-03-07T07:45:08.696933+0000) 2024-03-07T07:45:29.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:29 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:29.019+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.798043+0000 front 2024-03-07T07:44:49.797936+0000 (oldest deadline 2024-03-07T07:45:15.097839+0000) 2024-03-07T07:45:29.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:29 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:29.019+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:58.598925+0000 front 2024-03-07T07:44:58.598939+0000 (oldest deadline 2024-03-07T07:45:20.899124+0000) 2024-03-07T07:45:29.428 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:29 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:29.278+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.876690+0000 front 2024-03-07T07:44:27.876716+0000 (oldest deadline 2024-03-07T07:44:53.776918+0000) 2024-03-07T07:45:29.428 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:29 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:29.278+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:36.078333+0000 front 2024-03-07T07:44:36.078350+0000 (oldest deadline 2024-03-07T07:45:01.978091+0000) 2024-03-07T07:45:29.428 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:29 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:29.278+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:41.978705+0000 front 2024-03-07T07:44:41.978717+0000 (oldest deadline 2024-03-07T07:45:07.278468+0000) 2024-03-07T07:45:29.428 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:29 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:29.278+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:50.179200+0000 front 2024-03-07T07:44:50.179222+0000 (oldest deadline 2024-03-07T07:45:13.079347+0000) 2024-03-07T07:45:29.428 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:29 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:29.278+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:56.580422+0000 front 2024-03-07T07:44:56.580566+0000 (oldest deadline 2024-03-07T07:45:21.280325+0000) 2024-03-07T07:45:29.428 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:29 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:29.278+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:04.681795+0000 front 2024-03-07T07:45:04.681916+0000 (oldest deadline 2024-03-07T07:45:28.781659+0000) 2024-03-07T07:45:30.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:29 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:29.957+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:32.457358+0000 front 2024-03-07T07:44:32.457424+0000 (oldest deadline 2024-03-07T07:44:57.757460+0000) 2024-03-07T07:45:30.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:29 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:29.957+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:44.159176+0000 front 2024-03-07T07:44:44.159293+0000 (oldest deadline 2024-03-07T07:45:07.059342+0000) 2024-03-07T07:45:30.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:29 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:29.957+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.960471+0000 front 2024-03-07T07:44:49.960386+0000 (oldest deadline 2024-03-07T07:45:13.360864+0000) 2024-03-07T07:45:30.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:29 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:29.957+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:58.562587+0000 front 2024-03-07T07:44:58.562651+0000 (oldest deadline 2024-03-07T07:45:23.262492+0000) 2024-03-07T07:45:30.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:29 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:29.957+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:06.763337+0000 front 2024-03-07T07:45:06.763402+0000 (oldest deadline 2024-03-07T07:45:29.663502+0000) 2024-03-07T07:45:30.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:30 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:30.034+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:30.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:30 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:30.034+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:30.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:30 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:30.034+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:42.970262+0000 front 2024-03-07T07:44:42.970339+0000 (oldest deadline 2024-03-07T07:45:08.870388+0000) 2024-03-07T07:45:30.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:30 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:30.034+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:57.071861+0000 front 2024-03-07T07:44:57.071845+0000 (oldest deadline 2024-03-07T07:45:21.171805+0000) 2024-03-07T07:45:30.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:30 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:30.034+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:08.173327+0000 front 2024-03-07T07:45:08.173344+0000 (oldest deadline 2024-03-07T07:45:29.273385+0000) 2024-03-07T07:45:30.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:30 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:30.000+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:45:30.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:30 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:30.000+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:31.095363+0000 front 2024-03-07T07:44:31.095381+0000 (oldest deadline 2024-03-07T07:45:00.495899+0000) 2024-03-07T07:45:30.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:30 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:30.000+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:40.496222+0000 front 2024-03-07T07:44:40.496309+0000 (oldest deadline 2024-03-07T07:45:08.696933+0000) 2024-03-07T07:45:30.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:30 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:30.000+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.798043+0000 front 2024-03-07T07:44:49.797936+0000 (oldest deadline 2024-03-07T07:45:15.097839+0000) 2024-03-07T07:45:30.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:30 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:30.000+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:58.598925+0000 front 2024-03-07T07:44:58.598939+0000 (oldest deadline 2024-03-07T07:45:20.899124+0000) 2024-03-07T07:45:30.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:30 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:30.000+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:04.900974+0000 front 2024-03-07T07:45:04.901052+0000 (oldest deadline 2024-03-07T07:45:29.600806+0000) 2024-03-07T07:45:30.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:30 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:30.320+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.876690+0000 front 2024-03-07T07:44:27.876716+0000 (oldest deadline 2024-03-07T07:44:53.776918+0000) 2024-03-07T07:45:30.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:30 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:30.320+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:36.078333+0000 front 2024-03-07T07:44:36.078350+0000 (oldest deadline 2024-03-07T07:45:01.978091+0000) 2024-03-07T07:45:30.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:30 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:30.320+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:41.978705+0000 front 2024-03-07T07:44:41.978717+0000 (oldest deadline 2024-03-07T07:45:07.278468+0000) 2024-03-07T07:45:30.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:30 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:30.320+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:50.179200+0000 front 2024-03-07T07:44:50.179222+0000 (oldest deadline 2024-03-07T07:45:13.079347+0000) 2024-03-07T07:45:30.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:30 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:30.320+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:56.580422+0000 front 2024-03-07T07:44:56.580566+0000 (oldest deadline 2024-03-07T07:45:21.280325+0000) 2024-03-07T07:45:30.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:30 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:30.320+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:04.681795+0000 front 2024-03-07T07:45:04.681916+0000 (oldest deadline 2024-03-07T07:45:28.781659+0000) 2024-03-07T07:45:31.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:30 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:30.947+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:32.457358+0000 front 2024-03-07T07:44:32.457424+0000 (oldest deadline 2024-03-07T07:44:57.757460+0000) 2024-03-07T07:45:31.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:30 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:30.947+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:44.159176+0000 front 2024-03-07T07:44:44.159293+0000 (oldest deadline 2024-03-07T07:45:07.059342+0000) 2024-03-07T07:45:31.307 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:30 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:30.947+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.960471+0000 front 2024-03-07T07:44:49.960386+0000 (oldest deadline 2024-03-07T07:45:13.360864+0000) 2024-03-07T07:45:31.307 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:30 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:30.947+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:58.562587+0000 front 2024-03-07T07:44:58.562651+0000 (oldest deadline 2024-03-07T07:45:23.262492+0000) 2024-03-07T07:45:31.307 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:30 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:30.947+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:06.763337+0000 front 2024-03-07T07:45:06.763402+0000 (oldest deadline 2024-03-07T07:45:29.663502+0000) 2024-03-07T07:45:31.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:31 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:31.071+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:31.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:31 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:31.071+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:31.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:31 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:31.071+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:42.970262+0000 front 2024-03-07T07:44:42.970339+0000 (oldest deadline 2024-03-07T07:45:08.870388+0000) 2024-03-07T07:45:31.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:31 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:31.071+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:57.071861+0000 front 2024-03-07T07:44:57.071845+0000 (oldest deadline 2024-03-07T07:45:21.171805+0000) 2024-03-07T07:45:31.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:31 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:31.071+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:08.173327+0000 front 2024-03-07T07:45:08.173344+0000 (oldest deadline 2024-03-07T07:45:29.273385+0000) 2024-03-07T07:45:31.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:30 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:30.970+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:45:31.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:30 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:30.970+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:31.095363+0000 front 2024-03-07T07:44:31.095381+0000 (oldest deadline 2024-03-07T07:45:00.495899+0000) 2024-03-07T07:45:31.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:30 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:30.970+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:40.496222+0000 front 2024-03-07T07:44:40.496309+0000 (oldest deadline 2024-03-07T07:45:08.696933+0000) 2024-03-07T07:45:31.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:30 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:30.970+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.798043+0000 front 2024-03-07T07:44:49.797936+0000 (oldest deadline 2024-03-07T07:45:15.097839+0000) 2024-03-07T07:45:31.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:30 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:30.970+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:58.598925+0000 front 2024-03-07T07:44:58.598939+0000 (oldest deadline 2024-03-07T07:45:20.899124+0000) 2024-03-07T07:45:31.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:30 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:30.970+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:04.900974+0000 front 2024-03-07T07:45:04.901052+0000 (oldest deadline 2024-03-07T07:45:29.600806+0000) 2024-03-07T07:45:31.677 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:31 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:31.287+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.876690+0000 front 2024-03-07T07:44:27.876716+0000 (oldest deadline 2024-03-07T07:44:53.776918+0000) 2024-03-07T07:45:31.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:31 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:31.287+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:36.078333+0000 front 2024-03-07T07:44:36.078350+0000 (oldest deadline 2024-03-07T07:45:01.978091+0000) 2024-03-07T07:45:31.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:31 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:31.287+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:41.978705+0000 front 2024-03-07T07:44:41.978717+0000 (oldest deadline 2024-03-07T07:45:07.278468+0000) 2024-03-07T07:45:31.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:31 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:31.287+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:50.179200+0000 front 2024-03-07T07:44:50.179222+0000 (oldest deadline 2024-03-07T07:45:13.079347+0000) 2024-03-07T07:45:31.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:31 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:31.287+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:56.580422+0000 front 2024-03-07T07:44:56.580566+0000 (oldest deadline 2024-03-07T07:45:21.280325+0000) 2024-03-07T07:45:31.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:31 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:31.287+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:04.681795+0000 front 2024-03-07T07:45:04.681916+0000 (oldest deadline 2024-03-07T07:45:28.781659+0000) 2024-03-07T07:45:32.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:31 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:31.910+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:32.457358+0000 front 2024-03-07T07:44:32.457424+0000 (oldest deadline 2024-03-07T07:44:57.757460+0000) 2024-03-07T07:45:32.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:31 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:31.910+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:44.159176+0000 front 2024-03-07T07:44:44.159293+0000 (oldest deadline 2024-03-07T07:45:07.059342+0000) 2024-03-07T07:45:32.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:31 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:31.910+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.960471+0000 front 2024-03-07T07:44:49.960386+0000 (oldest deadline 2024-03-07T07:45:13.360864+0000) 2024-03-07T07:45:32.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:31 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:31.910+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:58.562587+0000 front 2024-03-07T07:44:58.562651+0000 (oldest deadline 2024-03-07T07:45:23.262492+0000) 2024-03-07T07:45:32.307 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:31 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8[55778]: 2024-03-07T07:45:31.910+0000 7fb4cb90b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:06.763337+0000 front 2024-03-07T07:45:06.763402+0000 (oldest deadline 2024-03-07T07:45:29.663502+0000) 2024-03-07T07:45:32.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:32 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:32.055+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:32.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:32 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:32.055+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:32.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:32 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:32.055+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:42.970262+0000 front 2024-03-07T07:44:42.970339+0000 (oldest deadline 2024-03-07T07:45:08.870388+0000) 2024-03-07T07:45:32.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:32 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:32.055+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:57.071861+0000 front 2024-03-07T07:44:57.071845+0000 (oldest deadline 2024-03-07T07:45:21.171805+0000) 2024-03-07T07:45:32.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:32 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:32.055+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:08.173327+0000 front 2024-03-07T07:45:08.173344+0000 (oldest deadline 2024-03-07T07:45:29.273385+0000) 2024-03-07T07:45:32.356 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:31 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:31.987+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:45:32.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:31 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:31.987+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:31.095363+0000 front 2024-03-07T07:44:31.095381+0000 (oldest deadline 2024-03-07T07:45:00.495899+0000) 2024-03-07T07:45:32.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:31 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:31.987+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:40.496222+0000 front 2024-03-07T07:44:40.496309+0000 (oldest deadline 2024-03-07T07:45:08.696933+0000) 2024-03-07T07:45:32.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:31 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:31.987+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.798043+0000 front 2024-03-07T07:44:49.797936+0000 (oldest deadline 2024-03-07T07:45:15.097839+0000) 2024-03-07T07:45:32.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:31 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:31.987+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:58.598925+0000 front 2024-03-07T07:44:58.598939+0000 (oldest deadline 2024-03-07T07:45:20.899124+0000) 2024-03-07T07:45:32.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:31 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:31.987+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:04.900974+0000 front 2024-03-07T07:45:04.901052+0000 (oldest deadline 2024-03-07T07:45:29.600806+0000) 2024-03-07T07:45:32.677 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:32 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:32.309+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.876690+0000 front 2024-03-07T07:44:27.876716+0000 (oldest deadline 2024-03-07T07:44:53.776918+0000) 2024-03-07T07:45:32.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:32 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:32.309+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:36.078333+0000 front 2024-03-07T07:44:36.078350+0000 (oldest deadline 2024-03-07T07:45:01.978091+0000) 2024-03-07T07:45:32.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:32 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:32.309+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:41.978705+0000 front 2024-03-07T07:44:41.978717+0000 (oldest deadline 2024-03-07T07:45:07.278468+0000) 2024-03-07T07:45:32.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:32 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:32.309+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:50.179200+0000 front 2024-03-07T07:44:50.179222+0000 (oldest deadline 2024-03-07T07:45:13.079347+0000) 2024-03-07T07:45:32.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:32 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:32.309+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:56.580422+0000 front 2024-03-07T07:44:56.580566+0000 (oldest deadline 2024-03-07T07:45:21.280325+0000) 2024-03-07T07:45:32.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:32 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:32.309+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:04.681795+0000 front 2024-03-07T07:45:04.681916+0000 (oldest deadline 2024-03-07T07:45:28.781659+0000) 2024-03-07T07:45:32.806 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:32 smithi178 podman[72317]: 2024-03-07 07:45:32.715545255 +0000 UTC m=+5.588726753 container died bb785fe71ea5207ef7459c070452f54f0e73249c1523e7fb1b9ff4d4b6196e32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-07T07:45:33.303 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:33 smithi178 podman[72317]: 2024-03-07 07:45:33.102216784 +0000 UTC m=+5.975398273 container cleanup bb785fe71ea5207ef7459c070452f54f0e73249c1523e7fb1b9ff4d4b6196e32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-8b896a9, GIT_BRANCH=HEAD, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, CEPH_POINT_RELEASE=) 2024-03-07T07:45:33.303 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:33 smithi178 bash[72317]: ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8 2024-03-07T07:45:33.303 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:33 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:33.012+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:33.303 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:33 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:33.012+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:33.303 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:33 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:33.012+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:42.970262+0000 front 2024-03-07T07:44:42.970339+0000 (oldest deadline 2024-03-07T07:45:08.870388+0000) 2024-03-07T07:45:33.303 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:33 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:33.012+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:57.071861+0000 front 2024-03-07T07:44:57.071845+0000 (oldest deadline 2024-03-07T07:45:21.171805+0000) 2024-03-07T07:45:33.303 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:33 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:33.012+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:08.173327+0000 front 2024-03-07T07:45:08.173344+0000 (oldest deadline 2024-03-07T07:45:29.273385+0000) 2024-03-07T07:45:33.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:33 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:33.023+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:45:33.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:33 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:33.023+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:31.095363+0000 front 2024-03-07T07:44:31.095381+0000 (oldest deadline 2024-03-07T07:45:00.495899+0000) 2024-03-07T07:45:33.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:33 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:33.023+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:40.496222+0000 front 2024-03-07T07:44:40.496309+0000 (oldest deadline 2024-03-07T07:45:08.696933+0000) 2024-03-07T07:45:33.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:33 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:33.023+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.798043+0000 front 2024-03-07T07:44:49.797936+0000 (oldest deadline 2024-03-07T07:45:15.097839+0000) 2024-03-07T07:45:33.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:33 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:33.023+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:58.598925+0000 front 2024-03-07T07:44:58.598939+0000 (oldest deadline 2024-03-07T07:45:20.899124+0000) 2024-03-07T07:45:33.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:33 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:33.023+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:04.900974+0000 front 2024-03-07T07:45:04.901052+0000 (oldest deadline 2024-03-07T07:45:29.600806+0000) 2024-03-07T07:45:33.556 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:33 smithi178 podman[72335]: 2024-03-07 07:45:33.303422843 +0000 UTC m=+0.583573058 container remove bb785fe71ea5207ef7459c070452f54f0e73249c1523e7fb1b9ff4d4b6196e32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True) 2024-03-07T07:45:33.677 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:33 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:33.291+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.876690+0000 front 2024-03-07T07:44:27.876716+0000 (oldest deadline 2024-03-07T07:44:53.776918+0000) 2024-03-07T07:45:33.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:33 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:33.291+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:36.078333+0000 front 2024-03-07T07:44:36.078350+0000 (oldest deadline 2024-03-07T07:45:01.978091+0000) 2024-03-07T07:45:33.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:33 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:33.291+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:41.978705+0000 front 2024-03-07T07:44:41.978717+0000 (oldest deadline 2024-03-07T07:45:07.278468+0000) 2024-03-07T07:45:33.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:33 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:33.291+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:50.179200+0000 front 2024-03-07T07:44:50.179222+0000 (oldest deadline 2024-03-07T07:45:13.079347+0000) 2024-03-07T07:45:33.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:33 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:33.291+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:56.580422+0000 front 2024-03-07T07:44:56.580566+0000 (oldest deadline 2024-03-07T07:45:21.280325+0000) 2024-03-07T07:45:33.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:33 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:33.291+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:04.681795+0000 front 2024-03-07T07:45:04.681916+0000 (oldest deadline 2024-03-07T07:45:28.781659+0000) 2024-03-07T07:45:33.914 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:33 smithi178 podman[72424]: 2024-03-07 07:45:33.629455036 +0000 UTC m=+0.112396894 container create f380df671bf1c0dad2c2dd0b289d8b6cf2225c5f8c8cd84d5363ff153e3ec6b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-8b896a9, GIT_CLEAN=True, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-03-07T07:45:33.915 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:33 smithi178 podman[72424]: 2024-03-07 07:45:33.534695103 +0000 UTC m=+0.017636952 image pull e362cd7cebfee51077fb90b7f01895811140c9a4272edd83ad9910b05e3a46f6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2 2024-03-07T07:45:33.915 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:33 smithi178 podman[72424]: 2024-03-07 07:45:33.823460253 +0000 UTC m=+0.306402108 container init f380df671bf1c0dad2c2dd0b289d8b6cf2225c5f8c8cd84d5363ff153e3ec6b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, ceph=True) 2024-03-07T07:45:33.915 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:33 smithi178 podman[72424]: 2024-03-07 07:45:33.826551566 +0000 UTC m=+0.309493408 container start f380df671bf1c0dad2c2dd0b289d8b6cf2225c5f8c8cd84d5363ff153e3ec6b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240222, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_BRANCH=HEAD) 2024-03-07T07:45:33.915 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:33 smithi178 podman[72424]: 2024-03-07 07:45:33.915149187 +0000 UTC m=+0.398091039 container attach f380df671bf1c0dad2c2dd0b289d8b6cf2225c5f8c8cd84d5363ff153e3ec6b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222) 2024-03-07T07:45:34.306 INFO:journalctl@ceph.osd.8.smithi178.stdout:Mar 07 07:45:34 smithi178 podman[72424]: 2024-03-07 07:45:34.114752568 +0000 UTC m=+0.597694410 container died f380df671bf1c0dad2c2dd0b289d8b6cf2225c5f8c8cd84d5363ff153e3ec6b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240222) 2024-03-07T07:45:34.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:33 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:33.979+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:34.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:33 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:33.979+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:34.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:33 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:33.979+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:42.970262+0000 front 2024-03-07T07:44:42.970339+0000 (oldest deadline 2024-03-07T07:45:08.870388+0000) 2024-03-07T07:45:34.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:33 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:33.979+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:57.071861+0000 front 2024-03-07T07:44:57.071845+0000 (oldest deadline 2024-03-07T07:45:21.171805+0000) 2024-03-07T07:45:34.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:33 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:33.979+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:08.173327+0000 front 2024-03-07T07:45:08.173344+0000 (oldest deadline 2024-03-07T07:45:29.273385+0000) 2024-03-07T07:45:34.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:33 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:33.977+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:45:34.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:33 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:33.977+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:31.095363+0000 front 2024-03-07T07:44:31.095381+0000 (oldest deadline 2024-03-07T07:45:00.495899+0000) 2024-03-07T07:45:34.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:33 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:33.977+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:40.496222+0000 front 2024-03-07T07:44:40.496309+0000 (oldest deadline 2024-03-07T07:45:08.696933+0000) 2024-03-07T07:45:34.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:33 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:33.977+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.798043+0000 front 2024-03-07T07:44:49.797936+0000 (oldest deadline 2024-03-07T07:45:15.097839+0000) 2024-03-07T07:45:34.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:33 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:33.977+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:58.598925+0000 front 2024-03-07T07:44:58.598939+0000 (oldest deadline 2024-03-07T07:45:20.899124+0000) 2024-03-07T07:45:34.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:33 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:33.977+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:04.900974+0000 front 2024-03-07T07:45:04.901052+0000 (oldest deadline 2024-03-07T07:45:29.600806+0000) 2024-03-07T07:45:34.678 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@osd.8.service' 2024-03-07T07:45:34.679 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:34 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:34.305+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.876690+0000 front 2024-03-07T07:44:27.876716+0000 (oldest deadline 2024-03-07T07:44:53.776918+0000) 2024-03-07T07:45:34.679 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:34 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:34.305+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:36.078333+0000 front 2024-03-07T07:44:36.078350+0000 (oldest deadline 2024-03-07T07:45:01.978091+0000) 2024-03-07T07:45:34.679 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:34 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:34.305+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:41.978705+0000 front 2024-03-07T07:44:41.978717+0000 (oldest deadline 2024-03-07T07:45:07.278468+0000) 2024-03-07T07:45:34.679 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:34 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:34.305+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:50.179200+0000 front 2024-03-07T07:44:50.179222+0000 (oldest deadline 2024-03-07T07:45:13.079347+0000) 2024-03-07T07:45:34.679 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:34 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:34.305+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:56.580422+0000 front 2024-03-07T07:44:56.580566+0000 (oldest deadline 2024-03-07T07:45:21.280325+0000) 2024-03-07T07:45:34.679 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:34 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:34.305+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:04.681795+0000 front 2024-03-07T07:45:04.681916+0000 (oldest deadline 2024-03-07T07:45:28.781659+0000) 2024-03-07T07:45:34.719 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-07T07:45:34.719 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-03-07T07:45:34.719 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-03-07T07:45:34.720 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-826f6582-dc54-11ee-95c6-87774f69a715@osd.9 2024-03-07T07:45:35.033 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:34 smithi160 systemd[1]: Stopping Ceph osd.9 for 826f6582-dc54-11ee-95c6-87774f69a715... 2024-03-07T07:45:35.033 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:35 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:35.032+0000 7ff6013f3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-07T07:45:35.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:34 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:34.956+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:35.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:34 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:34.956+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:35.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:34 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:34.956+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:42.970262+0000 front 2024-03-07T07:44:42.970339+0000 (oldest deadline 2024-03-07T07:45:08.870388+0000) 2024-03-07T07:45:35.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:34 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:34.956+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:57.071861+0000 front 2024-03-07T07:44:57.071845+0000 (oldest deadline 2024-03-07T07:45:21.171805+0000) 2024-03-07T07:45:35.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:34 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:34.956+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:08.173327+0000 front 2024-03-07T07:45:08.173344+0000 (oldest deadline 2024-03-07T07:45:29.273385+0000) 2024-03-07T07:45:35.322 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:35 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:35.032+0000 7ff6013f3700 -1 osd.9 90 *** Got signal Terminated *** 2024-03-07T07:45:35.322 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:35 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:35.032+0000 7ff6013f3700 -1 osd.9 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-07T07:45:35.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:35 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:35.014+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:45:35.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:35 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:35.014+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:31.095363+0000 front 2024-03-07T07:44:31.095381+0000 (oldest deadline 2024-03-07T07:45:00.495899+0000) 2024-03-07T07:45:35.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:35 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:35.014+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:40.496222+0000 front 2024-03-07T07:44:40.496309+0000 (oldest deadline 2024-03-07T07:45:08.696933+0000) 2024-03-07T07:45:35.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:35 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:35.014+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.798043+0000 front 2024-03-07T07:44:49.797936+0000 (oldest deadline 2024-03-07T07:45:15.097839+0000) 2024-03-07T07:45:35.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:35 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:35.014+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:58.598925+0000 front 2024-03-07T07:44:58.598939+0000 (oldest deadline 2024-03-07T07:45:20.899124+0000) 2024-03-07T07:45:35.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:35 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:35.014+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:04.900974+0000 front 2024-03-07T07:45:04.901052+0000 (oldest deadline 2024-03-07T07:45:29.600806+0000) 2024-03-07T07:45:35.677 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:35 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:35.322+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.876690+0000 front 2024-03-07T07:44:27.876716+0000 (oldest deadline 2024-03-07T07:44:53.776918+0000) 2024-03-07T07:45:35.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:35 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:35.322+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:36.078333+0000 front 2024-03-07T07:44:36.078350+0000 (oldest deadline 2024-03-07T07:45:01.978091+0000) 2024-03-07T07:45:35.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:35 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:35.322+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:41.978705+0000 front 2024-03-07T07:44:41.978717+0000 (oldest deadline 2024-03-07T07:45:07.278468+0000) 2024-03-07T07:45:35.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:35 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:35.322+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:50.179200+0000 front 2024-03-07T07:44:50.179222+0000 (oldest deadline 2024-03-07T07:45:13.079347+0000) 2024-03-07T07:45:35.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:35 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:35.322+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:56.580422+0000 front 2024-03-07T07:44:56.580566+0000 (oldest deadline 2024-03-07T07:45:21.280325+0000) 2024-03-07T07:45:35.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:35 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:35.322+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:04.681795+0000 front 2024-03-07T07:45:04.681916+0000 (oldest deadline 2024-03-07T07:45:28.781659+0000) 2024-03-07T07:45:36.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:35 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:35.972+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:36.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:35 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:35.972+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:36.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:35 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:35.972+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:42.970262+0000 front 2024-03-07T07:44:42.970339+0000 (oldest deadline 2024-03-07T07:45:08.870388+0000) 2024-03-07T07:45:36.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:35 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:35.972+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:57.071861+0000 front 2024-03-07T07:44:57.071845+0000 (oldest deadline 2024-03-07T07:45:21.171805+0000) 2024-03-07T07:45:36.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:35 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:35.972+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:08.173327+0000 front 2024-03-07T07:45:08.173344+0000 (oldest deadline 2024-03-07T07:45:29.273385+0000) 2024-03-07T07:45:36.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:36 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:36.060+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:45:36.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:36 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:36.060+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:31.095363+0000 front 2024-03-07T07:44:31.095381+0000 (oldest deadline 2024-03-07T07:45:00.495899+0000) 2024-03-07T07:45:36.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:36 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:36.060+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:40.496222+0000 front 2024-03-07T07:44:40.496309+0000 (oldest deadline 2024-03-07T07:45:08.696933+0000) 2024-03-07T07:45:36.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:36 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:36.060+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.798043+0000 front 2024-03-07T07:44:49.797936+0000 (oldest deadline 2024-03-07T07:45:15.097839+0000) 2024-03-07T07:45:36.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:36 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:36.060+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:58.598925+0000 front 2024-03-07T07:44:58.598939+0000 (oldest deadline 2024-03-07T07:45:20.899124+0000) 2024-03-07T07:45:36.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:36 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:36.060+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:04.900974+0000 front 2024-03-07T07:45:04.901052+0000 (oldest deadline 2024-03-07T07:45:29.600806+0000) 2024-03-07T07:45:36.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:36 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:36.347+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.876690+0000 front 2024-03-07T07:44:27.876716+0000 (oldest deadline 2024-03-07T07:44:53.776918+0000) 2024-03-07T07:45:36.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:36 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:36.347+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:36.078333+0000 front 2024-03-07T07:44:36.078350+0000 (oldest deadline 2024-03-07T07:45:01.978091+0000) 2024-03-07T07:45:36.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:36 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:36.347+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:41.978705+0000 front 2024-03-07T07:44:41.978717+0000 (oldest deadline 2024-03-07T07:45:07.278468+0000) 2024-03-07T07:45:36.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:36 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:36.347+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:50.179200+0000 front 2024-03-07T07:44:50.179222+0000 (oldest deadline 2024-03-07T07:45:13.079347+0000) 2024-03-07T07:45:36.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:36 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:36.347+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:56.580422+0000 front 2024-03-07T07:44:56.580566+0000 (oldest deadline 2024-03-07T07:45:21.280325+0000) 2024-03-07T07:45:36.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:36 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:36.347+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:04.681795+0000 front 2024-03-07T07:45:04.681916+0000 (oldest deadline 2024-03-07T07:45:28.781659+0000) 2024-03-07T07:45:37.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:36 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:36.980+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:37.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:36 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:36.980+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:37.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:36 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:36.980+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:42.970262+0000 front 2024-03-07T07:44:42.970339+0000 (oldest deadline 2024-03-07T07:45:08.870388+0000) 2024-03-07T07:45:37.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:36 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:36.980+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:57.071861+0000 front 2024-03-07T07:44:57.071845+0000 (oldest deadline 2024-03-07T07:45:21.171805+0000) 2024-03-07T07:45:37.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:36 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:36.980+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:08.173327+0000 front 2024-03-07T07:45:08.173344+0000 (oldest deadline 2024-03-07T07:45:29.273385+0000) 2024-03-07T07:45:37.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:37 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:37.023+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:45:37.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:37 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:37.023+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:31.095363+0000 front 2024-03-07T07:44:31.095381+0000 (oldest deadline 2024-03-07T07:45:00.495899+0000) 2024-03-07T07:45:37.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:37 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:37.023+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:40.496222+0000 front 2024-03-07T07:44:40.496309+0000 (oldest deadline 2024-03-07T07:45:08.696933+0000) 2024-03-07T07:45:37.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:37 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:37.023+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.798043+0000 front 2024-03-07T07:44:49.797936+0000 (oldest deadline 2024-03-07T07:45:15.097839+0000) 2024-03-07T07:45:37.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:37 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:37.023+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:58.598925+0000 front 2024-03-07T07:44:58.598939+0000 (oldest deadline 2024-03-07T07:45:20.899124+0000) 2024-03-07T07:45:37.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:37 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:37.023+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:04.900974+0000 front 2024-03-07T07:45:04.901052+0000 (oldest deadline 2024-03-07T07:45:29.600806+0000) 2024-03-07T07:45:37.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:37 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:37.392+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.876690+0000 front 2024-03-07T07:44:27.876716+0000 (oldest deadline 2024-03-07T07:44:53.776918+0000) 2024-03-07T07:45:37.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:37 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:37.392+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:36.078333+0000 front 2024-03-07T07:44:36.078350+0000 (oldest deadline 2024-03-07T07:45:01.978091+0000) 2024-03-07T07:45:37.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:37 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:37.392+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:41.978705+0000 front 2024-03-07T07:44:41.978717+0000 (oldest deadline 2024-03-07T07:45:07.278468+0000) 2024-03-07T07:45:37.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:37 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:37.392+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:50.179200+0000 front 2024-03-07T07:44:50.179222+0000 (oldest deadline 2024-03-07T07:45:13.079347+0000) 2024-03-07T07:45:37.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:37 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:37.392+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:56.580422+0000 front 2024-03-07T07:44:56.580566+0000 (oldest deadline 2024-03-07T07:45:21.280325+0000) 2024-03-07T07:45:37.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:37 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:37.392+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:04.681795+0000 front 2024-03-07T07:45:04.681916+0000 (oldest deadline 2024-03-07T07:45:28.781659+0000) 2024-03-07T07:45:38.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:37 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:37.931+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:38.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:37 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:37.931+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:38.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:37 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:37.931+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:42.970262+0000 front 2024-03-07T07:44:42.970339+0000 (oldest deadline 2024-03-07T07:45:08.870388+0000) 2024-03-07T07:45:38.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:37 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:37.931+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:57.071861+0000 front 2024-03-07T07:44:57.071845+0000 (oldest deadline 2024-03-07T07:45:21.171805+0000) 2024-03-07T07:45:38.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:37 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:37.931+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:08.173327+0000 front 2024-03-07T07:45:08.173344+0000 (oldest deadline 2024-03-07T07:45:29.273385+0000) 2024-03-07T07:45:38.356 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:38 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:38.028+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:45:38.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:38 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:38.028+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:31.095363+0000 front 2024-03-07T07:44:31.095381+0000 (oldest deadline 2024-03-07T07:45:00.495899+0000) 2024-03-07T07:45:38.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:38 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:38.028+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:40.496222+0000 front 2024-03-07T07:44:40.496309+0000 (oldest deadline 2024-03-07T07:45:08.696933+0000) 2024-03-07T07:45:38.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:38 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:38.028+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.798043+0000 front 2024-03-07T07:44:49.797936+0000 (oldest deadline 2024-03-07T07:45:15.097839+0000) 2024-03-07T07:45:38.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:38 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:38.028+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:58.598925+0000 front 2024-03-07T07:44:58.598939+0000 (oldest deadline 2024-03-07T07:45:20.899124+0000) 2024-03-07T07:45:38.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:38 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:38.028+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:04.900974+0000 front 2024-03-07T07:45:04.901052+0000 (oldest deadline 2024-03-07T07:45:29.600806+0000) 2024-03-07T07:45:38.677 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:38 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:38.370+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.876690+0000 front 2024-03-07T07:44:27.876716+0000 (oldest deadline 2024-03-07T07:44:53.776918+0000) 2024-03-07T07:45:38.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:38 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:38.370+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:36.078333+0000 front 2024-03-07T07:44:36.078350+0000 (oldest deadline 2024-03-07T07:45:01.978091+0000) 2024-03-07T07:45:38.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:38 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:38.370+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:41.978705+0000 front 2024-03-07T07:44:41.978717+0000 (oldest deadline 2024-03-07T07:45:07.278468+0000) 2024-03-07T07:45:38.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:38 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:38.370+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:50.179200+0000 front 2024-03-07T07:44:50.179222+0000 (oldest deadline 2024-03-07T07:45:13.079347+0000) 2024-03-07T07:45:38.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:38 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:38.370+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:56.580422+0000 front 2024-03-07T07:44:56.580566+0000 (oldest deadline 2024-03-07T07:45:21.280325+0000) 2024-03-07T07:45:38.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:38 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:38.370+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:04.681795+0000 front 2024-03-07T07:45:04.681916+0000 (oldest deadline 2024-03-07T07:45:28.781659+0000) 2024-03-07T07:45:39.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:38 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:38.889+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:39.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:38 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:38.889+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:39.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:38 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:38.889+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:42.970262+0000 front 2024-03-07T07:44:42.970339+0000 (oldest deadline 2024-03-07T07:45:08.870388+0000) 2024-03-07T07:45:39.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:38 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:38.889+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:57.071861+0000 front 2024-03-07T07:44:57.071845+0000 (oldest deadline 2024-03-07T07:45:21.171805+0000) 2024-03-07T07:45:39.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:38 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:38.889+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:08.173327+0000 front 2024-03-07T07:45:08.173344+0000 (oldest deadline 2024-03-07T07:45:29.273385+0000) 2024-03-07T07:45:39.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:39 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:39.047+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:45:39.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:39 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:39.047+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:31.095363+0000 front 2024-03-07T07:44:31.095381+0000 (oldest deadline 2024-03-07T07:45:00.495899+0000) 2024-03-07T07:45:39.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:39 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:39.047+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:40.496222+0000 front 2024-03-07T07:44:40.496309+0000 (oldest deadline 2024-03-07T07:45:08.696933+0000) 2024-03-07T07:45:39.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:39 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:39.047+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.798043+0000 front 2024-03-07T07:44:49.797936+0000 (oldest deadline 2024-03-07T07:45:15.097839+0000) 2024-03-07T07:45:39.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:39 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:39.047+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:58.598925+0000 front 2024-03-07T07:44:58.598939+0000 (oldest deadline 2024-03-07T07:45:20.899124+0000) 2024-03-07T07:45:39.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:39 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:39.047+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:04.900974+0000 front 2024-03-07T07:45:04.901052+0000 (oldest deadline 2024-03-07T07:45:29.600806+0000) 2024-03-07T07:45:39.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:39 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:39.323+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.876690+0000 front 2024-03-07T07:44:27.876716+0000 (oldest deadline 2024-03-07T07:44:53.776918+0000) 2024-03-07T07:45:39.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:39 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:39.323+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:36.078333+0000 front 2024-03-07T07:44:36.078350+0000 (oldest deadline 2024-03-07T07:45:01.978091+0000) 2024-03-07T07:45:39.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:39 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:39.323+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:41.978705+0000 front 2024-03-07T07:44:41.978717+0000 (oldest deadline 2024-03-07T07:45:07.278468+0000) 2024-03-07T07:45:39.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:39 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:39.323+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:50.179200+0000 front 2024-03-07T07:44:50.179222+0000 (oldest deadline 2024-03-07T07:45:13.079347+0000) 2024-03-07T07:45:39.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:39 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:39.323+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:56.580422+0000 front 2024-03-07T07:44:56.580566+0000 (oldest deadline 2024-03-07T07:45:21.280325+0000) 2024-03-07T07:45:39.678 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:39 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9[67330]: 2024-03-07T07:45:39.323+0000 7ff5fd9e7700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:04.681795+0000 front 2024-03-07T07:45:04.681916+0000 (oldest deadline 2024-03-07T07:45:28.781659+0000) 2024-03-07T07:45:40.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:39 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:39.912+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:40.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:39 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:39.912+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:40.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:39 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:39.912+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:42.970262+0000 front 2024-03-07T07:44:42.970339+0000 (oldest deadline 2024-03-07T07:45:08.870388+0000) 2024-03-07T07:45:40.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:39 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:39.912+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:57.071861+0000 front 2024-03-07T07:44:57.071845+0000 (oldest deadline 2024-03-07T07:45:21.171805+0000) 2024-03-07T07:45:40.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:39 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:39.912+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:08.173327+0000 front 2024-03-07T07:45:08.173344+0000 (oldest deadline 2024-03-07T07:45:29.273385+0000) 2024-03-07T07:45:40.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:39 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:39.912+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6822 osd.6 since back 2024-03-07T07:45:10.974782+0000 front 2024-03-07T07:45:10.974879+0000 (oldest deadline 2024-03-07T07:45:39.175172+0000) 2024-03-07T07:45:40.356 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:40 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:40.076+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:45:40.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:40 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:40.076+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:31.095363+0000 front 2024-03-07T07:44:31.095381+0000 (oldest deadline 2024-03-07T07:45:00.495899+0000) 2024-03-07T07:45:40.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:40 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:40.076+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:40.496222+0000 front 2024-03-07T07:44:40.496309+0000 (oldest deadline 2024-03-07T07:45:08.696933+0000) 2024-03-07T07:45:40.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:40 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:40.076+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.798043+0000 front 2024-03-07T07:44:49.797936+0000 (oldest deadline 2024-03-07T07:45:15.097839+0000) 2024-03-07T07:45:40.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:40 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:40.076+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:58.598925+0000 front 2024-03-07T07:44:58.598939+0000 (oldest deadline 2024-03-07T07:45:20.899124+0000) 2024-03-07T07:45:40.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:40 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:40.076+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:04.900974+0000 front 2024-03-07T07:45:04.901052+0000 (oldest deadline 2024-03-07T07:45:29.600806+0000) 2024-03-07T07:45:40.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:40 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:40.076+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6822 osd.6 since back 2024-03-07T07:45:15.902864+0000 front 2024-03-07T07:45:15.902900+0000 (oldest deadline 2024-03-07T07:45:39.402813+0000) 2024-03-07T07:45:40.766 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:40 smithi160 podman[88261]: 2024-03-07 07:45:40.476865619 +0000 UTC m=+5.555573718 container died f3e963a7dfe38076d3202a6482b44e336ea3e063e073edfa827b16edff89d062 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9, org.label-schema.build-date=20240222, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-8b896a9, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-07T07:45:41.177 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:40 smithi160 podman[88261]: 2024-03-07 07:45:40.932561602 +0000 UTC m=+6.011269697 container cleanup f3e963a7dfe38076d3202a6482b44e336ea3e063e073edfa827b16edff89d062 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-8b896a9, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-07T07:45:41.178 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:40 smithi160 bash[88261]: ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9 2024-03-07T07:45:41.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:40 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:40.960+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:41.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:40 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:40.960+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:41.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:40 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:40.960+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:42.970262+0000 front 2024-03-07T07:44:42.970339+0000 (oldest deadline 2024-03-07T07:45:08.870388+0000) 2024-03-07T07:45:41.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:40 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:40.960+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:57.071861+0000 front 2024-03-07T07:44:57.071845+0000 (oldest deadline 2024-03-07T07:45:21.171805+0000) 2024-03-07T07:45:41.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:40 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:40.960+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:08.173327+0000 front 2024-03-07T07:45:08.173344+0000 (oldest deadline 2024-03-07T07:45:29.273385+0000) 2024-03-07T07:45:41.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:40 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:40.960+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6822 osd.6 since back 2024-03-07T07:45:10.974782+0000 front 2024-03-07T07:45:10.974879+0000 (oldest deadline 2024-03-07T07:45:39.175172+0000) 2024-03-07T07:45:41.356 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:41 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:41.071+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:45:41.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:41 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:41.071+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:31.095363+0000 front 2024-03-07T07:44:31.095381+0000 (oldest deadline 2024-03-07T07:45:00.495899+0000) 2024-03-07T07:45:41.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:41 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:41.071+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:40.496222+0000 front 2024-03-07T07:44:40.496309+0000 (oldest deadline 2024-03-07T07:45:08.696933+0000) 2024-03-07T07:45:41.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:41 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:41.071+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.798043+0000 front 2024-03-07T07:44:49.797936+0000 (oldest deadline 2024-03-07T07:45:15.097839+0000) 2024-03-07T07:45:41.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:41 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:41.071+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:58.598925+0000 front 2024-03-07T07:44:58.598939+0000 (oldest deadline 2024-03-07T07:45:20.899124+0000) 2024-03-07T07:45:41.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:41 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:41.071+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:04.900974+0000 front 2024-03-07T07:45:04.901052+0000 (oldest deadline 2024-03-07T07:45:29.600806+0000) 2024-03-07T07:45:41.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:41 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:41.071+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6822 osd.6 since back 2024-03-07T07:45:15.902864+0000 front 2024-03-07T07:45:15.902900+0000 (oldest deadline 2024-03-07T07:45:39.402813+0000) 2024-03-07T07:45:41.575 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:41 smithi160 podman[88278]: 2024-03-07 07:45:41.267504609 +0000 UTC m=+0.782704375 container remove f3e963a7dfe38076d3202a6482b44e336ea3e063e073edfa827b16edff89d062 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, maintainer=Guillaume Abrioux ) 2024-03-07T07:45:41.870 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:41 smithi160 podman[88372]: 2024-03-07 07:45:41.576500348 +0000 UTC m=+0.114813932 container create bba1d2c44c944a48e3e8586cffa9308a20ec1dfd27190b6c10f491d65ed787c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-07T07:45:41.870 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:41 smithi160 podman[88372]: 2024-03-07 07:45:41.480217658 +0000 UTC m=+0.018531246 image pull e362cd7cebfee51077fb90b7f01895811140c9a4272edd83ad9910b05e3a46f6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2 2024-03-07T07:45:41.870 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:41 smithi160 podman[88372]: 2024-03-07 07:45:41.813054388 +0000 UTC m=+0.351367965 container init bba1d2c44c944a48e3e8586cffa9308a20ec1dfd27190b6c10f491d65ed787c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-07T07:45:41.870 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:41 smithi160 podman[88372]: 2024-03-07 07:45:41.816164104 +0000 UTC m=+0.354477681 container start bba1d2c44c944a48e3e8586cffa9308a20ec1dfd27190b6c10f491d65ed787c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.vendor=CentOS) 2024-03-07T07:45:41.870 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:41 smithi160 podman[88372]: 2024-03-07 07:45:41.870587772 +0000 UTC m=+0.408901383 container attach bba1d2c44c944a48e3e8586cffa9308a20ec1dfd27190b6c10f491d65ed787c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9-deactivate, GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-qdb-8b896a9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-03-07T07:45:42.138 INFO:journalctl@ceph.osd.9.smithi160.stdout:Mar 07 07:45:42 smithi160 podman[88372]: 2024-03-07 07:45:42.1089932 +0000 UTC m=+0.647306777 container died bba1d2c44c944a48e3e8586cffa9308a20ec1dfd27190b6c10f491d65ed787c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-lusov-qdb-8b896a9, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-07T07:45:42.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:41 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:41.925+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:42.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:41 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:41.925+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:42.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:41 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:41.925+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:42.970262+0000 front 2024-03-07T07:44:42.970339+0000 (oldest deadline 2024-03-07T07:45:08.870388+0000) 2024-03-07T07:45:42.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:41 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:41.925+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:57.071861+0000 front 2024-03-07T07:44:57.071845+0000 (oldest deadline 2024-03-07T07:45:21.171805+0000) 2024-03-07T07:45:42.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:41 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:41.925+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:08.173327+0000 front 2024-03-07T07:45:08.173344+0000 (oldest deadline 2024-03-07T07:45:29.273385+0000) 2024-03-07T07:45:42.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:41 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:41.925+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6822 osd.6 since back 2024-03-07T07:45:10.974782+0000 front 2024-03-07T07:45:10.974879+0000 (oldest deadline 2024-03-07T07:45:39.175172+0000) 2024-03-07T07:45:42.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:42 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:42.090+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:45:42.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:42 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:42.090+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:31.095363+0000 front 2024-03-07T07:44:31.095381+0000 (oldest deadline 2024-03-07T07:45:00.495899+0000) 2024-03-07T07:45:42.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:42 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:42.090+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:40.496222+0000 front 2024-03-07T07:44:40.496309+0000 (oldest deadline 2024-03-07T07:45:08.696933+0000) 2024-03-07T07:45:42.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:42 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:42.090+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.798043+0000 front 2024-03-07T07:44:49.797936+0000 (oldest deadline 2024-03-07T07:45:15.097839+0000) 2024-03-07T07:45:42.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:42 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:42.090+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:58.598925+0000 front 2024-03-07T07:44:58.598939+0000 (oldest deadline 2024-03-07T07:45:20.899124+0000) 2024-03-07T07:45:42.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:42 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:42.090+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:04.900974+0000 front 2024-03-07T07:45:04.901052+0000 (oldest deadline 2024-03-07T07:45:29.600806+0000) 2024-03-07T07:45:42.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:42 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:42.090+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6822 osd.6 since back 2024-03-07T07:45:15.902864+0000 front 2024-03-07T07:45:15.902900+0000 (oldest deadline 2024-03-07T07:45:39.402813+0000) 2024-03-07T07:45:42.674 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@osd.9.service' 2024-03-07T07:45:42.716 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-07T07:45:42.716 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-03-07T07:45:42.716 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-03-07T07:45:42.716 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-826f6582-dc54-11ee-95c6-87774f69a715@osd.10 2024-03-07T07:45:43.021 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:42 smithi177 systemd[1]: Stopping Ceph osd.10 for 826f6582-dc54-11ee-95c6-87774f69a715... 2024-03-07T07:45:43.021 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:43 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:43.021+0000 7f186101f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-07T07:45:43.022 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:43 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:43.021+0000 7f186101f700 -1 osd.10 90 *** Got signal Terminated *** 2024-03-07T07:45:43.022 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:43 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:43.021+0000 7f186101f700 -1 osd.10 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-07T07:45:43.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:42 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:42.883+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:43.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:42 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:42.883+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:43.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:42 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:42.883+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:42.970262+0000 front 2024-03-07T07:44:42.970339+0000 (oldest deadline 2024-03-07T07:45:08.870388+0000) 2024-03-07T07:45:43.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:42 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:42.883+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:57.071861+0000 front 2024-03-07T07:44:57.071845+0000 (oldest deadline 2024-03-07T07:45:21.171805+0000) 2024-03-07T07:45:43.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:42 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:42.883+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:08.173327+0000 front 2024-03-07T07:45:08.173344+0000 (oldest deadline 2024-03-07T07:45:29.273385+0000) 2024-03-07T07:45:43.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:42 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:42.883+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6822 osd.6 since back 2024-03-07T07:45:10.974782+0000 front 2024-03-07T07:45:10.974879+0000 (oldest deadline 2024-03-07T07:45:39.175172+0000) 2024-03-07T07:45:43.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:43 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:43.139+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:45:43.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:43 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:43.139+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:31.095363+0000 front 2024-03-07T07:44:31.095381+0000 (oldest deadline 2024-03-07T07:45:00.495899+0000) 2024-03-07T07:45:43.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:43 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:43.139+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:40.496222+0000 front 2024-03-07T07:44:40.496309+0000 (oldest deadline 2024-03-07T07:45:08.696933+0000) 2024-03-07T07:45:43.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:43 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:43.139+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.798043+0000 front 2024-03-07T07:44:49.797936+0000 (oldest deadline 2024-03-07T07:45:15.097839+0000) 2024-03-07T07:45:43.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:43 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:43.139+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:58.598925+0000 front 2024-03-07T07:44:58.598939+0000 (oldest deadline 2024-03-07T07:45:20.899124+0000) 2024-03-07T07:45:43.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:43 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:43.139+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:04.900974+0000 front 2024-03-07T07:45:04.901052+0000 (oldest deadline 2024-03-07T07:45:29.600806+0000) 2024-03-07T07:45:43.357 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:43 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:43.139+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6822 osd.6 since back 2024-03-07T07:45:15.902864+0000 front 2024-03-07T07:45:15.902900+0000 (oldest deadline 2024-03-07T07:45:39.402813+0000) 2024-03-07T07:45:44.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:43 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:43.870+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:44.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:43 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:43.870+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:44.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:43 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:43.870+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:42.970262+0000 front 2024-03-07T07:44:42.970339+0000 (oldest deadline 2024-03-07T07:45:08.870388+0000) 2024-03-07T07:45:44.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:43 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:43.870+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:57.071861+0000 front 2024-03-07T07:44:57.071845+0000 (oldest deadline 2024-03-07T07:45:21.171805+0000) 2024-03-07T07:45:44.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:43 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:43.870+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:08.173327+0000 front 2024-03-07T07:45:08.173344+0000 (oldest deadline 2024-03-07T07:45:29.273385+0000) 2024-03-07T07:45:44.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:43 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:43.870+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6822 osd.6 since back 2024-03-07T07:45:10.974782+0000 front 2024-03-07T07:45:10.974879+0000 (oldest deadline 2024-03-07T07:45:39.175172+0000) 2024-03-07T07:45:44.606 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:44 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:44.183+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:45:44.607 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:44 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:44.183+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:31.095363+0000 front 2024-03-07T07:44:31.095381+0000 (oldest deadline 2024-03-07T07:45:00.495899+0000) 2024-03-07T07:45:44.607 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:44 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:44.183+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:40.496222+0000 front 2024-03-07T07:44:40.496309+0000 (oldest deadline 2024-03-07T07:45:08.696933+0000) 2024-03-07T07:45:44.607 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:44 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:44.183+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.798043+0000 front 2024-03-07T07:44:49.797936+0000 (oldest deadline 2024-03-07T07:45:15.097839+0000) 2024-03-07T07:45:44.607 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:44 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:44.183+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:58.598925+0000 front 2024-03-07T07:44:58.598939+0000 (oldest deadline 2024-03-07T07:45:20.899124+0000) 2024-03-07T07:45:44.607 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:44 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:44.183+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:04.900974+0000 front 2024-03-07T07:45:04.901052+0000 (oldest deadline 2024-03-07T07:45:29.600806+0000) 2024-03-07T07:45:44.607 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:44 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:44.183+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6822 osd.6 since back 2024-03-07T07:45:15.902864+0000 front 2024-03-07T07:45:15.902900+0000 (oldest deadline 2024-03-07T07:45:39.402813+0000) 2024-03-07T07:45:45.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:44 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:44.844+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:45.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:44 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:44.844+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:45.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:44 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:44.844+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:42.970262+0000 front 2024-03-07T07:44:42.970339+0000 (oldest deadline 2024-03-07T07:45:08.870388+0000) 2024-03-07T07:45:45.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:44 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:44.844+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:57.071861+0000 front 2024-03-07T07:44:57.071845+0000 (oldest deadline 2024-03-07T07:45:21.171805+0000) 2024-03-07T07:45:45.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:44 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:44.844+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:08.173327+0000 front 2024-03-07T07:45:08.173344+0000 (oldest deadline 2024-03-07T07:45:29.273385+0000) 2024-03-07T07:45:45.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:44 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:44.844+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6822 osd.6 since back 2024-03-07T07:45:10.974782+0000 front 2024-03-07T07:45:10.974879+0000 (oldest deadline 2024-03-07T07:45:39.175172+0000) 2024-03-07T07:45:45.607 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:45 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:45.173+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:45:45.607 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:45 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:45.173+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:31.095363+0000 front 2024-03-07T07:44:31.095381+0000 (oldest deadline 2024-03-07T07:45:00.495899+0000) 2024-03-07T07:45:45.607 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:45 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:45.173+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:40.496222+0000 front 2024-03-07T07:44:40.496309+0000 (oldest deadline 2024-03-07T07:45:08.696933+0000) 2024-03-07T07:45:45.607 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:45 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:45.173+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.798043+0000 front 2024-03-07T07:44:49.797936+0000 (oldest deadline 2024-03-07T07:45:15.097839+0000) 2024-03-07T07:45:45.607 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:45 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:45.173+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:58.598925+0000 front 2024-03-07T07:44:58.598939+0000 (oldest deadline 2024-03-07T07:45:20.899124+0000) 2024-03-07T07:45:45.607 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:45 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:45.173+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:04.900974+0000 front 2024-03-07T07:45:04.901052+0000 (oldest deadline 2024-03-07T07:45:29.600806+0000) 2024-03-07T07:45:45.607 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:45 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:45.173+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6822 osd.6 since back 2024-03-07T07:45:15.902864+0000 front 2024-03-07T07:45:15.902900+0000 (oldest deadline 2024-03-07T07:45:39.402813+0000) 2024-03-07T07:45:46.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:45 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:45.883+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:46.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:45 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:45.883+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:46.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:45 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:45.883+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:42.970262+0000 front 2024-03-07T07:44:42.970339+0000 (oldest deadline 2024-03-07T07:45:08.870388+0000) 2024-03-07T07:45:46.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:45 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:45.883+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:57.071861+0000 front 2024-03-07T07:44:57.071845+0000 (oldest deadline 2024-03-07T07:45:21.171805+0000) 2024-03-07T07:45:46.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:45 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:45.883+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:08.173327+0000 front 2024-03-07T07:45:08.173344+0000 (oldest deadline 2024-03-07T07:45:29.273385+0000) 2024-03-07T07:45:46.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:45 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:45.883+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6822 osd.6 since back 2024-03-07T07:45:10.974782+0000 front 2024-03-07T07:45:10.974879+0000 (oldest deadline 2024-03-07T07:45:39.175172+0000) 2024-03-07T07:45:46.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:45 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:45.883+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6820 osd.7 since back 2024-03-07T07:45:22.676192+0000 front 2024-03-07T07:45:22.676259+0000 (oldest deadline 2024-03-07T07:45:44.976158+0000) 2024-03-07T07:45:46.608 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:46 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:46.181+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:45:46.608 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:46 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:46.181+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:31.095363+0000 front 2024-03-07T07:44:31.095381+0000 (oldest deadline 2024-03-07T07:45:00.495899+0000) 2024-03-07T07:45:46.608 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:46 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:46.181+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:40.496222+0000 front 2024-03-07T07:44:40.496309+0000 (oldest deadline 2024-03-07T07:45:08.696933+0000) 2024-03-07T07:45:46.608 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:46 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:46.181+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.798043+0000 front 2024-03-07T07:44:49.797936+0000 (oldest deadline 2024-03-07T07:45:15.097839+0000) 2024-03-07T07:45:46.608 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:46 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:46.181+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:58.598925+0000 front 2024-03-07T07:44:58.598939+0000 (oldest deadline 2024-03-07T07:45:20.899124+0000) 2024-03-07T07:45:46.608 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:46 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:46.181+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:04.900974+0000 front 2024-03-07T07:45:04.901052+0000 (oldest deadline 2024-03-07T07:45:29.600806+0000) 2024-03-07T07:45:46.608 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:46 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:46.181+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6822 osd.6 since back 2024-03-07T07:45:15.902864+0000 front 2024-03-07T07:45:15.902900+0000 (oldest deadline 2024-03-07T07:45:39.402813+0000) 2024-03-07T07:45:47.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:46 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:46.857+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:47.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:46 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:46.857+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:47.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:46 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:46.857+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:42.970262+0000 front 2024-03-07T07:44:42.970339+0000 (oldest deadline 2024-03-07T07:45:08.870388+0000) 2024-03-07T07:45:47.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:46 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:46.857+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:57.071861+0000 front 2024-03-07T07:44:57.071845+0000 (oldest deadline 2024-03-07T07:45:21.171805+0000) 2024-03-07T07:45:47.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:46 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:46.857+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:08.173327+0000 front 2024-03-07T07:45:08.173344+0000 (oldest deadline 2024-03-07T07:45:29.273385+0000) 2024-03-07T07:45:47.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:46 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:46.857+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6822 osd.6 since back 2024-03-07T07:45:10.974782+0000 front 2024-03-07T07:45:10.974879+0000 (oldest deadline 2024-03-07T07:45:39.175172+0000) 2024-03-07T07:45:47.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:46 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:46.857+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6820 osd.7 since back 2024-03-07T07:45:22.676192+0000 front 2024-03-07T07:45:22.676259+0000 (oldest deadline 2024-03-07T07:45:44.976158+0000) 2024-03-07T07:45:47.607 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:47 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:47.168+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:27.594975+0000 front 2024-03-07T07:44:27.594937+0000 (oldest deadline 2024-03-07T07:44:51.095086+0000) 2024-03-07T07:45:47.607 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:47 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:47.168+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:31.095363+0000 front 2024-03-07T07:44:31.095381+0000 (oldest deadline 2024-03-07T07:45:00.495899+0000) 2024-03-07T07:45:47.607 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:47 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:47.168+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:40.496222+0000 front 2024-03-07T07:44:40.496309+0000 (oldest deadline 2024-03-07T07:45:08.696933+0000) 2024-03-07T07:45:47.607 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:47 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:47.168+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6814 osd.3 since back 2024-03-07T07:44:49.798043+0000 front 2024-03-07T07:44:49.797936+0000 (oldest deadline 2024-03-07T07:45:15.097839+0000) 2024-03-07T07:45:47.607 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:47 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:47.168+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:58.598925+0000 front 2024-03-07T07:44:58.598939+0000 (oldest deadline 2024-03-07T07:45:20.899124+0000) 2024-03-07T07:45:47.607 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:47 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:47.168+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:04.900974+0000 front 2024-03-07T07:45:04.901052+0000 (oldest deadline 2024-03-07T07:45:29.600806+0000) 2024-03-07T07:45:47.607 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:47 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10[61482]: 2024-03-07T07:45:47.168+0000 7f185d613700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.160:6822 osd.6 since back 2024-03-07T07:45:15.902864+0000 front 2024-03-07T07:45:15.902900+0000 (oldest deadline 2024-03-07T07:45:39.402813+0000) 2024-03-07T07:45:48.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:47 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:47.869+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:48.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:47 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:47.869+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:48.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:47 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:47.869+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:42.970262+0000 front 2024-03-07T07:44:42.970339+0000 (oldest deadline 2024-03-07T07:45:08.870388+0000) 2024-03-07T07:45:48.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:47 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:47.869+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:57.071861+0000 front 2024-03-07T07:44:57.071845+0000 (oldest deadline 2024-03-07T07:45:21.171805+0000) 2024-03-07T07:45:48.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:47 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:47.869+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:08.173327+0000 front 2024-03-07T07:45:08.173344+0000 (oldest deadline 2024-03-07T07:45:29.273385+0000) 2024-03-07T07:45:48.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:47 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:47.869+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6822 osd.6 since back 2024-03-07T07:45:10.974782+0000 front 2024-03-07T07:45:10.974879+0000 (oldest deadline 2024-03-07T07:45:39.175172+0000) 2024-03-07T07:45:48.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:47 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:47.869+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6820 osd.7 since back 2024-03-07T07:45:22.676192+0000 front 2024-03-07T07:45:22.676259+0000 (oldest deadline 2024-03-07T07:45:44.976158+0000) 2024-03-07T07:45:48.688 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:48 smithi177 podman[70776]: 2024-03-07 07:45:48.411169206 +0000 UTC m=+5.495809709 container died 85c773932687ff948d815fdbb658c374f36e3acb0cd49aa41d9d7b6d4eb39b64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-8b896a9, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, CEPH_POINT_RELEASE=) 2024-03-07T07:45:49.038 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:48 smithi177 podman[70776]: 2024-03-07 07:45:48.771256759 +0000 UTC m=+5.855897269 container cleanup 85c773932687ff948d815fdbb658c374f36e3acb0cd49aa41d9d7b6d4eb39b64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-07T07:45:49.039 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:48 smithi177 bash[70776]: ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10 2024-03-07T07:45:49.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:48 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:48.856+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:49.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:48 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:48.856+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:49.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:48 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:48.856+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:42.970262+0000 front 2024-03-07T07:44:42.970339+0000 (oldest deadline 2024-03-07T07:45:08.870388+0000) 2024-03-07T07:45:49.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:48 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:48.856+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:57.071861+0000 front 2024-03-07T07:44:57.071845+0000 (oldest deadline 2024-03-07T07:45:21.171805+0000) 2024-03-07T07:45:49.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:48 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:48.856+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:08.173327+0000 front 2024-03-07T07:45:08.173344+0000 (oldest deadline 2024-03-07T07:45:29.273385+0000) 2024-03-07T07:45:49.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:48 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:48.856+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6822 osd.6 since back 2024-03-07T07:45:10.974782+0000 front 2024-03-07T07:45:10.974879+0000 (oldest deadline 2024-03-07T07:45:39.175172+0000) 2024-03-07T07:45:49.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:48 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:48.856+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6820 osd.7 since back 2024-03-07T07:45:22.676192+0000 front 2024-03-07T07:45:22.676259+0000 (oldest deadline 2024-03-07T07:45:44.976158+0000) 2024-03-07T07:45:49.356 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:49 smithi177 podman[70793]: 2024-03-07 07:45:49.039052108 +0000 UTC m=+0.622830700 container remove 85c773932687ff948d815fdbb658c374f36e3acb0cd49aa41d9d7b6d4eb39b64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-8b896a9, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-07T07:45:49.838 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:49 smithi177 podman[70884]: 2024-03-07 07:45:49.373146674 +0000 UTC m=+0.105220952 container create 5116b95124ffae94021b4fa06505366d17e163c1f8fee95cb5d4dd69c0b732ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, RELEASE=wip-lusov-qdb-8b896a9, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-07T07:45:49.838 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:49 smithi177 podman[70884]: 2024-03-07 07:45:49.285814249 +0000 UTC m=+0.017888543 image pull e362cd7cebfee51077fb90b7f01895811140c9a4272edd83ad9910b05e3a46f6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2 2024-03-07T07:45:49.838 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:49 smithi177 podman[70884]: 2024-03-07 07:45:49.550540608 +0000 UTC m=+0.282614891 container init 5116b95124ffae94021b4fa06505366d17e163c1f8fee95cb5d4dd69c0b732ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-07T07:45:49.838 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:49 smithi177 podman[70884]: 2024-03-07 07:45:49.553577434 +0000 UTC m=+0.285651714 container start 5116b95124ffae94021b4fa06505366d17e163c1f8fee95cb5d4dd69c0b732ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-8b896a9) 2024-03-07T07:45:49.838 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:49 smithi177 podman[70884]: 2024-03-07 07:45:49.616969853 +0000 UTC m=+0.349044124 container attach 5116b95124ffae94021b4fa06505366d17e163c1f8fee95cb5d4dd69c0b732ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-8b896a9) 2024-03-07T07:45:50.106 INFO:journalctl@ceph.osd.10.smithi177.stdout:Mar 07 07:45:49 smithi177 podman[70884]: 2024-03-07 07:45:49.83868489 +0000 UTC m=+0.570759183 container died 5116b95124ffae94021b4fa06505366d17e163c1f8fee95cb5d4dd69c0b732ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-10-deactivate, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-07T07:45:50.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:49 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:49.868+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:50.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:49 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:49.868+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:50.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:49 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:49.868+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:42.970262+0000 front 2024-03-07T07:44:42.970339+0000 (oldest deadline 2024-03-07T07:45:08.870388+0000) 2024-03-07T07:45:50.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:49 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:49.868+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:57.071861+0000 front 2024-03-07T07:44:57.071845+0000 (oldest deadline 2024-03-07T07:45:21.171805+0000) 2024-03-07T07:45:50.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:49 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:49.868+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:08.173327+0000 front 2024-03-07T07:45:08.173344+0000 (oldest deadline 2024-03-07T07:45:29.273385+0000) 2024-03-07T07:45:50.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:49 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:49.868+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6822 osd.6 since back 2024-03-07T07:45:10.974782+0000 front 2024-03-07T07:45:10.974879+0000 (oldest deadline 2024-03-07T07:45:39.175172+0000) 2024-03-07T07:45:50.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:49 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:49.868+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6820 osd.7 since back 2024-03-07T07:45:22.676192+0000 front 2024-03-07T07:45:22.676259+0000 (oldest deadline 2024-03-07T07:45:44.976158+0000) 2024-03-07T07:45:50.536 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@osd.10.service' 2024-03-07T07:45:50.579 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-07T07:45:50.579 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-03-07T07:45:50.579 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-03-07T07:45:50.579 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-826f6582-dc54-11ee-95c6-87774f69a715@osd.11 2024-03-07T07:45:50.859 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:50 smithi178 systemd[1]: Stopping Ceph osd.11 for 826f6582-dc54-11ee-95c6-87774f69a715... 2024-03-07T07:45:51.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:50 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:50.858+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:51.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:50 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:50.858+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:51.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:50 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:50.858+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:42.970262+0000 front 2024-03-07T07:44:42.970339+0000 (oldest deadline 2024-03-07T07:45:08.870388+0000) 2024-03-07T07:45:51.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:50 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:50.858+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:57.071861+0000 front 2024-03-07T07:44:57.071845+0000 (oldest deadline 2024-03-07T07:45:21.171805+0000) 2024-03-07T07:45:51.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:50 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:50.858+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:08.173327+0000 front 2024-03-07T07:45:08.173344+0000 (oldest deadline 2024-03-07T07:45:29.273385+0000) 2024-03-07T07:45:51.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:50 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:50.858+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6822 osd.6 since back 2024-03-07T07:45:10.974782+0000 front 2024-03-07T07:45:10.974879+0000 (oldest deadline 2024-03-07T07:45:39.175172+0000) 2024-03-07T07:45:51.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:50 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:50.858+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6820 osd.7 since back 2024-03-07T07:45:22.676192+0000 front 2024-03-07T07:45:22.676259+0000 (oldest deadline 2024-03-07T07:45:44.976158+0000) 2024-03-07T07:45:51.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:50 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:50.879+0000 7f2e57348700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-07T07:45:51.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:50 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:50.879+0000 7f2e57348700 -1 osd.11 90 *** Got signal Terminated *** 2024-03-07T07:45:51.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:50 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:50.879+0000 7f2e57348700 -1 osd.11 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-07T07:45:52.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:51 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:51.848+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:52.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:51 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:51.848+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:52.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:51 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:51.848+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:42.970262+0000 front 2024-03-07T07:44:42.970339+0000 (oldest deadline 2024-03-07T07:45:08.870388+0000) 2024-03-07T07:45:52.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:51 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:51.848+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:57.071861+0000 front 2024-03-07T07:44:57.071845+0000 (oldest deadline 2024-03-07T07:45:21.171805+0000) 2024-03-07T07:45:52.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:51 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:51.848+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:08.173327+0000 front 2024-03-07T07:45:08.173344+0000 (oldest deadline 2024-03-07T07:45:29.273385+0000) 2024-03-07T07:45:52.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:51 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:51.848+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6822 osd.6 since back 2024-03-07T07:45:10.974782+0000 front 2024-03-07T07:45:10.974879+0000 (oldest deadline 2024-03-07T07:45:39.175172+0000) 2024-03-07T07:45:52.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:51 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:51.848+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6820 osd.7 since back 2024-03-07T07:45:22.676192+0000 front 2024-03-07T07:45:22.676259+0000 (oldest deadline 2024-03-07T07:45:44.976158+0000) 2024-03-07T07:45:53.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:52 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:52.849+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:53.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:52 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:52.849+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:53.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:52 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:52.849+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:42.970262+0000 front 2024-03-07T07:44:42.970339+0000 (oldest deadline 2024-03-07T07:45:08.870388+0000) 2024-03-07T07:45:53.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:52 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:52.849+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:57.071861+0000 front 2024-03-07T07:44:57.071845+0000 (oldest deadline 2024-03-07T07:45:21.171805+0000) 2024-03-07T07:45:53.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:52 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:52.849+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:08.173327+0000 front 2024-03-07T07:45:08.173344+0000 (oldest deadline 2024-03-07T07:45:29.273385+0000) 2024-03-07T07:45:53.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:52 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:52.849+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6822 osd.6 since back 2024-03-07T07:45:10.974782+0000 front 2024-03-07T07:45:10.974879+0000 (oldest deadline 2024-03-07T07:45:39.175172+0000) 2024-03-07T07:45:53.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:52 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:52.849+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6820 osd.7 since back 2024-03-07T07:45:22.676192+0000 front 2024-03-07T07:45:22.676259+0000 (oldest deadline 2024-03-07T07:45:44.976158+0000) 2024-03-07T07:45:54.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:53 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:53.814+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:54.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:53 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:53.814+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:54.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:53 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:53.814+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:42.970262+0000 front 2024-03-07T07:44:42.970339+0000 (oldest deadline 2024-03-07T07:45:08.870388+0000) 2024-03-07T07:45:54.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:53 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:53.814+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:57.071861+0000 front 2024-03-07T07:44:57.071845+0000 (oldest deadline 2024-03-07T07:45:21.171805+0000) 2024-03-07T07:45:54.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:53 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:53.814+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:08.173327+0000 front 2024-03-07T07:45:08.173344+0000 (oldest deadline 2024-03-07T07:45:29.273385+0000) 2024-03-07T07:45:54.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:53 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:53.814+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6822 osd.6 since back 2024-03-07T07:45:10.974782+0000 front 2024-03-07T07:45:10.974879+0000 (oldest deadline 2024-03-07T07:45:39.175172+0000) 2024-03-07T07:45:54.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:53 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:53.814+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6820 osd.7 since back 2024-03-07T07:45:22.676192+0000 front 2024-03-07T07:45:22.676259+0000 (oldest deadline 2024-03-07T07:45:44.976158+0000) 2024-03-07T07:45:55.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:54 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:54.841+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:55.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:54 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:54.841+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:55.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:54 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:54.841+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:42.970262+0000 front 2024-03-07T07:44:42.970339+0000 (oldest deadline 2024-03-07T07:45:08.870388+0000) 2024-03-07T07:45:55.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:54 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:54.841+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:57.071861+0000 front 2024-03-07T07:44:57.071845+0000 (oldest deadline 2024-03-07T07:45:21.171805+0000) 2024-03-07T07:45:55.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:54 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:54.841+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:08.173327+0000 front 2024-03-07T07:45:08.173344+0000 (oldest deadline 2024-03-07T07:45:29.273385+0000) 2024-03-07T07:45:55.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:54 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:54.841+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6822 osd.6 since back 2024-03-07T07:45:10.974782+0000 front 2024-03-07T07:45:10.974879+0000 (oldest deadline 2024-03-07T07:45:39.175172+0000) 2024-03-07T07:45:55.307 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:54 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:54.841+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6820 osd.7 since back 2024-03-07T07:45:22.676192+0000 front 2024-03-07T07:45:22.676259+0000 (oldest deadline 2024-03-07T07:45:44.976158+0000) 2024-03-07T07:45:56.221 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:55 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:55.823+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6806 osd.0 since back 2024-03-07T07:44:26.067556+0000 front 2024-03-07T07:44:26.067530+0000 (oldest deadline 2024-03-07T07:44:51.967647+0000) 2024-03-07T07:45:56.221 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:55 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:55.823+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-03-07T07:44:33.068973+0000 front 2024-03-07T07:44:33.068843+0000 (oldest deadline 2024-03-07T07:44:57.768787+0000) 2024-03-07T07:45:56.221 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:55 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:55.823+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-03-07T07:44:42.970262+0000 front 2024-03-07T07:44:42.970339+0000 (oldest deadline 2024-03-07T07:45:08.870388+0000) 2024-03-07T07:45:56.221 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:55 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:55.823+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-03-07T07:44:57.071861+0000 front 2024-03-07T07:44:57.071845+0000 (oldest deadline 2024-03-07T07:45:21.171805+0000) 2024-03-07T07:45:56.221 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:55 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:55.823+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-03-07T07:45:08.173327+0000 front 2024-03-07T07:45:08.173344+0000 (oldest deadline 2024-03-07T07:45:29.273385+0000) 2024-03-07T07:45:56.221 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:55 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:55.823+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.160:6822 osd.6 since back 2024-03-07T07:45:10.974782+0000 front 2024-03-07T07:45:10.974879+0000 (oldest deadline 2024-03-07T07:45:39.175172+0000) 2024-03-07T07:45:56.221 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:55 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:55.823+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.177:6820 osd.7 since back 2024-03-07T07:45:22.676192+0000 front 2024-03-07T07:45:22.676259+0000 (oldest deadline 2024-03-07T07:45:44.976158+0000) 2024-03-07T07:45:56.221 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:55 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11[62825]: 2024-03-07T07:45:55.823+0000 7f2e4f934700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.178:6820 osd.8 since back 2024-03-07T07:45:30.077932+0000 front 2024-03-07T07:45:30.077967+0000 (oldest deadline 2024-03-07T07:45:55.378094+0000) 2024-03-07T07:45:56.496 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:56 smithi178 podman[72541]: 2024-03-07 07:45:56.220573322 +0000 UTC m=+5.449733793 container died 809c2f9a667190e6fd5e7b1020397830a9c8cddd7a0ba65837b294178b5419cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-07T07:45:56.806 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:56 smithi178 podman[72541]: 2024-03-07 07:45:56.595696743 +0000 UTC m=+5.824857192 container cleanup 809c2f9a667190e6fd5e7b1020397830a9c8cddd7a0ba65837b294178b5419cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-lusov-qdb-8b896a9, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222) 2024-03-07T07:45:56.806 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:56 smithi178 bash[72541]: ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11 2024-03-07T07:45:57.208 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:56 smithi178 podman[72570]: 2024-03-07 07:45:56.86365122 +0000 UTC m=+0.636661536 container remove 809c2f9a667190e6fd5e7b1020397830a9c8cddd7a0ba65837b294178b5419cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-07T07:45:57.542 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:57 smithi178 podman[72662]: 2024-03-07 07:45:57.112418265 +0000 UTC m=+0.017671481 image pull e362cd7cebfee51077fb90b7f01895811140c9a4272edd83ad9910b05e3a46f6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2 2024-03-07T07:45:57.542 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:57 smithi178 podman[72662]: 2024-03-07 07:45:57.223456265 +0000 UTC m=+0.128709494 container create 0857b0c3448eb5b393bb618ce94654fbe4dce11596875dbfd5f95e6baf488af0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-8b896a9, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.vendor=CentOS) 2024-03-07T07:45:57.542 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:57 smithi178 podman[72662]: 2024-03-07 07:45:57.442531987 +0000 UTC m=+0.347785202 container init 0857b0c3448eb5b393bb618ce94654fbe4dce11596875dbfd5f95e6baf488af0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-07T07:45:57.542 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:57 smithi178 podman[72662]: 2024-03-07 07:45:57.445542026 +0000 UTC m=+0.350795243 container start 0857b0c3448eb5b393bb618ce94654fbe4dce11596875dbfd5f95e6baf488af0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b) 2024-03-07T07:45:57.542 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:57 smithi178 podman[72662]: 2024-03-07 07:45:57.542635024 +0000 UTC m=+0.447888240 container attach 0857b0c3448eb5b393bb618ce94654fbe4dce11596875dbfd5f95e6baf488af0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11-deactivate, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-8b896a9) 2024-03-07T07:45:57.806 INFO:journalctl@ceph.osd.11.smithi178.stdout:Mar 07 07:45:57 smithi178 podman[72662]: 2024-03-07 07:45:57.734549174 +0000 UTC m=+0.639802375 container died 0857b0c3448eb5b393bb618ce94654fbe4dce11596875dbfd5f95e6baf488af0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-osd-11-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, GIT_CLEAN=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-07T07:45:58.320 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@osd.11.service' 2024-03-07T07:45:58.361 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-07T07:45:58.361 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-03-07T07:45:58.362 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-03-07T07:45:58.362 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-826f6582-dc54-11ee-95c6-87774f69a715@mds.a 2024-03-07T07:45:58.642 INFO:journalctl@ceph.mds.a.smithi160.stdout:Mar 07 07:45:58 smithi160 systemd[1]: Stopping Ceph mds.a for 826f6582-dc54-11ee-95c6-87774f69a715... 2024-03-07T07:45:58.927 INFO:journalctl@ceph.mds.a.smithi160.stdout:Mar 07 07:45:58 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-a[74023]: 2024-03-07T07:45:58.641+0000 7f0c12b2d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-07T07:45:58.927 INFO:journalctl@ceph.mds.a.smithi160.stdout:Mar 07 07:45:58 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-a[74023]: 2024-03-07T07:45:58.641+0000 7f0c12b2d700 -1 mds.a *** got signal Terminated *** 2024-03-07T07:46:01.178 INFO:journalctl@ceph.mds.a.smithi160.stdout:Mar 07 07:46:00 smithi160 podman[88488]: 2024-03-07 07:46:00.900714443 +0000 UTC m=+2.343732988 container died 9cb5a59bf35606c23c679224999a00de63828cc8824dd66f6e42d5b97bad698e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-a, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-8b896a9, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-07T07:46:01.626 INFO:journalctl@ceph.mds.a.smithi160.stdout:Mar 07 07:46:01 smithi160 podman[88488]: 2024-03-07 07:46:01.408344908 +0000 UTC m=+2.851363456 container cleanup 9cb5a59bf35606c23c679224999a00de63828cc8824dd66f6e42d5b97bad698e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-a, org.label-schema.build-date=20240222, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-8b896a9, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-07T07:46:01.626 INFO:journalctl@ceph.mds.a.smithi160.stdout:Mar 07 07:46:01 smithi160 bash[88488]: ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-a 2024-03-07T07:46:01.750 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@mds.a.service' 2024-03-07T07:46:01.788 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-07T07:46:01.788 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-03-07T07:46:01.788 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-03-07T07:46:01.789 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-826f6582-dc54-11ee-95c6-87774f69a715@mds.d 2024-03-07T07:46:02.177 INFO:journalctl@ceph.mds.d.smithi160.stdout:Mar 07 07:46:01 smithi160 systemd[1]: Stopping Ceph mds.d for 826f6582-dc54-11ee-95c6-87774f69a715... 2024-03-07T07:46:02.177 INFO:journalctl@ceph.mds.d.smithi160.stdout:Mar 07 07:46:02 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-d[73071]: 2024-03-07T07:46:02.103+0000 7f05e1a64700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-07T07:46:02.178 INFO:journalctl@ceph.mds.d.smithi160.stdout:Mar 07 07:46:02 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-d[73071]: 2024-03-07T07:46:02.103+0000 7f05e1a64700 -1 mds.d *** got signal Terminated *** 2024-03-07T07:46:05.479 INFO:journalctl@ceph.mds.d.smithi160.stdout:Mar 07 07:46:05 smithi160 podman[88611]: 2024-03-07 07:46:05.217196196 +0000 UTC m=+3.205315632 container died b876bcd750c29b84c7ed3821f7f7245fe229d4da46e7189bcbc6d085b91ee911 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-d, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-07T07:46:05.837 INFO:journalctl@ceph.mds.d.smithi160.stdout:Mar 07 07:46:05 smithi160 podman[88611]: 2024-03-07 07:46:05.578573617 +0000 UTC m=+3.566693054 container cleanup b876bcd750c29b84c7ed3821f7f7245fe229d4da46e7189bcbc6d085b91ee911 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-07T07:46:05.837 INFO:journalctl@ceph.mds.d.smithi160.stdout:Mar 07 07:46:05 smithi160 bash[88611]: ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-d 2024-03-07T07:46:05.999 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@mds.d.service' 2024-03-07T07:46:06.035 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-07T07:46:06.035 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-03-07T07:46:06.035 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-03-07T07:46:06.035 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-826f6582-dc54-11ee-95c6-87774f69a715@mds.g 2024-03-07T07:46:06.427 INFO:journalctl@ceph.mds.g.smithi160.stdout:Mar 07 07:46:06 smithi160 systemd[1]: Stopping Ceph mds.g for 826f6582-dc54-11ee-95c6-87774f69a715... 2024-03-07T07:46:06.428 INFO:journalctl@ceph.mds.g.smithi160.stdout:Mar 07 07:46:06 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-g[73663]: 2024-03-07T07:46:06.330+0000 7f209fe2e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-07T07:46:06.428 INFO:journalctl@ceph.mds.g.smithi160.stdout:Mar 07 07:46:06 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-g[73663]: 2024-03-07T07:46:06.330+0000 7f209fe2e700 -1 mds.g *** got signal Terminated *** 2024-03-07T07:46:10.540 INFO:journalctl@ceph.mds.g.smithi160.stdout:Mar 07 07:46:10 smithi160 podman[88730]: 2024-03-07 07:46:10.195282531 +0000 UTC m=+3.943447418 container died ad2e3e0c37fbe51a3f2852fa654451b9096687bf80d080ba311fb9213a614d22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-g, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-8b896a9, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b) 2024-03-07T07:46:10.899 INFO:journalctl@ceph.mds.g.smithi160.stdout:Mar 07 07:46:10 smithi160 podman[88730]: 2024-03-07 07:46:10.681876841 +0000 UTC m=+4.430041736 container cleanup ad2e3e0c37fbe51a3f2852fa654451b9096687bf80d080ba311fb9213a614d22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-g, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b) 2024-03-07T07:46:10.899 INFO:journalctl@ceph.mds.g.smithi160.stdout:Mar 07 07:46:10 smithi160 bash[88730]: ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-g 2024-03-07T07:46:11.032 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@mds.g.service' 2024-03-07T07:46:11.066 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-07T07:46:11.066 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-03-07T07:46:11.066 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-03-07T07:46:11.066 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-826f6582-dc54-11ee-95c6-87774f69a715@mds.j 2024-03-07T07:46:11.177 INFO:journalctl@ceph.mds.j.smithi160.stdout:Mar 07 07:46:11 smithi160 systemd[1]: Stopping Ceph mds.j for 826f6582-dc54-11ee-95c6-87774f69a715... 2024-03-07T07:46:11.677 INFO:journalctl@ceph.mds.j.smithi160.stdout:Mar 07 07:46:11 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-j[72721]: 2024-03-07T07:46:11.368+0000 7fa37ace0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-07T07:46:11.678 INFO:journalctl@ceph.mds.j.smithi160.stdout:Mar 07 07:46:11 smithi160 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-j[72721]: 2024-03-07T07:46:11.368+0000 7fa37ace0700 -1 mds.j *** got signal Terminated *** 2024-03-07T07:46:14.644 INFO:journalctl@ceph.mds.j.smithi160.stdout:Mar 07 07:46:14 smithi160 podman[88851]: 2024-03-07 07:46:14.348671153 +0000 UTC m=+3.070716333 container died 219cb8715400277ef18909512abbd20ce27453cda0ff0ab1d498d9491875bf7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-j, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b) 2024-03-07T07:46:14.927 INFO:journalctl@ceph.mds.j.smithi160.stdout:Mar 07 07:46:14 smithi160 podman[88851]: 2024-03-07 07:46:14.702265994 +0000 UTC m=+3.424311181 container cleanup 219cb8715400277ef18909512abbd20ce27453cda0ff0ab1d498d9491875bf7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-j, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD) 2024-03-07T07:46:14.928 INFO:journalctl@ceph.mds.j.smithi160.stdout:Mar 07 07:46:14 smithi160 bash[88851]: ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-j 2024-03-07T07:46:15.125 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@mds.j.service' 2024-03-07T07:46:15.159 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-07T07:46:15.159 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-03-07T07:46:15.160 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-03-07T07:46:15.160 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-826f6582-dc54-11ee-95c6-87774f69a715@mds.b 2024-03-07T07:46:15.522 INFO:journalctl@ceph.mds.b.smithi177.stdout:Mar 07 07:46:15 smithi177 systemd[1]: Stopping Ceph mds.b for 826f6582-dc54-11ee-95c6-87774f69a715... 2024-03-07T07:46:15.857 INFO:journalctl@ceph.mds.b.smithi177.stdout:Mar 07 07:46:15 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-b[67020]: 2024-03-07T07:46:15.521+0000 7ffb8a02b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-07T07:46:15.857 INFO:journalctl@ceph.mds.b.smithi177.stdout:Mar 07 07:46:15 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-b[67020]: 2024-03-07T07:46:15.521+0000 7ffb8a02b700 -1 mds.b *** got signal Terminated *** 2024-03-07T07:46:19.829 INFO:journalctl@ceph.mds.b.smithi177.stdout:Mar 07 07:46:19 smithi177 podman[71002]: 2024-03-07 07:46:19.526586142 +0000 UTC m=+4.175895479 container died 5121d6ee5cd475884facacac9b89ca5cac2b84ae3958b456cdb35eea85a2599b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-8b896a9, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-07T07:46:19.829 INFO:journalctl@ceph.mds.b.smithi177.stdout:Mar 07 07:46:19 smithi177 podman[71002]: 2024-03-07 07:46:19.829386998 +0000 UTC m=+4.478696335 container cleanup 5121d6ee5cd475884facacac9b89ca5cac2b84ae3958b456cdb35eea85a2599b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-b, RELEASE=wip-lusov-qdb-8b896a9, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b) 2024-03-07T07:46:20.106 INFO:journalctl@ceph.mds.b.smithi177.stdout:Mar 07 07:46:19 smithi177 bash[71002]: ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-b 2024-03-07T07:46:20.107 INFO:journalctl@ceph.mds.b.smithi177.stdout:Mar 07 07:46:20 smithi177 podman[71016]: 2024-03-07 07:46:20.063798177 +0000 UTC m=+0.527560251 container remove 5121d6ee5cd475884facacac9b89ca5cac2b84ae3958b456cdb35eea85a2599b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-b, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-8b896a9, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-07T07:46:20.201 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@mds.b.service' 2024-03-07T07:46:20.238 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-07T07:46:20.239 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-03-07T07:46:20.239 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-03-07T07:46:20.239 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-826f6582-dc54-11ee-95c6-87774f69a715@mds.e 2024-03-07T07:46:20.566 INFO:journalctl@ceph.mds.e.smithi177.stdout:Mar 07 07:46:20 smithi177 systemd[1]: Stopping Ceph mds.e for 826f6582-dc54-11ee-95c6-87774f69a715... 2024-03-07T07:46:20.857 INFO:journalctl@ceph.mds.e.smithi177.stdout:Mar 07 07:46:20 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-e[66391]: 2024-03-07T07:46:20.565+0000 7f5a32981700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-07T07:46:20.857 INFO:journalctl@ceph.mds.e.smithi177.stdout:Mar 07 07:46:20 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-e[66391]: 2024-03-07T07:46:20.565+0000 7f5a32981700 -1 mds.e *** got signal Terminated *** 2024-03-07T07:46:24.357 INFO:journalctl@ceph.mds.e.smithi177.stdout:Mar 07 07:46:24 smithi177 podman[71121]: 2024-03-07 07:46:24.057254136 +0000 UTC m=+3.598567607 container died f8fb2c249ecdc7ccd4ad9c35f83d050e5456abd2edb55de0b22e777d3f3bfe0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-e, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-8b896a9, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-07T07:46:24.774 INFO:journalctl@ceph.mds.e.smithi177.stdout:Mar 07 07:46:24 smithi177 podman[71121]: 2024-03-07 07:46:24.590493859 +0000 UTC m=+4.131807323 container cleanup f8fb2c249ecdc7ccd4ad9c35f83d050e5456abd2edb55de0b22e777d3f3bfe0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-07T07:46:24.774 INFO:journalctl@ceph.mds.e.smithi177.stdout:Mar 07 07:46:24 smithi177 bash[71121]: ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-e 2024-03-07T07:46:24.995 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@mds.e.service' 2024-03-07T07:46:25.032 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-07T07:46:25.032 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-03-07T07:46:25.032 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-03-07T07:46:25.032 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-826f6582-dc54-11ee-95c6-87774f69a715@mds.h 2024-03-07T07:46:25.344 INFO:journalctl@ceph.mds.h.smithi177.stdout:Mar 07 07:46:25 smithi177 systemd[1]: Stopping Ceph mds.h for 826f6582-dc54-11ee-95c6-87774f69a715... 2024-03-07T07:46:25.607 INFO:journalctl@ceph.mds.h.smithi177.stdout:Mar 07 07:46:25 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-h[66705]: 2024-03-07T07:46:25.343+0000 7fea342cf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-07T07:46:25.607 INFO:journalctl@ceph.mds.h.smithi177.stdout:Mar 07 07:46:25 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-h[66705]: 2024-03-07T07:46:25.343+0000 7fea342cf700 -1 mds.h *** got signal Terminated *** 2024-03-07T07:46:28.327 INFO:journalctl@ceph.mds.h.smithi177.stdout:Mar 07 07:46:28 smithi177 podman[71254]: 2024-03-07 07:46:28.069255929 +0000 UTC m=+2.823319755 container died 41cf76184494cc9ec4064b6b59fd0441e86f8e43bcafb45c0677b07ff04a8396 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-h, GIT_BRANCH=HEAD, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-8b896a9, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0) 2024-03-07T07:46:28.607 INFO:journalctl@ceph.mds.h.smithi177.stdout:Mar 07 07:46:28 smithi177 podman[71254]: 2024-03-07 07:46:28.401995522 +0000 UTC m=+3.156059339 container cleanup 41cf76184494cc9ec4064b6b59fd0441e86f8e43bcafb45c0677b07ff04a8396 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-h, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-8b896a9, maintainer=Guillaume Abrioux , GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-07T07:46:28.607 INFO:journalctl@ceph.mds.h.smithi177.stdout:Mar 07 07:46:28 smithi177 bash[71254]: ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-h 2024-03-07T07:46:28.607 INFO:journalctl@ceph.mds.h.smithi177.stdout:Mar 07 07:46:28 smithi177 podman[71268]: 2024-03-07 07:46:28.578091004 +0000 UTC m=+0.506243145 container remove 41cf76184494cc9ec4064b6b59fd0441e86f8e43bcafb45c0677b07ff04a8396 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-h, ceph=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-07T07:46:28.817 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@mds.h.service' 2024-03-07T07:46:28.852 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-07T07:46:28.852 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-03-07T07:46:28.852 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-03-07T07:46:28.853 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-826f6582-dc54-11ee-95c6-87774f69a715@mds.k 2024-03-07T07:46:29.106 INFO:journalctl@ceph.mds.k.smithi177.stdout:Mar 07 07:46:28 smithi177 systemd[1]: Stopping Ceph mds.k for 826f6582-dc54-11ee-95c6-87774f69a715... 2024-03-07T07:46:29.607 INFO:journalctl@ceph.mds.k.smithi177.stdout:Mar 07 07:46:29 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-k[66070]: 2024-03-07T07:46:29.154+0000 7f647d9d2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-07T07:46:29.607 INFO:journalctl@ceph.mds.k.smithi177.stdout:Mar 07 07:46:29 smithi177 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-k[66070]: 2024-03-07T07:46:29.155+0000 7f647d9d2700 -1 mds.k *** got signal Terminated *** 2024-03-07T07:46:31.762 INFO:journalctl@ceph.mds.k.smithi177.stdout:Mar 07 07:46:31 smithi177 podman[71374]: 2024-03-07 07:46:31.441114963 +0000 UTC m=+2.376401544 container died 3a9e5ab2868658995fc4b33fc24267e794d7e50c334174c482a6e5b87d72ea41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.license=GPLv2) 2024-03-07T07:46:32.106 INFO:journalctl@ceph.mds.k.smithi177.stdout:Mar 07 07:46:31 smithi177 podman[71374]: 2024-03-07 07:46:31.76329702 +0000 UTC m=+2.698583598 container cleanup 3a9e5ab2868658995fc4b33fc24267e794d7e50c334174c482a6e5b87d72ea41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-k, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-8b896a9, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2) 2024-03-07T07:46:32.107 INFO:journalctl@ceph.mds.k.smithi177.stdout:Mar 07 07:46:31 smithi177 bash[71374]: ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-k 2024-03-07T07:46:32.107 INFO:journalctl@ceph.mds.k.smithi177.stdout:Mar 07 07:46:31 smithi177 podman[71388]: 2024-03-07 07:46:31.972499247 +0000 UTC m=+0.530118073 container remove 3a9e5ab2868658995fc4b33fc24267e794d7e50c334174c482a6e5b87d72ea41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-k, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, RELEASE=wip-lusov-qdb-8b896a9, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-03-07T07:46:32.194 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@mds.k.service' 2024-03-07T07:46:32.228 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-07T07:46:32.228 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-03-07T07:46:32.228 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-03-07T07:46:32.229 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-826f6582-dc54-11ee-95c6-87774f69a715@mds.c 2024-03-07T07:46:32.521 INFO:journalctl@ceph.mds.c.smithi178.stdout:Mar 07 07:46:32 smithi178 systemd[1]: Stopping Ceph mds.c for 826f6582-dc54-11ee-95c6-87774f69a715... 2024-03-07T07:46:32.786 INFO:journalctl@ceph.mds.c.smithi178.stdout:Mar 07 07:46:32 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-c[68313]: 2024-03-07T07:46:32.521+0000 7fd501192700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-07T07:46:32.786 INFO:journalctl@ceph.mds.c.smithi178.stdout:Mar 07 07:46:32 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-c[68313]: 2024-03-07T07:46:32.521+0000 7fd501192700 -1 mds.c *** got signal Terminated *** 2024-03-07T07:46:36.339 INFO:journalctl@ceph.mds.c.smithi178.stdout:Mar 07 07:46:36 smithi178 podman[72766]: 2024-03-07 07:46:36.080074191 +0000 UTC m=+3.660112277 container died 51b264020bfc092c759c090ec6363167f33438a587cd19137f22c6993b218c32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-8b896a9, CEPH_POINT_RELEASE=, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.license=GPLv2) 2024-03-07T07:46:36.714 INFO:journalctl@ceph.mds.c.smithi178.stdout:Mar 07 07:46:36 smithi178 podman[72766]: 2024-03-07 07:46:36.4389017 +0000 UTC m=+4.018939763 container cleanup 51b264020bfc092c759c090ec6363167f33438a587cd19137f22c6993b218c32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-c, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240222, RELEASE=wip-lusov-qdb-8b896a9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-07T07:46:36.715 INFO:journalctl@ceph.mds.c.smithi178.stdout:Mar 07 07:46:36 smithi178 bash[72766]: ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-c 2024-03-07T07:46:36.859 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@mds.c.service' 2024-03-07T07:46:36.897 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-07T07:46:36.898 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-03-07T07:46:36.898 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-03-07T07:46:36.898 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-826f6582-dc54-11ee-95c6-87774f69a715@mds.f 2024-03-07T07:46:36.979 INFO:journalctl@ceph.mds.f.smithi178.stdout:Mar 07 07:46:36 smithi178 systemd[1]: Stopping Ceph mds.f for 826f6582-dc54-11ee-95c6-87774f69a715... 2024-03-07T07:46:37.306 INFO:journalctl@ceph.mds.f.smithi178.stdout:Mar 07 07:46:37 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-f[68003]: 2024-03-07T07:46:37.209+0000 7ff2fd009700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-07T07:46:37.306 INFO:journalctl@ceph.mds.f.smithi178.stdout:Mar 07 07:46:37 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-f[68003]: 2024-03-07T07:46:37.209+0000 7ff2fd009700 -1 mds.f *** got signal Terminated *** 2024-03-07T07:46:40.326 INFO:journalctl@ceph.mds.f.smithi178.stdout:Mar 07 07:46:40 smithi178 podman[72899]: 2024-03-07 07:46:40.075326165 +0000 UTC m=+2.960559841 container died 4cedda18d060a877173b5dd4528dd97f9baa6fdaeae1d2910f6e663604e0cd34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-f, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=) 2024-03-07T07:46:40.601 INFO:journalctl@ceph.mds.f.smithi178.stdout:Mar 07 07:46:40 smithi178 podman[72899]: 2024-03-07 07:46:40.417484795 +0000 UTC m=+3.302718472 container cleanup 4cedda18d060a877173b5dd4528dd97f9baa6fdaeae1d2910f6e663604e0cd34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-f, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-8b896a9) 2024-03-07T07:46:40.602 INFO:journalctl@ceph.mds.f.smithi178.stdout:Mar 07 07:46:40 smithi178 bash[72899]: ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-f 2024-03-07T07:46:40.836 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@mds.f.service' 2024-03-07T07:46:40.901 INFO:journalctl@ceph.mds.f.smithi178.stdout:Mar 07 07:46:40 smithi178 podman[72914]: 2024-03-07 07:46:40.602062721 +0000 UTC m=+0.520046541 container remove 4cedda18d060a877173b5dd4528dd97f9baa6fdaeae1d2910f6e663604e0cd34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-f, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-8b896a9, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-03-07T07:46:40.901 INFO:journalctl@ceph.mds.f.smithi178.stdout:Mar 07 07:46:40 smithi178 systemd[1]: ceph-826f6582-dc54-11ee-95c6-87774f69a715@mds.f.service: Deactivated successfully. 2024-03-07T07:46:40.901 INFO:journalctl@ceph.mds.f.smithi178.stdout:Mar 07 07:46:40 smithi178 systemd[1]: Stopped Ceph mds.f for 826f6582-dc54-11ee-95c6-87774f69a715. 2024-03-07T07:46:40.901 INFO:journalctl@ceph.mds.f.smithi178.stdout:Mar 07 07:46:40 smithi178 systemd[1]: ceph-826f6582-dc54-11ee-95c6-87774f69a715@mds.f.service: Consumed 1.305s CPU time. 2024-03-07T07:46:40.912 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-07T07:46:40.912 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-03-07T07:46:40.913 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-03-07T07:46:40.913 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-826f6582-dc54-11ee-95c6-87774f69a715@mds.i 2024-03-07T07:46:41.237 INFO:journalctl@ceph.mds.i.smithi178.stdout:Mar 07 07:46:40 smithi178 systemd[1]: Stopping Ceph mds.i for 826f6582-dc54-11ee-95c6-87774f69a715... 2024-03-07T07:46:41.556 INFO:journalctl@ceph.mds.i.smithi178.stdout:Mar 07 07:46:41 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-i[68618]: 2024-03-07T07:46:41.237+0000 7f950f8f8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-07T07:46:41.556 INFO:journalctl@ceph.mds.i.smithi178.stdout:Mar 07 07:46:41 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-i[68618]: 2024-03-07T07:46:41.237+0000 7f950f8f8700 -1 mds.i *** got signal Terminated *** 2024-03-07T07:46:44.329 INFO:journalctl@ceph.mds.i.smithi178.stdout:Mar 07 07:46:44 smithi178 podman[73019]: 2024-03-07 07:46:44.072804585 +0000 UTC m=+2.948431337 container died cc48c1537dafd0a324d5024673cdd99b77589c1df9fa14c8a900ef818a05642e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-8b896a9) 2024-03-07T07:46:44.722 INFO:journalctl@ceph.mds.i.smithi178.stdout:Mar 07 07:46:44 smithi178 podman[73019]: 2024-03-07 07:46:44.429527589 +0000 UTC m=+3.305154337 container cleanup cc48c1537dafd0a324d5024673cdd99b77589c1df9fa14c8a900ef818a05642e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-i, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-8b896a9, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-07T07:46:44.722 INFO:journalctl@ceph.mds.i.smithi178.stdout:Mar 07 07:46:44 smithi178 bash[73019]: ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-i 2024-03-07T07:46:44.859 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@mds.i.service' 2024-03-07T07:46:44.893 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-07T07:46:44.893 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-03-07T07:46:44.893 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-03-07T07:46:44.893 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-826f6582-dc54-11ee-95c6-87774f69a715@mds.l 2024-03-07T07:46:45.306 INFO:journalctl@ceph.mds.l.smithi178.stdout:Mar 07 07:46:44 smithi178 systemd[1]: Stopping Ceph mds.l for 826f6582-dc54-11ee-95c6-87774f69a715... 2024-03-07T07:46:45.306 INFO:journalctl@ceph.mds.l.smithi178.stdout:Mar 07 07:46:45 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-l[67685]: 2024-03-07T07:46:45.216+0000 7f3ed12d5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-07T07:46:45.306 INFO:journalctl@ceph.mds.l.smithi178.stdout:Mar 07 07:46:45 smithi178 ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-l[67685]: 2024-03-07T07:46:45.216+0000 7f3ed12d5700 -1 mds.l *** got signal Terminated *** 2024-03-07T07:46:48.367 INFO:journalctl@ceph.mds.l.smithi178.stdout:Mar 07 07:46:48 smithi178 podman[73141]: 2024-03-07 07:46:48.075129983 +0000 UTC m=+2.964254041 container died 0ca9fa5a8119f99bb0a18a465590b46c2948341877837da0d3e94aa096f5f4f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-l, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-07T07:46:48.367 INFO:journalctl@ceph.mds.l.smithi178.stdout:Mar 07 07:46:48 smithi178 podman[73141]: 2024-03-07 07:46:48.367363835 +0000 UTC m=+3.256487849 container cleanup 0ca9fa5a8119f99bb0a18a465590b46c2948341877837da0d3e94aa096f5f4f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-l, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-8b896a9, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True) 2024-03-07T07:46:48.716 INFO:journalctl@ceph.mds.l.smithi178.stdout:Mar 07 07:46:48 smithi178 bash[73141]: ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-l 2024-03-07T07:46:48.717 INFO:journalctl@ceph.mds.l.smithi178.stdout:Mar 07 07:46:48 smithi178 podman[73155]: 2024-03-07 07:46:48.601818115 +0000 UTC m=+0.518960652 container remove 0ca9fa5a8119f99bb0a18a465590b46c2948341877837da0d3e94aa096f5f4f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:58596b386710a6ae3b0df85c2ea61cd2225482bf755ea6fac190290483e010c2, name=ceph-826f6582-dc54-11ee-95c6-87774f69a715-mds-l, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-8b896a9, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-07T07:46:48.730 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-826f6582-dc54-11ee-95c6-87774f69a715@mds.l.service' 2024-03-07T07:46:48.763 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-07T07:46:48.763 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-03-07T07:46:48.764 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 826f6582-dc54-11ee-95c6-87774f69a715 --force --keep-logs 2024-03-07T07:46:48.954 INFO:teuthology.orchestra.run.smithi160.stdout:Deleting cluster with fsid: 826f6582-dc54-11ee-95c6-87774f69a715 2024-03-07T07:46:51.478 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 826f6582-dc54-11ee-95c6-87774f69a715 --force --keep-logs 2024-03-07T07:46:51.670 INFO:teuthology.orchestra.run.smithi177.stdout:Deleting cluster with fsid: 826f6582-dc54-11ee-95c6-87774f69a715 2024-03-07T07:46:54.149 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 826f6582-dc54-11ee-95c6-87774f69a715 --force --keep-logs 2024-03-07T07:46:54.339 INFO:teuthology.orchestra.run.smithi178.stdout:Deleting cluster with fsid: 826f6582-dc54-11ee-95c6-87774f69a715 2024-03-07T07:46:56.732 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-07T07:46:56.767 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-07T07:46:56.802 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-07T07:46:56.831 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-07T07:46:56.835 DEBUG:teuthology.misc:Transferring archived files from smithi160:/var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/crash to /home/teuthworker/archive/leonidus-2024-03-07_05:46:12-fs-wip-lusov-qdb-distro-default-smithi/7583972/remote/smithi160/crash 2024-03-07T07:46:56.837 DEBUG:teuthology.orchestra.run.smithi160:> sudo tar c -f - -C /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/crash -- . 2024-03-07T07:46:56.876 INFO:teuthology.orchestra.run.smithi160.stderr:tar: /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/crash: Cannot open: No such file or directory 2024-03-07T07:46:56.876 INFO:teuthology.orchestra.run.smithi160.stderr:tar: Error is not recoverable: exiting now 2024-03-07T07:46:56.878 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/crash to /home/teuthworker/archive/leonidus-2024-03-07_05:46:12-fs-wip-lusov-qdb-distro-default-smithi/7583972/remote/smithi177/crash 2024-03-07T07:46:56.879 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar c -f - -C /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/crash -- . 2024-03-07T07:46:56.919 INFO:teuthology.orchestra.run.smithi177.stderr:tar: /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/crash: Cannot open: No such file or directory 2024-03-07T07:46:56.919 INFO:teuthology.orchestra.run.smithi177.stderr:tar: Error is not recoverable: exiting now 2024-03-07T07:46:56.921 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/crash to /home/teuthworker/archive/leonidus-2024-03-07_05:46:12-fs-wip-lusov-qdb-distro-default-smithi/7583972/remote/smithi178/crash 2024-03-07T07:46:56.922 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/crash -- . 2024-03-07T07:46:56.959 INFO:teuthology.orchestra.run.smithi178.stderr:tar: /var/lib/ceph/826f6582-dc54-11ee-95c6-87774f69a715/crash: Cannot open: No such file or directory 2024-03-07T07:46:56.960 INFO:teuthology.orchestra.run.smithi178.stderr:tar: Error is not recoverable: exiting now 2024-03-07T07:46:56.961 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-07T07:46:56.961 DEBUG:teuthology.orchestra.run.smithi160:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-03-07T07:46:56.996 INFO:teuthology.orchestra.run.smithi160.stdout:2024-03-07T07:33:07.090913+0000 mon.a (mon.0) 191 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-07T07:46:56.996 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-03-07T07:46:56.996 DEBUG:teuthology.orchestra.run.smithi160:> sudo egrep '\[SEC\]' /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-03-07T07:46:57.070 DEBUG:teuthology.orchestra.run.smithi160:> sudo egrep '\[ERR\]' /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-03-07T07:46:57.142 DEBUG:teuthology.orchestra.run.smithi160:> sudo egrep '\[WRN\]' /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-03-07T07:46:57.213 INFO:teuthology.orchestra.run.smithi160.stdout:2024-03-07T07:33:07.090913+0000 mon.a (mon.0) 191 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-07T07:46:57.213 INFO:tasks.cephadm:Compressing logs... 2024-03-07T07:46:57.213 DEBUG:teuthology.orchestra.run.smithi160:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-07T07:46:57.256 DEBUG:teuthology.orchestra.run.smithi177:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-07T07:46:57.258 DEBUG:teuthology.orchestra.run.smithi178:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-07T07:46:57.280 INFO:teuthology.orchestra.run.smithi160.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-07T07:46:57.281 INFO:teuthology.orchestra.run.smithi160.stderr:: No such file or directory 2024-03-07T07:46:57.281 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-osd.6.log 2024-03-07T07:46:57.281 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose/var/log/ceph/cephadm.log: -- /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mon.a.log 2024-03-07T07:46:57.281 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph.audit.log 2024-03-07T07:46:57.281 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-osd.6.log: /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mds.g.log 2024-03-07T07:46:57.283 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mds.j.log 2024-03-07T07:46:57.283 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-osd.3.log 2024-03-07T07:46:57.283 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mgr.x.log 2024-03-07T07:46:57.283 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-osd.0.log 2024-03-07T07:46:57.285 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mgr.x.log: 90.1% -- replaced with /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph.audit.log.gz 2024-03-07T07:46:57.285 INFO:teuthology.orchestra.run.smithi160.stderr: 94.7% -- replaced with /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mds.g.log.gz 2024-03-07T07:46:57.285 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph.cephadm.log 2024-03-07T07:46:57.285 INFO:teuthology.orchestra.run.smithi160.stderr: 94.6%/var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-osd.0.log: -- replaced with /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mds.j.log.gzgzip -5 --verbose -- /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph.log 2024-03-07T07:46:57.285 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-03-07T07:46:57.285 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph.cephadm.log: 83.1% -- replaced with /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph.cephadm.log.gz 2024-03-07T07:46:57.287 INFO:teuthology.orchestra.run.smithi178.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log: No such file or directory 2024-03-07T07:46:57.287 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-03-07T07:46:57.287 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mds.l.log 2024-03-07T07:46:57.287 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph.audit.log 2024-03-07T07:46:57.287 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose --/var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mds.l.log: /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mds.i.log 2024-03-07T07:46:57.287 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-osd.11.log 2024-03-07T07:46:57.295 INFO:teuthology.orchestra.run.smithi160.stderr: 93.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-07T07:46:57.295 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-volume.log 2024-03-07T07:46:57.296 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-osd.9.log 2024-03-07T07:46:57.296 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mds.d.log 2024-03-07T07:46:57.296 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-osd.9.log: 86.2% -- replaced with /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph.log.gz 2024-03-07T07:46:57.296 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mds.a.log 2024-03-07T07:46:57.296 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mds.d.log: 94.6% -- replaced with /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mds.d.log.gz 2024-03-07T07:46:57.300 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mds.a.log: 94.7% -- replaced with /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mds.a.log.gz 2024-03-07T07:46:57.303 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mgr.z.log 2024-03-07T07:46:57.303 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mds.i.log: /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-osd.11.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-07T07:46:57.303 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-osd.8.log 2024-03-07T07:46:57.303 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-osd.5.log 2024-03-07T07:46:57.304 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mds.f.log 2024-03-07T07:46:57.304 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph.cephadm.log 2024-03-07T07:46:57.304 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mds.f.log: 90.3% -- replaced with /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph.audit.log.gz 2024-03-07T07:46:57.304 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph.log 2024-03-07T07:46:57.304 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph.cephadm.log: 82.9% -- replaced with /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph.cephadm.log.gz 2024-03-07T07:46:57.304 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-osd.2.log 2024-03-07T07:46:57.304 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph.log: 86.2% -- replaced with /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph.log.gz 2024-03-07T07:46:57.304 INFO:teuthology.orchestra.run.smithi178.stderr: 92.2% -- replaced with /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mgr.z.log.gz 2024-03-07T07:46:57.304 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mds.c.log 2024-03-07T07:46:57.304 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-volume.log 2024-03-07T07:46:57.305 INFO:teuthology.orchestra.run.smithi177.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-07T07:46:57.305 INFO:teuthology.orchestra.run.smithi177.stderr:: No such file or directory 2024-03-07T07:46:57.305 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-osd.10.log 2024-03-07T07:46:57.305 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph.audit.log 2024-03-07T07:46:57.306 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mds.b.log 2024-03-07T07:46:57.306 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-osd.4.log 2024-03-07T07:46:57.306 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-osd.1.log 2024-03-07T07:46:57.306 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mds.b.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-07T07:46:57.306 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mds.k.log 2024-03-07T07:46:57.306 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mds.h.log 2024-03-07T07:46:57.306 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-osd.1.log: /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-osd.7.log 2024-03-07T07:46:57.306 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mds.e.log 2024-03-07T07:46:57.306 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-osd.7.log: 90.2% -- replaced with /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph.audit.log.gz 2024-03-07T07:46:57.306 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mgr.y.log 2024-03-07T07:46:57.306 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mds.e.log: 94.7% -- replaced with /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mds.k.log.gz 2024-03-07T07:46:57.306 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph.cephadm.log 2024-03-07T07:46:57.306 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mgr.y.log: 94.8% -- replaced with /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mds.e.log.gz 2024-03-07T07:46:57.307 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph.log 2024-03-07T07:46:57.307 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph.cephadm.log: 83.1% -- replaced with /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph.cephadm.log.gz 2024-03-07T07:46:57.307 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-volume.log 2024-03-07T07:46:57.307 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph.log: 86.2% -- replaced with /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph.log.gz 2024-03-07T07:46:57.307 INFO:teuthology.orchestra.run.smithi177.stderr: 92.1%gzip -5 --verbose -- /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mon.b.log 2024-03-07T07:46:57.307 INFO:teuthology.orchestra.run.smithi177.stderr: -- replaced with /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mgr.y.log.gz 2024-03-07T07:46:57.307 INFO:teuthology.orchestra.run.smithi160.stderr: 91.0% -- replaced with /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-volume.log.gz 2024-03-07T07:46:57.310 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mon.c.log 2024-03-07T07:46:57.313 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-volume.log: /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mon.b.log: 90.9% -- replaced with /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-volume.log.gz 2024-03-07T07:46:57.322 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-volume.log: /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mon.c.log: 94.0% -- replaced with /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mds.l.log.gz 2024-03-07T07:46:57.324 INFO:teuthology.orchestra.run.smithi178.stderr: 91.0% -- replaced with /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-volume.log.gz 2024-03-07T07:46:57.327 INFO:teuthology.orchestra.run.smithi177.stderr: 94.1% -- replaced with /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mds.h.log.gz 2024-03-07T07:46:57.331 INFO:teuthology.orchestra.run.smithi177.stderr: 92.6% -- replaced with /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mds.b.log.gz 2024-03-07T07:46:57.348 INFO:teuthology.orchestra.run.smithi178.stderr: 92.5% -- replaced with /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mds.c.log.gz 2024-03-07T07:46:57.351 INFO:teuthology.orchestra.run.smithi178.stderr: 93.5% -- replaced with /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mds.f.log.gz 2024-03-07T07:46:57.472 INFO:teuthology.orchestra.run.smithi160.stderr: 89.6% -- replaced with /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mgr.x.log.gz 2024-03-07T07:46:57.484 INFO:teuthology.orchestra.run.smithi177.stderr: 91.9% -- replaced with /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mon.b.log.gz 2024-03-07T07:46:57.553 INFO:teuthology.orchestra.run.smithi178.stderr: 92.7% -- replaced with /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mds.i.log.gz 2024-03-07T07:46:57.593 INFO:teuthology.orchestra.run.smithi160.stderr: 92.6% -- replaced with /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-osd.9.log.gz 2024-03-07T07:46:57.605 INFO:teuthology.orchestra.run.smithi178.stderr: 91.9% -- replaced with /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mon.c.log.gz 2024-03-07T07:46:57.649 INFO:teuthology.orchestra.run.smithi178.stderr: 92.7% -- replaced with /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-osd.8.log.gz 2024-03-07T07:46:57.790 INFO:teuthology.orchestra.run.smithi160.stderr: 92.7% -- replaced with /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-osd.0.log.gz 2024-03-07T07:46:57.803 INFO:teuthology.orchestra.run.smithi178.stderr: 92.7% -- replaced with /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-osd.5.log.gz 2024-03-07T07:46:57.814 INFO:teuthology.orchestra.run.smithi177.stderr: 92.4% -- replaced with /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-osd.1.log.gz 2024-03-07T07:46:57.827 INFO:teuthology.orchestra.run.smithi177.stderr: 92.4% -- replaced with /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-osd.4.log.gz 2024-03-07T07:46:57.832 INFO:teuthology.orchestra.run.smithi160.stderr: 92.7% -- replaced with /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-osd.3.log.gz 2024-03-07T07:46:57.841 INFO:teuthology.orchestra.run.smithi160.stderr: 90.6% -- replaced with /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-mon.a.log.gz 2024-03-07T07:46:57.848 INFO:teuthology.orchestra.run.smithi177.stderr: 92.4% -- replaced with /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-osd.10.log.gz 2024-03-07T07:46:58.144 INFO:teuthology.orchestra.run.smithi178.stderr: 93.6% -- replaced with /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-osd.11.log.gz 2024-03-07T07:46:58.197 INFO:teuthology.orchestra.run.smithi177.stderr: 93.3% -- replaced with /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-osd.7.log.gz 2024-03-07T07:46:58.199 INFO:teuthology.orchestra.run.smithi177.stderr: 2024-03-07T07:46:58.199 INFO:teuthology.orchestra.run.smithi177.stderr:real 0m0.926s 2024-03-07T07:46:58.199 INFO:teuthology.orchestra.run.smithi177.stderr:user 0m2.737s 2024-03-07T07:46:58.199 INFO:teuthology.orchestra.run.smithi177.stderr:sys 0m0.125s 2024-03-07T07:46:58.207 INFO:teuthology.orchestra.run.smithi160.stderr: 93.2% -- replaced with /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-osd.6.log.gz 2024-03-07T07:46:58.209 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-03-07T07:46:58.209 INFO:teuthology.orchestra.run.smithi160.stderr:real 0m0.942s 2024-03-07T07:46:58.210 INFO:teuthology.orchestra.run.smithi160.stderr:user 0m2.978s 2024-03-07T07:46:58.210 INFO:teuthology.orchestra.run.smithi160.stderr:sys 0m0.128s 2024-03-07T07:46:58.255 INFO:teuthology.orchestra.run.smithi178.stderr: 93.3% -- replaced with /var/log/ceph/826f6582-dc54-11ee-95c6-87774f69a715/ceph-osd.2.log.gz 2024-03-07T07:46:58.257 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-03-07T07:46:58.257 INFO:teuthology.orchestra.run.smithi178.stderr:real 0m0.985s 2024-03-07T07:46:58.257 INFO:teuthology.orchestra.run.smithi178.stderr:user 0m3.291s 2024-03-07T07:46:58.258 INFO:teuthology.orchestra.run.smithi178.stderr:sys 0m0.151s 2024-03-07T07:46:58.258 INFO:tasks.cephadm:Archiving logs... 2024-03-07T07:46:58.258 DEBUG:teuthology.misc:Transferring archived files from smithi160:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-03-07_05:46:12-fs-wip-lusov-qdb-distro-default-smithi/7583972/remote/smithi160/log 2024-03-07T07:46:58.259 DEBUG:teuthology.orchestra.run.smithi160:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-07T07:46:58.755 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-03-07_05:46:12-fs-wip-lusov-qdb-distro-default-smithi/7583972/remote/smithi177/log 2024-03-07T07:46:58.755 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-07T07:46:59.181 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-03-07_05:46:12-fs-wip-lusov-qdb-distro-default-smithi/7583972/remote/smithi178/log 2024-03-07T07:46:59.182 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-07T07:46:59.666 INFO:tasks.cephadm:Removing cluster... 2024-03-07T07:46:59.667 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 826f6582-dc54-11ee-95c6-87774f69a715 --force 2024-03-07T07:46:59.858 INFO:teuthology.orchestra.run.smithi160.stdout:Deleting cluster with fsid: 826f6582-dc54-11ee-95c6-87774f69a715 2024-03-07T07:47:00.170 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 826f6582-dc54-11ee-95c6-87774f69a715 --force 2024-03-07T07:47:00.363 INFO:teuthology.orchestra.run.smithi177.stdout:Deleting cluster with fsid: 826f6582-dc54-11ee-95c6-87774f69a715 2024-03-07T07:47:00.619 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 826f6582-dc54-11ee-95c6-87774f69a715 --force 2024-03-07T07:47:00.810 INFO:teuthology.orchestra.run.smithi178.stdout:Deleting cluster with fsid: 826f6582-dc54-11ee-95c6-87774f69a715 2024-03-07T07:47:01.064 INFO:tasks.cephadm:Removing cephadm ... 2024-03-07T07:47:01.064 DEBUG:teuthology.orchestra.run.smithi160:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-07T07:47:01.080 DEBUG:teuthology.orchestra.run.smithi177:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-07T07:47:01.095 DEBUG:teuthology.orchestra.run.smithi178:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-07T07:47:01.110 INFO:tasks.cephadm:Teardown complete 2024-03-07T07:47:01.111 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-07T07:47:01.123 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-07T07:47:01.123 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-07T07:47:01.125 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-07T07:47:01.138 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-07T07:47:01.249 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-07T07:47:01.259 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-07T07:47:01.260 DEBUG:teuthology.orchestra.run.smithi160:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-07T07:47:01.262 DEBUG:teuthology.orchestra.run.smithi177:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-07T07:47:01.264 DEBUG:teuthology.orchestra.run.smithi178:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-07T07:47:01.274 INFO:teuthology.orchestra.run.smithi160.stderr:bash: line 1: ntpq: command not found 2024-03-07T07:47:01.276 INFO:teuthology.orchestra.run.smithi177.stderr:bash: line 1: ntpq: command not found 2024-03-07T07:47:01.278 INFO:teuthology.orchestra.run.smithi178.stderr:bash: line 1: ntpq: command not found 2024-03-07T07:47:01.278 INFO:teuthology.orchestra.run.smithi160.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-07T07:47:01.279 INFO:teuthology.orchestra.run.smithi160.stdout:=============================================================================== 2024-03-07T07:47:01.279 INFO:teuthology.orchestra.run.smithi160.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 43 +2122us[+2122us] +/- 133ms 2024-03-07T07:47:01.279 INFO:teuthology.orchestra.run.smithi160.stdout:^* hv02.front.sepia.ceph.com 3 6 377 44 -711us[ -711us] +/- 54ms 2024-03-07T07:47:01.279 INFO:teuthology.orchestra.run.smithi160.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 108 -119us[ -119us] +/- 65ms 2024-03-07T07:47:01.279 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-03-07T07:47:01.280 INFO:teuthology.orchestra.run.smithi177.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-07T07:47:01.280 INFO:teuthology.orchestra.run.smithi177.stdout:=============================================================================== 2024-03-07T07:47:01.280 INFO:teuthology.orchestra.run.smithi177.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 46 +2123us[+2118us] +/- 133ms 2024-03-07T07:47:01.280 INFO:teuthology.orchestra.run.smithi177.stdout:^* hv02.front.sepia.ceph.com 3 6 377 45 -727us[ -733us] +/- 54ms 2024-03-07T07:47:01.280 INFO:teuthology.orchestra.run.smithi177.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 43 -129us[ -129us] +/- 66ms 2024-03-07T07:47:01.281 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-03-07T07:47:01.281 INFO:teuthology.orchestra.run.smithi178.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-07T07:47:01.282 INFO:teuthology.orchestra.run.smithi178.stdout:=============================================================================== 2024-03-07T07:47:01.282 INFO:teuthology.orchestra.run.smithi178.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 42 +2120us[+2120us] +/- 133ms 2024-03-07T07:47:01.282 INFO:teuthology.orchestra.run.smithi178.stdout:^* hv02.front.sepia.ceph.com 3 6 377 43 -733us[ -763us] +/- 54ms 2024-03-07T07:47:01.282 INFO:teuthology.orchestra.run.smithi178.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 43 -128us[ -128us] +/- 66ms 2024-03-07T07:47:01.282 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-03-07T07:47:01.283 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-07T07:47:01.293 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-07T07:47:01.293 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-07T07:47:01.303 DEBUG:teuthology.orchestra.run.smithi160:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-07T07:47:01.396 DEBUG:teuthology.orchestra.run.smithi177:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-07T07:47:01.466 DEBUG:teuthology.orchestra.run.smithi178:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-07T07:47:01.538 DEBUG:teuthology.orchestra.run.smithi160:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-07T07:47:01.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-07T07:47:01.568 DEBUG:teuthology.orchestra.run.smithi177:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-07T07:47:01.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-07T07:47:01.598 DEBUG:teuthology.orchestra.run.smithi178:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-07T07:47:01.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-07T07:47:01.629 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-07T07:47:01.640 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-07T07:47:01.650 INFO:teuthology.task.internal:Duration was 1434.256311 seconds 2024-03-07T07:47:01.650 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-07T07:47:01.661 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-07T07:47:01.661 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-07T07:47:01.664 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-07T07:47:01.666 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-07T07:47:01.705 INFO:teuthology.orchestra.run.smithi160.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-07T07:47:01.706 INFO:teuthology.orchestra.run.smithi177.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-07T07:47:01.711 INFO:teuthology.orchestra.run.smithi178.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-07T07:47:02.138 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-07T07:47:02.138 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi160.front.sepia.ceph.com 2024-03-07T07:47:02.138 DEBUG:teuthology.orchestra.run.smithi160:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-07T07:47:02.157 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi177.front.sepia.ceph.com 2024-03-07T07:47:02.158 DEBUG:teuthology.orchestra.run.smithi177:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-07T07:47:02.203 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi178.front.sepia.ceph.com 2024-03-07T07:47:02.204 DEBUG:teuthology.orchestra.run.smithi178:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-07T07:47:02.238 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-07T07:47:02.239 DEBUG:teuthology.orchestra.run.smithi160:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-07T07:47:02.241 DEBUG:teuthology.orchestra.run.smithi177:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-07T07:47:02.246 DEBUG:teuthology.orchestra.run.smithi178:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-07T07:47:02.307 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-07T07:47:02.307 DEBUG:teuthology.orchestra.run.smithi160:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-07T07:47:02.310 DEBUG:teuthology.orchestra.run.smithi177:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-07T07:47:02.314 DEBUG:teuthology.orchestra.run.smithi178:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-07T07:47:02.659 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-07T07:47:02.670 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-07T07:47:02.671 DEBUG:teuthology.orchestra.run.smithi160:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-07T07:47:02.697 DEBUG:teuthology.orchestra.run.smithi177:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-07T07:47:02.725 DEBUG:teuthology.orchestra.run.smithi178:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-07T07:47:02.752 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-07T07:47:02.763 DEBUG:teuthology.orchestra.run.smithi160:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-07T07:47:02.766 DEBUG:teuthology.orchestra.run.smithi177:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-07T07:47:02.768 DEBUG:teuthology.orchestra.run.smithi178:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-07T07:47:02.791 INFO:teuthology.orchestra.run.smithi160.stdout:kernel.core_pattern = core 2024-03-07T07:47:02.792 INFO:teuthology.orchestra.run.smithi177.stdout:kernel.core_pattern = core 2024-03-07T07:47:02.817 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern = core 2024-03-07T07:47:02.848 DEBUG:teuthology.orchestra.run.smithi160:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-07T07:47:02.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-07T07:47:02.862 DEBUG:teuthology.orchestra.run.smithi177:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-07T07:47:02.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-07T07:47:02.876 DEBUG:teuthology.orchestra.run.smithi178:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-07T07:47:02.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-07T07:47:02.902 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-07T07:47:02.913 INFO:teuthology.task.internal:Transferring archived files... 2024-03-07T07:47:02.914 DEBUG:teuthology.misc:Transferring archived files from smithi160:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-03-07_05:46:12-fs-wip-lusov-qdb-distro-default-smithi/7583972/remote/smithi160 2024-03-07T07:47:02.914 DEBUG:teuthology.orchestra.run.smithi160:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-07T07:47:02.958 DEBUG:teuthology.misc:Transferring archived files from smithi177:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-03-07_05:46:12-fs-wip-lusov-qdb-distro-default-smithi/7583972/remote/smithi177 2024-03-07T07:47:02.958 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-07T07:47:02.997 DEBUG:teuthology.misc:Transferring archived files from smithi178:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-03-07_05:46:12-fs-wip-lusov-qdb-distro-default-smithi/7583972/remote/smithi178 2024-03-07T07:47:02.998 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-07T07:47:03.036 INFO:teuthology.task.internal:Removing archive directory... 2024-03-07T07:47:03.037 DEBUG:teuthology.orchestra.run.smithi160:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-07T07:47:03.039 DEBUG:teuthology.orchestra.run.smithi177:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-07T07:47:03.041 DEBUG:teuthology.orchestra.run.smithi178:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-07T07:47:03.084 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-07T07:47:03.097 INFO:teuthology.task.internal:Not uploading archives. 2024-03-07T07:47:03.097 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-07T07:47:03.108 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-07T07:47:03.108 DEBUG:teuthology.orchestra.run.smithi160:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-07T07:47:03.110 DEBUG:teuthology.orchestra.run.smithi177:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-07T07:47:03.112 DEBUG:teuthology.orchestra.run.smithi178:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-07T07:47:03.122 INFO:teuthology.orchestra.run.smithi160.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 7 07:47 /home/ubuntu/cephtest 2024-03-07T07:47:03.124 INFO:teuthology.orchestra.run.smithi177.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 7 07:47 /home/ubuntu/cephtest 2024-03-07T07:47:03.139 INFO:teuthology.orchestra.run.smithi178.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 7 07:47 /home/ubuntu/cephtest 2024-03-07T07:47:03.140 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-07T07:47:03.151 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-07T07:47:03.174 INFO:teuthology.nuke:Checking targets against current locks 2024-03-07T07:47:03.205 DEBUG:teuthology.nuke:shortname: smithi160 2024-03-07T07:47:03.206 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-07T07:47:03.234 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi160.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-07_05:46:12-fs-wip-lusov-qdb-distro-default-smithi/7583972', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-07 07:13:03.390007', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEnyQs9uQLXfZEoZ2Tm1Acn3ntgsNJxul0I9jisv5u1TsPGeb7Dy8C3PsCPRw/v5VgTOSA98m6Qu0PVkzhvgwL8='} 2024-03-07T07:47:03.235 DEBUG:teuthology.nuke:shortname: smithi177 2024-03-07T07:47:03.235 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-07T07:47:03.264 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi177.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-07_05:46:12-fs-wip-lusov-qdb-distro-default-smithi/7583972', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-07 07:13:03.392931', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNSrXChjdXzc1UOt8ymKReExe1A2gYLoXHB15wWccOLA0f3kgA5e58U3hDgLQZWVEtC+oR9mEcL2IKWTwfj4L5Q='} 2024-03-07T07:47:03.267 DEBUG:teuthology.nuke:shortname: smithi178 2024-03-07T07:47:03.268 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-07T07:47:03.290 INFO:teuthology.orchestra.console.smithi160:Power off 2024-03-07T07:47:03.290 DEBUG:teuthology.orchestra.console.smithi160:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-07T07:47:03.301 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi178.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-07_05:46:12-fs-wip-lusov-qdb-distro-default-smithi/7583972', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-07 07:13:03.391561', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKaf4hAhG8x96cvNOfFgEuyoN0ZyKUf9N/UJUeUXGqudGR6+w3nylXQxPI8ee2sw2bVpW90zIrChT50Rm5PGoPk='} 2024-03-07T07:47:03.314 DEBUG:teuthology.orchestra.console.smithi160:power off output: Chassis Power Control: Down/Off 2024-03-07T07:47:03.314 DEBUG:teuthology.orchestra.console.smithi160:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-07T07:47:03.335 INFO:teuthology.orchestra.console.smithi177:Power off 2024-03-07T07:47:03.336 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-07T07:47:03.349 DEBUG:teuthology.orchestra.console.smithi160:check power output: Chassis Power is on 2024-03-07T07:47:03.359 DEBUG:teuthology.orchestra.console.smithi177:power off output: Chassis Power Control: Down/Off 2024-03-07T07:47:03.359 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-07T07:47:03.377 INFO:teuthology.orchestra.console.smithi178:Power off 2024-03-07T07:47:03.378 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-07T07:47:03.386 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is on 2024-03-07T07:47:03.400 DEBUG:teuthology.orchestra.console.smithi178:power off output: Chassis Power Control: Down/Off 2024-03-07T07:47:03.403 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-07T07:47:03.432 DEBUG:teuthology.orchestra.console.smithi178:check power output: Chassis Power is on 2024-03-07T07:47:07.350 DEBUG:teuthology.orchestra.console.smithi160:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-07T07:47:07.387 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-07T07:47:07.434 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-07T07:47:07.463 DEBUG:teuthology.orchestra.console.smithi160:check power output: Chassis Power is on 2024-03-07T07:47:07.497 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is on 2024-03-07T07:47:07.544 DEBUG:teuthology.orchestra.console.smithi178:check power output: Chassis Power is on 2024-03-07T07:47:11.468 DEBUG:teuthology.orchestra.console.smithi160:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-07T07:47:11.498 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-07T07:47:11.545 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-07T07:47:11.578 DEBUG:teuthology.orchestra.console.smithi160:check power output: Chassis Power is off 2024-03-07T07:47:11.608 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is off 2024-03-07T07:47:11.655 DEBUG:teuthology.orchestra.console.smithi178:check power output: Chassis Power is off 2024-03-07T07:47:11.679 INFO:teuthology.orchestra.console.smithi160:Power off completed 2024-03-07T07:47:11.708 INFO:teuthology.orchestra.console.smithi177:Power off completed 2024-03-07T07:47:11.756 INFO:teuthology.orchestra.console.smithi178:Power off completed 2024-03-07T07:47:11.861 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/suites/fsync-tester}} duration: 1434.2563111782074 failure_reason: '"2024-03-07T07:33:07.090913+0000 mon.a (mon.0) 191 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log' flavor: default owner: scheduled_leonidus@teuthology success: false 2024-03-07T07:47:11.862 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-07T07:47:11.946 INFO:teuthology.run:FAIL